Flickr Local Backup?

Flickr is dead / dying. Yahoo! is owned by someone like CBS… i’m not even sure i want to know. Time to get out. Maybe not completely, but that’s a different story.

My plan is to make a local backup / archive of everything that i’ve uploaded. In theory there is a copy of everything elsewhere, but as previous discussed, my organisational skills leave a little to be desired.

Pick a tool. There is a Python Flickr API kit thing, so presumably someone will have written a backup tool. Checkout GitHub and see what is there. The most recently updated / highly rated choice is Flickrmirrorer. Lets give that a go:

$ git clone --recursive https://github.com/markdoliner/flickrmirrorer.git
Cloning into 'flickrmirrorer'...
remote: Counting objects: 483, done.
remote: Total 483 (delta 0), reused 0 (delta 0), pack-reused 483
Receiving objects: 100% (483/483), 104.01 KiB | 0 bytes/s, done.
Resolving deltas: 100% (225/225), done.
$ cd flickrmirrorer/
$ ./flickrmirrorer.py --help
Traceback (most recent call last):
 File "./flickrmirrorer.py", line 49, in <module>
 import requests
ImportError: No module named requests

Ah, fun… Python libraries. This always ends well. According to the README.md we need:

  • python 2.something or python 3.anything
  • python dateutil
    • Ubuntu: apt-get install python-dateutil
  • python flickrapi library 2.0 or newer.
  • python requests

I’ve not done any Python dev on this machine, and it’s macOS not Linux, so no apt-get here. The first thing to try is installing pip… with easy_install. Sometimes it’s pip, sometimes it’s easy_install. They might well be equivalent… no idea!

$ sudo easy_install pip
...
$ sudo pip install requests
...
$ sudo pip install dateutils
...
$ sudo easy_install flickrapi

Might have just as easily been able to install flickrapi with pip, but they said they wanted easy_install. And now:

$ ./flickrmirrorer.py --help
usage: flickrmirrorer.py [-h] [-v] [-q] [-s] [--ignore-views]
 [--ignore-photos] [--ignore-videos]
 [--delete-unknown]
 destdir

Create a local mirror of your flickr data.

positional arguments:
 destdir the path to where the mirror shall be stored

optional arguments:
 -h, --help show this help message and exit
 -v, --verbose print progress information to stdout
 -q, --quiet print nothing to stdout if the mirror succeeds
 -s, --statistics print transfer-statistics at the end
 --ignore-views do not include views-counter in metadata
 --ignore-photos do not mirror photos
 --ignore-videos do not mirror videos
 --delete-unknown delete unrecognized files in the destination directory.
 Warning: if you choose to ignore photos or videos, they
 will be deleted!

Cool. Now all we need to do is setup a disk to keep all this stuff and kick it off:

$ ./flickrmirrorer.py /Volumes/FlickrDump/backup/
0:121: execution error: "https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read" doesn’t understand the “open location” message. (-1708)

Please authorize Flickr Mirrorer to read your photos, titles, tags, etc.

1. Visit https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read
2. Click "OK, I'LL AUTHORIZE IT"
3. Copy and paste the code here and press 'return'

XYZ-ABC-123

Fetching 33991776973.jpg
Updated metadata for 33991776973.jpg
Fetching 32923862986.jpg
Updated metadata for 32923862986.jpg
Fetching 31996362424.jpg
...

This will presumably take a while. An hour or so in:

$ ls -1 *.jpg | wc -l
 900

Edit: of course it died right after this was posted…

Traceback (most recent call last):
 File "./flickrmirrorer.py", line 812, in <module>
 main()
 File "./flickrmirrorer.py", line 807, in main
 mirrorer.run()
 File "./flickrmirrorer.py", line 195, in run
 self._run_helper()
 File "./flickrmirrorer.py", line 237, in _run_helper
 self._download_all_photos()
 File "./flickrmirrorer.py", line 288, in _download_all_photos
 new_files |= self._download_photo(photo)
 File "./flickrmirrorer.py", line 398, in _download_photo
 photo_datetime = get_photo_datetime(photo)
 File "./flickrmirrorer.py", line 147, in get_photo_datetime
 return dateutil.parser.parse(photo['datetaken'])
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 697, in parse
 return DEFAULTPARSER.parse(timestr, **kwargs)
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 310, in parse
 ret = default.replace(**repl)
ValueError: month must be in 1..12

<sigh>

It seems that some of the images in Flickr has no date taken data at all. Isn’t that fun. Time for a little hackery:

$ git diff flickrmirrorer.py 
diff --git a/flickrmirrorer.py b/flickrmirrorer.py
index ef9aa85..29df9a2 100755
--- a/flickrmirrorer.py
+++ b/flickrmirrorer.py
@@ -144,7 +144,11 @@ def get_photo_datetime(photo):
 datetime.datetime
 """
 if photo['datetakenunknown'] == "0":
- return dateutil.parser.parse(photo['datetaken'])
+ try:
+ return dateutil.parser.parse(photo['datetaken'])
+ except ValueError:
+ # now what?
+ return datetime.datetime(1970,1,1,0,0,0,0)
 
 try:
 parsed = datetime.datetime.strptime(photo['title'], '%Y%m%d_%H%M%S')

Which is to say:

  • wrap the parsing of ‘datetaken’ in a try / catch
  • return the beginning of unix time and move on

Suppose it might have been easier to edit the metadata on flickr… but where is the fun in that!

Photo Archiving?

I have a problem… yes, yes, i know. This is different.

One of my photography projects (lets call it IWTFY) is quite popular. This means that periodically there are requests for sets of images. When this happens i end up trolling through Lightroom libraries on several machines looking for images, exporting them at an appropriate size / dpi, gathering them into one place, packaging them for distribution, uploading them to be downloaded. Bleugh!

What i need is a place that i can:

  • archive / upload a full-sized processed image
  • tag / label all images
  • search / select subsets
  • define an export size
  • have everything processed into a download bundle

Does such a thing exist? Is it worth paying for? Should i just get my shit together and import everything into a new Lightroom library on one machine?

Help?

Light Boxes

No idea why these advertising light boxes are the way that they are, but they are each end of the S-Bahn platforms at Hamburg Airport. Having walked past them, and occasionally taken the odd digicrap of them, here they are on film. In this case Kodak Portra 800 in the Natura Black.

It turns out that blue lights in a dimly lit underground station are about at close to a torture test as you can get for fast colour negative film. The obviously different light levels really didn’t help with a point and shoot. All of which made them an adventure to process – the white balance still isn’t consistent, but i give up!

Mystery why there are they and never contain any advertising… all other slots are always occupied, with these left open. Maybe they are a token gesture towards making the airport more relaxing / less commercial… or art? Ha!

Book Covers

Tangentially related to the previous post…

25778105.jpg

That’s Iain’s latest book (or maybe it isn’t – a few months have passed…) The cover is a crop of the following photograph of mine, taken with the previously maligned Natura Black 1.9:

9628135464_c719b63a19_z.jpg

Not sure it’s the crop that i’d have made, but i’m notoriously bad at cropping my shots.

It strikes me as somewhat unusual to have covers that are “full bleed” photographs at the moment. Maybe it was never a thing… but certainly the current trend is to more designed and graphical covers. The IWTFY books have eschewed that direction and, to my eye, look all the more distinctive for it.

Both IWTFY covers have also been fairly abstract portraits of M., taken in the bath with a digital P&S… that is a trend which seems difficult to continue.

Sticky Shutter

The olympus xa2XA2 is pretty much my favourite P&S camera. I’ve travelled with one for years. And i mean years – M. had one when we met…

That particular one died a few years ago. The shutter had started to get unreliable and eventually completely unresponsive.

It was a sad day, but a new one was purchased on Yahoo! Auction, and point and shoot life carried on unabated.

This problem with the shutter is apparently common, and the inevitable is now happening to the replacement. It’s hard to get too upset that a camera that was produced in 1980, and has likely seen hundreds of rolls of film, starts to give up the ghost. However, at this point it’s likely that any replacement that i find is quickly going to suffer the same fate.

Time to start to learn to fix it! According to this “tutorial” it should be simply a matter of opening it up, cleaning, tweaking the contacts, and getting it back together. It’s the last part of that which gives me pause… the alternative is to take it to the grumpy camera repair man and have him do it. Less embarrassing than turning up with a bag of pieces.

Oh, and while we’re on the subject of P&S cameras, there is also the Fujifilm Natura S. Another purchase from Yahoo! Auction… and i have to say that i don’t like it. Some people rave about them, but unless you’re *only* shooting 1600 ASA film, it’s a horrible experience. The flash tends to be very eager to fire, even when you’d expect that it wouldn’t, the menu system is junk, settings don’t survive a power cycle, it has no exposure compensation, which leads to “fun” DX coding hacks (scraping paint of film canisters with your keys…), and to top it all off, the focus on mine is really soft in a lot of situations. Despite some reasonable results over the last couple of years, it has never become the replacement that i hoped it would be.