Flickr Local Backup?

Flickr is dead / dying. Yahoo! is owned by someone like CBS… i’m not even sure i want to know. Time to get out. Maybe not completely, but that’s a different story.

My plan is to make a local backup / archive of everything that i’ve uploaded. In theory there is a copy of everything elsewhere, but as previous discussed, my organisational skills leave a little to be desired.

Pick a tool. There is a Python Flickr API kit thing, so presumably someone will have written a backup tool. Checkout GitHub and see what is there. The most recently updated / highly rated choice is Flickrmirrorer. Lets give that a go:

$ git clone --recursive https://github.com/markdoliner/flickrmirrorer.git
Cloning into 'flickrmirrorer'...
remote: Counting objects: 483, done.
remote: Total 483 (delta 0), reused 0 (delta 0), pack-reused 483
Receiving objects: 100% (483/483), 104.01 KiB | 0 bytes/s, done.
Resolving deltas: 100% (225/225), done.
$ cd flickrmirrorer/
$ ./flickrmirrorer.py --help
Traceback (most recent call last):
 File "./flickrmirrorer.py", line 49, in <module>
 import requests
ImportError: No module named requests

Ah, fun… Python libraries. This always ends well. According to the README.md we need:

  • python 2.something or python 3.anything
  • python dateutil
    • Ubuntu: apt-get install python-dateutil
  • python flickrapi library 2.0 or newer.
  • python requests

I’ve not done any Python dev on this machine, and it’s macOS not Linux, so no apt-get here. The first thing to try is installing pip… with easy_install. Sometimes it’s pip, sometimes it’s easy_install. They might well be equivalent… no idea!

$ sudo easy_install pip
...
$ sudo pip install requests
...
$ sudo pip install dateutils
...
$ sudo easy_install flickrapi

Might have just as easily been able to install flickrapi with pip, but they said they wanted easy_install. And now:

$ ./flickrmirrorer.py --help
usage: flickrmirrorer.py [-h] [-v] [-q] [-s] [--ignore-views]
 [--ignore-photos] [--ignore-videos]
 [--delete-unknown]
 destdir

Create a local mirror of your flickr data.

positional arguments:
 destdir the path to where the mirror shall be stored

optional arguments:
 -h, --help show this help message and exit
 -v, --verbose print progress information to stdout
 -q, --quiet print nothing to stdout if the mirror succeeds
 -s, --statistics print transfer-statistics at the end
 --ignore-views do not include views-counter in metadata
 --ignore-photos do not mirror photos
 --ignore-videos do not mirror videos
 --delete-unknown delete unrecognized files in the destination directory.
 Warning: if you choose to ignore photos or videos, they
 will be deleted!

Cool. Now all we need to do is setup a disk to keep all this stuff and kick it off:

$ ./flickrmirrorer.py /Volumes/FlickrDump/backup/
0:121: execution error: "https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read" doesn’t understand the “open location” message. (-1708)

Please authorize Flickr Mirrorer to read your photos, titles, tags, etc.

1. Visit https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read
2. Click "OK, I'LL AUTHORIZE IT"
3. Copy and paste the code here and press 'return'

XYZ-ABC-123

Fetching 33991776973.jpg
Updated metadata for 33991776973.jpg
Fetching 32923862986.jpg
Updated metadata for 32923862986.jpg
Fetching 31996362424.jpg
...

This will presumably take a while. An hour or so in:

$ ls -1 *.jpg | wc -l
 900

Edit: of course it died right after this was posted…

Traceback (most recent call last):
 File "./flickrmirrorer.py", line 812, in <module>
 main()
 File "./flickrmirrorer.py", line 807, in main
 mirrorer.run()
 File "./flickrmirrorer.py", line 195, in run
 self._run_helper()
 File "./flickrmirrorer.py", line 237, in _run_helper
 self._download_all_photos()
 File "./flickrmirrorer.py", line 288, in _download_all_photos
 new_files |= self._download_photo(photo)
 File "./flickrmirrorer.py", line 398, in _download_photo
 photo_datetime = get_photo_datetime(photo)
 File "./flickrmirrorer.py", line 147, in get_photo_datetime
 return dateutil.parser.parse(photo['datetaken'])
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 697, in parse
 return DEFAULTPARSER.parse(timestr, **kwargs)
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 310, in parse
 ret = default.replace(**repl)
ValueError: month must be in 1..12

<sigh>

It seems that some of the images in Flickr has no date taken data at all. Isn’t that fun. Time for a little hackery:

$ git diff flickrmirrorer.py 
diff --git a/flickrmirrorer.py b/flickrmirrorer.py
index ef9aa85..29df9a2 100755
--- a/flickrmirrorer.py
+++ b/flickrmirrorer.py
@@ -144,7 +144,11 @@ def get_photo_datetime(photo):
 datetime.datetime
 """
 if photo['datetakenunknown'] == "0":
- return dateutil.parser.parse(photo['datetaken'])
+ try:
+ return dateutil.parser.parse(photo['datetaken'])
+ except ValueError:
+ # now what?
+ return datetime.datetime(1970,1,1,0,0,0,0)
 
 try:
 parsed = datetime.datetime.strptime(photo['title'], '%Y%m%d_%H%M%S')

Which is to say:

  • wrap the parsing of ‘datetaken’ in a try / catch
  • return the beginning of unix time and move on

Suppose it might have been easier to edit the metadata on flickr… but where is the fun in that!

Advertisements

4 thoughts on “Flickr Local Backup?

  1. And where do I go now. I don’t care about my flickr photos at all. I have a local copy. The flickr photos are just copies from my local stash anyway.

    But where else too? I looked into 500px, but the people there just take utter boring photos. Flickr still has more interesting things going on there.

    Or should I completely stop? Try to setup my own service (sure not). I don’t know.

    • sure i don’t know! the community that made flickr an interesting proposition has mostly faded away, to the extent that it doesn’t really matter into which void you are posting…

      the whole SNS thing seems to be reduced to it original premise: better targeted ads. there is very little incentive for the users to hang around and be monetised. not really clear that we’ll see new internet scale communities like flickr form again.

      perhaps it’s time to something super-curated, like a portfolio site?

  2. Pingback: Just | It'll All End In Tears

Wise words...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s