… No!

Last weekend / week some valiant efforts were made to import a Blogger export file into the latest WordPress release running on my laptop. Just getting that shitshow on the road was not a walk in the park. I’ll leave the next paragraph as a warning to future travellers on macOS:

WordPress will not connect to MySQL / MariaDB on localhost, update your config to use

How PHP manages to be so consistently bad is a source of amazement.

My initial excitement about having found a plugin was tempered by the eventual realisation that the hooks are only there when saving a new entry. Presumably the code could be re-used in a pipeline that uses the Blogger REST API to pull entries, pushing them into WordPress, attaching images and rewriting img tags as it goes.

My hopes of getting this done without writing my own toolchain are slowly fading into the ether…


You do it to yourself, you do
And that’s what really hurts
Is you do it to yourself, just you
You and no-one else
You do it to yourself
You do it to yourself

That’s the thing, if i was better at ignoring things…

Last weekend’s adventure was making a local backup of all my images in Flickr. This was really about getting to a point where it wouldn’t matter if that account got closed, or the new owners (Verizon?) decided to move everything, etc. It was a good start, but there is another (bigger) problem that is going to stop me from moving forward: the IWTFY blog.

There are hundred… i don’t know, maybe thousands of posts there that link directly to the images hosted on Flickr. Oops. Besides that being against the terms of service it rather neatly ties me to keeping the Flickr account for as long as there is an IWTFY blog… and that’s ongoing for a decade at this point!

Obviously a plan is needed!

  1. Export IWTFY from Blogger / Google – this seems straightforward.
  2. Setup a local instance of WordPress where i can run plugins.
  3. Write a plugin that pulls all of the images into the local WordPress instance
  4. Export IWTFY from the local instance
  5. Import into a hosted WordPress

This feels very much like a process that someone else must have already been through, and either made available / sells as a service… right? Proceeding onwards assuming that i’m not able to find that person / service, it looks like it isn’t difficult to use something like  the WP_Http API call to grab images from within a blog post. And somewhere in here there has to be a way loop through all the entries in blog. No idea what kind of WordPress *thing* i’m supposed to be coding, but yeah… Bildung!

Can’t actually believe i’m going to do this. Save me from myself? Please.

[Yes, the tags are something of a honeypot.]

Edit: someone already wrote the plugin i need! Import External Images – even if it doesn’t do exactly what is needed, its only a little PHP hacking… how bad could it be?!

Signal Desktop (Update)

My weekend adventures got me as far as a running Election branch. Something was obviously a little weird as it was opening with a javascript debugger taking up the right third of the app window. Not ideal, but easy to close.

And then i tried to link it to the account on my phone… no dice.

What i’d built was the development version, which defaults to connecting to the test / staging servers. For any of this experiment to be useful it needs to connect to the production servers.

These are all defined in the /config directory, but it wasn’t obvious to me how to change to another config. In the package.json there were references to ${env.SIGNAL_ENV} but exporting that to the shell with a value of ‘production’ didn’t help.

A little reading about my new friend ‘yarn’ showed that there was a ‘–production=true’ CLI option. Passing that along with ‘yarn install’ broke things.. but the command:

$ yarn start --production=true

Moves things along and makes it possible to link to a phone account. The process is actually pretty smooth – you just take a picture of a QRCode, give the client a name and you’re done.

What i’ve still not worked out is how to tell my other new friend ‘electron-packager’ to package the production version. Until that is worked out i’m stuck with the Muon version… which isn’t a bad place – it has been very stable thus far.

Edit: It seems ‘electron-packager’ was not my friend after all, ‘yarn’ however i’m starting to like! The ‘package.json’ contains a ‘scripts’ section defining a bunch of targets, one of which ‘pack-prod’ is ‘SIGNAL_ENV=production npm run dist’ which seems exactly what i need. And, indeed:

$ yarn pack-prod

spat out a ‘dist’ dir containing a working ‘production’ Signal.app.

The end.

Flickr Local Backup?

Flickr is dead / dying. Yahoo! is owned by someone like CBS… i’m not even sure i want to know. Time to get out. Maybe not completely, but that’s a different story.

My plan is to make a local backup / archive of everything that i’ve uploaded. In theory there is a copy of everything elsewhere, but as previous discussed, my organisational skills leave a little to be desired.

Pick a tool. There is a Python Flickr API kit thing, so presumably someone will have written a backup tool. Checkout GitHub and see what is there. The most recently updated / highly rated choice is Flickrmirrorer. Lets give that a go:

$ git clone --recursive https://github.com/markdoliner/flickrmirrorer.git
Cloning into 'flickrmirrorer'...
remote: Counting objects: 483, done.
remote: Total 483 (delta 0), reused 0 (delta 0), pack-reused 483
Receiving objects: 100% (483/483), 104.01 KiB | 0 bytes/s, done.
Resolving deltas: 100% (225/225), done.
$ cd flickrmirrorer/
$ ./flickrmirrorer.py --help
Traceback (most recent call last):
 File "./flickrmirrorer.py", line 49, in <module>
 import requests
ImportError: No module named requests

Ah, fun… Python libraries. This always ends well. According to the README.md we need:

  • python 2.something or python 3.anything
  • python dateutil
    • Ubuntu: apt-get install python-dateutil
  • python flickrapi library 2.0 or newer.
  • python requests

I’ve not done any Python dev on this machine, and it’s macOS not Linux, so no apt-get here. The first thing to try is installing pip… with easy_install. Sometimes it’s pip, sometimes it’s easy_install. They might well be equivalent… no idea!

$ sudo easy_install pip
$ sudo pip install requests
$ sudo pip install dateutils
$ sudo easy_install flickrapi

Might have just as easily been able to install flickrapi with pip, but they said they wanted easy_install. And now:

$ ./flickrmirrorer.py --help
usage: flickrmirrorer.py [-h] [-v] [-q] [-s] [--ignore-views]
 [--ignore-photos] [--ignore-videos]

Create a local mirror of your flickr data.

positional arguments:
 destdir the path to where the mirror shall be stored

optional arguments:
 -h, --help show this help message and exit
 -v, --verbose print progress information to stdout
 -q, --quiet print nothing to stdout if the mirror succeeds
 -s, --statistics print transfer-statistics at the end
 --ignore-views do not include views-counter in metadata
 --ignore-photos do not mirror photos
 --ignore-videos do not mirror videos
 --delete-unknown delete unrecognized files in the destination directory.
 Warning: if you choose to ignore photos or videos, they
 will be deleted!

Cool. Now all we need to do is setup a disk to keep all this stuff and kick it off:

$ ./flickrmirrorer.py /Volumes/FlickrDump/backup/
0:121: execution error: "https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read" doesn’t understand the “open location” message. (-1708)

Please authorize Flickr Mirrorer to read your photos, titles, tags, etc.

1. Visit https://www.flickr.com/services/oauth/authorize?oauth_token=<token>&perms=read
3. Copy and paste the code here and press 'return'


Fetching 33991776973.jpg
Updated metadata for 33991776973.jpg
Fetching 32923862986.jpg
Updated metadata for 32923862986.jpg
Fetching 31996362424.jpg

This will presumably take a while. An hour or so in:

$ ls -1 *.jpg | wc -l

Edit: of course it died right after this was posted…

Traceback (most recent call last):
 File "./flickrmirrorer.py", line 812, in <module>
 File "./flickrmirrorer.py", line 807, in main
 File "./flickrmirrorer.py", line 195, in run
 File "./flickrmirrorer.py", line 237, in _run_helper
 File "./flickrmirrorer.py", line 288, in _download_all_photos
 new_files |= self._download_photo(photo)
 File "./flickrmirrorer.py", line 398, in _download_photo
 photo_datetime = get_photo_datetime(photo)
 File "./flickrmirrorer.py", line 147, in get_photo_datetime
 return dateutil.parser.parse(photo['datetaken'])
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 697, in parse
 return DEFAULTPARSER.parse(timestr, **kwargs)
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py", line 310, in parse
 ret = default.replace(**repl)
ValueError: month must be in 1..12


It seems that some of the images in Flickr has no date taken data at all. Isn’t that fun. Time for a little hackery:

$ git diff flickrmirrorer.py 
diff --git a/flickrmirrorer.py b/flickrmirrorer.py
index ef9aa85..29df9a2 100755
--- a/flickrmirrorer.py
+++ b/flickrmirrorer.py
@@ -144,7 +144,11 @@ def get_photo_datetime(photo):
 if photo['datetakenunknown'] == "0":
- return dateutil.parser.parse(photo['datetaken'])
+ try:
+ return dateutil.parser.parse(photo['datetaken'])
+ except ValueError:
+ # now what?
+ return datetime.datetime(1970,1,1,0,0,0,0)
 parsed = datetime.datetime.strptime(photo['title'], '%Y%m%d_%H%M%S')

Which is to say:

  • wrap the parsing of ‘datetaken’ in a try / catch
  • return the beginning of unix time and move on

Suppose it might have been easier to edit the metadata on flickr… but where is the fun in that!

Signal Desktop (without Chrome)

As unfathomable as i’m sure this is, i’m not a fan of the Chrome browser – don’t think i’ve ever used it. One of those ‘on principle’ things of which i seem to be so fond. Unfortunately the Signal Desktop application runs as a Chrome App… imagine their surprise when Big G declared that it would stop supporting Chrome Apps in the near future. Sorry, getting distracted.

If you want to run a Signal Desktop App there are now options. They most revolve around frameworks that provide a browser platform to applications without actually being a browser, and have names like Electron and Muon.

After a few failed attempts to create a working development environment for Election, and build the official Signal Desktop Electron branch i stumbled upon a Muon branch. This is obviously not at all official, and completely unsupported, but it works really well! If nothing else it will get me by until an official Electron build is produced by Whisper Systems.

My build environment for this was:

  • Xcode (think that’s where my git binaries come from…)
  • macports
    • nodejs7
    • npm4

As usual it was more of a pain to get the environment setup than it was to follow the build instructions. Make sure you install nodejs7 before npm4 otherwise you get stuck with an old version… i should probably move to Brew, etc.

The end result was a OS X Signal.app that paired perfectly with the account on my phone, and has been completely stable / trouble free for the last week. It doesn’t look like the branch is going to be maintained – at least nothing has been merged to it from upstream for nearly two weeks, so this isn’t a long term solution. However, if like me you’re moving more and more of your communications away from mail to signal, it’s a good way to save yourself from typing on a phone!

Edit: i’m an idiot. Building the Electron branch is simple.

$ git clone --recursive https://github.com/WhisperSystems/Signal-Desktop.git
Cloning into 'Signal-Desktop'...
remote: Counting objects: 22939, done.
remote: Compressing objects: 100% (241/241), done.
remote: Total 22939 (delta 333), reused 546 (delta 324), pack-reused 22373
Receiving objects: 100% (22939/22939), 49.29 MiB | 1.04 MiB/s, done.
Resolving deltas: 100% (13362/13362), done.
$ cd Signal-Desktop/
$ git checkout electron
Branch electron set up to track remote branch electron from origin.
Switched to a new branch 'electron'
$ npm install

Now wait for a while… and eventually:

$ npm start

> signal-desktop@1.0.10 start /Users/<redacted>/src/Signal-Desktop
> electron .

Wait for app ready
setting AUMID
reading package.json
NODE_ENV development
NODE_CONFIG_DIR /Users/<redacted>/src/Signal-Desktop/config
HOSTNAME undefined
userData /Users/<redacted>/Library/Application Support/Signal-development
app ready

signal desktop

Edit 2: Sven educated me further, and now i can package an application. This requires another exciting adventure with npm, and installing another tool, the ‘electron-packager‘:

$ npm install electron-packager -g

This let me run:

$ ~/.local/lib/npm/bin/electron-packager .
Packaging app for platform darwin x64 using electron v1.6.7
Wrote new app to /Users/<redacted>/src/Signal-Desktop/Signal-darwin-x64

Which outputs a macOS application bundle ‘Signal.app’. It is a little “heavy”:

$ du -sh Signal.app
156M Signal.app

But that’s the end of the trail. From git to a usable Signal Desktop application… it’s probably time i went outside.