A (Chaotic) Reffective Update

Things got a little chaotic for a while. Now that the number of active and (until recently) new infections is much lower, any outbreaks (like the recent Tönnies case in Gütersloh) have a dramatic effect on R.

[There was a paper showing how the sensitivity of R increased as it was lower, how small changes had large impacts… but now i cant find it. Will update if i come across it again!]

Makes sense, and didn’t seem to bother anyone. The overall trend in Germany is still positive. Locally there are some issues (Neuköln, Berlin, etc) but it looks mostly under control.

Here in Hamburg, over the last month or so, the average is 20 – 30 cases every 7 days. Not great. Not terrible.

Unscrupulous DM-SMR Shenanigans

At the end of last year i put together a Synology NAS containing Western Digital Red (NAS) drives. My goal was to pull all of the data scattered across multiple aging machines and external drives into one place. All of that worked out just fine.

Earlier this year a “scandal” broke where WD was found to have started shipping DM-SMR drives in part of a lineup where CMR was expected. In most cases this would be invisible to the user. However, in use cases such as NAS, certain operations would degenerate and become stupidly slow.

The original table showed that drives smaller than 8TB were now being shipped as SMR:

Not good – my new drives were 4TB – right in the middle of the bad range. An additional table showed the SKUs of drives which were effected:

Hmm. That is not the SKU that appears on my invoice. The parts supplied are WD40EFRX, perhaps i got lucky? Having pulled the drives from the NAS to check, it seems that i did indeed get lucky! There is a good write-up and extensive benchmark on Serve The Home which compares the performance of WD40EFAX and WD40EFRX labelled drives. Wasn’t looking forward to fighting the good fight with WD over having being mis-sold.

And, that is the point here – for most cases the performance of SMR and CMR drives is indistinguishable, it’s only when you go to rebuild a array, swap out a bad drive, create a hot spare, etc. that you start to have issues. For drives that are explicitly sold for use in NAS this is an unacceptable ‘bait-and-switch’.

It seems likely that WD will be forced to replace the drives that were mis-sold, but the amount of time and effort they have put into playing down their deception is likely to cost them a lot more in the long run.

An R(effective) Update

First time we’ve been above one for a while. The numbers have been rather odd lately – several days of reporting errors. It’s possible that this is a reflection of that, but the change in R for the 7-day average is harder to explain away.

Edit: today R(effective) for Germany went up again, and currently stands at 1.20. Not good news. This has, imo, been on the cards for a while. During the initial phase of the lockdown the decline in active cases (total number of infected minus recovered minus dead) was declining fairly linearly. You can see that clearly here:

(Been too busy / lazy to plot it out myself. The above is from the worldometers site, which is different dataset than RKI, but good enough for the purposes of this discussion.)

You can see that from around second week of May the incline on the graph changes, and starts to flatten out. This indicates that new infections are now being detected at the same rate as people are recovering. In recent days things have been close enough that small errors in reporting have seen the first upticks in active cases since shortly after the peak. Consequently it should come as no surprise that the R-effective number would be around 1. The total number (around 8500) is still declining, but much more slowly.

What really rankles is that had Germany stayed on it’s existing path for just a few more weeks the number of infections would have been down in the hundreds by now, certainly at a level where Track & Trace™ would have been a sustainable strategy.

It’s not obvious what happens now. There is talk of degree of transmission being much lower outside (makes sense just with dispersion) and that warmer weather also helps. Whether this is enough to keep a lid on things until autumn is unclear. All of the cluster cases seem to be churches, or illegal indoor gatherings, which suggests that if people dont congregate indoors… there is a little hope.

Guess we’ll see… fingers-crossed.

A Little More R in R

Couple of things were bothering me:

  • the key wasn’t really a key it was a subtitle
  • the title needs a date
  • data wasn’t getting automatically downloaded
  • column naming was a mess
  • structure / formating of the ggplot was inconsistent

Sunday morning is the obvious time to fix such issues! Below is new plot, with a key, and built from data pulled from RKI.

R3

And here is my updated script:

library(readr)
library(readxl)
library(ggplot2)

download.file("https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Projekte_RKI/Nowcasting_Zahlen.xlsx?__blob=publicationFile", "Nowcasting_Zahlen.xlsx")
nz_data <- read_excel("Nowcasting_Zahlen.xlsx", sheet = "Nowcast_R")
names(nz_data) <- c("date","new","new_under","new_over","new2", "new2_under", "new2_over", "R", "R_under", "R_over", "R7", "R7_under", "R7_over")
g <- ggplot(data = nz_data)
g <- g + geom_line(mapping = aes(x = date, y = R, color = "4 day"))
g <- g + geom_ribbon(mapping = aes(x = date, y = R, ymin = R_under, ymax = R_over), alpha = 0.3)
g <- g + geom_line(mapping = aes(x = date, y = R7, color = "7 day"))
g <- g + geom_ribbon(mapping = aes(x = date, y = R7, ymin = R7_under, ymax = R7_over), alpha = 0.5)
g <- g + ggtitle(label = sprintf("R-effective for Germany (%s)", format(Sys.Date(), format = "%b %d %Y")))
g <- g + ylab("R") + xlab("Date")
g <- g + scale_color_manual(values = c('4 day' = 'firebrick', '7 day' = 'darkblue'))
g <- g + labs(color = 'Average')
g

Now might be a good time to go outside and not think about this!

A Little R in R

As usually happens when learning something new, it gets pointed out that while i might have achieved my goal, there is a far cooler way of doing the same thing! In this case that would be R.

RinR

Now it seems a shame to have got this far into a career without ever having played around in R. The above is pretty close to the simplest plot posssible, but hey, it was fun!

Continue reading

R Numbers

A few days ago there was a post on /r/de showing a nice graph of now famous R number for Germany:

hscehxf6c4y41

Exciting for a couple of reasons: that the data-set was available (i’d been looking for it for a while); the reported number was over 1 … which isn’t good.

The post linked to a .pynb script that had been used to produce the image. The code was obviously Python and somehow related to Jupyter Notebook – time to learn something new!

Since the original script was published the Excel spreadsheet download added a cover sheet, which obviously broke things. Having patched things up a little (and translated the labels to English), here is the updated plot:

plot_2020-05-15

One of the labels has gone missing… oops.

Update: the missing label was important! The above graph is for a new RKI dataset that tracks R on a 7 day average, the original series was too sensitive (see below). For “reasons” that series doesn’t include error estimates for the last data points, which broke calculation done on the final point. Below is a new plot on the 4 day averaged series – now really an update of the original image:

plot_2020-05-15

As you can see the R value for the original chart has been revised down, and the current value remains below 1. The brief bump up was attributed to the infection of slaughter house slave labour, housed in cramped shared dormitories. Come of Germany – be better than that!

Update: weekends are obviously good for tracking down datasets and visualizations! There is a GitLab project running model simulations on the regional data, below is the plot for Hamburg:

hamburg

Having no background in epidemiology (or statistical analysis…) all i can do is accept it as presented, and note that it correlates well with the recent decline in reported new infections. The RKI made an interesting observation on the sensitivity of R the other day, noting that as the number of active cases (infections) fell any new hotspots (such as the slaughterhouse outbreaks) would have a larger impact on the reported number.

Concatenation with ffmpeg

This is has been something that regularly happens. Never remember how to do it…

Create a file that contains each file to be concatenated:

% for i in `ls`; do
echo file \'$i\' >> files.txt
done
jje@wretched cat % cat files.txt
file '1.mp4'
file '2.mp4'
file '3.mp4'

and now let ffmpeg do it’s magic:

% ffmpeg -f concat -safe 0 -i files.txt -c copy out.mp4

This works well for sites that make youtube-dl download files in pieces… and having written this, i’ll no longer need to reference it again.

The State of “Social” (Redux)

This post, back in 2018, mused a little about the general state of social networking, and the failure of Mastadon to gain any traction. Despite the obvious and ongoing twin clusterfucks of Facebook and Twitter, little seems to have changed.

Since 2018 i’ve (not for the first time) deleted my Twitter account, and will not be going back. They don’t seem to have noticed…

Both Facebook and Twitter appear to have doubled down on the strategy of “outrage generates engagement. engagement generates profits”, despite the obvious and one might say catastrophic impact it has had on civil society. Even the mainstream press is driven by the vagaries of tweet storms. Facebook looks (from the outside) to be an high stakes version of the Stanford Prison Experiment being run as a profit centre. WhatsApp and Instagram? Please don’t look behind the curtain!

The group of humans with whom i’ve been luck enough to become acquainted are, to varying degrees: intelligent, socially liberal, technically savvy, literate, politically aware, beautiful, artistic, and, er… lovely. That’s why i try to stay in touch with them despite them being spread out over multiple continents.

Unfortunately keeping in touch is not easy. This is, mea maxima culpa, entirely a problem of my own making. Currently there are sets of friends or individuals spread across a large number of … systems. These range from Email (it should be called GMail at this point), XMPP and SMS to iMessage, Signal, and even Discord. Not including Mastadon as nobody ever even remembers it exists.

A large majority of those people are active on Twitter, Instagram, WhatsApp, others on Facebook. The path of least resistance would definitely be to surrender to the herd, and i completely understand that it’s me that makes all this difficult… and yet, i’m not going back.

Everything has a price. Some aspect of the performative / promotional / social engagement with traditional public social networks makes it worthwhile for my friends to continue that relationship. The balance of power has definitely shifted, but people who profited from the early phase largesse are struggling to move on now the situation has become borderline abusive. The societal damage is spread thin enough, especially for those of us who have the free time to waste worrying about it, that it makes sense to try wait it out. Maybe the balance will shift again? It won’t, at least not back from whence it came!

The idea that a new form of social networking, one that reflects and respects the real life boundaries of our actual friendships, will evolve or emerge from the current mess of highly commercialized exploitation now seems ridiculous to me. Prior transitions (MySpace to Facebook, etc) feel like poor guides to the future as the context in which they happened is entirely different. Sure the kids are gravitating to different venues, away from their parents, but the owners of new venues are that same predatory ad tech creeps, with a bag full of “How do you do, Fellow Kids” memes slung over their shoulder.

[As an aside, i’m really intrigued by the concept of posting being in some way “performative”, the micro-doses of attention coming from posting to something like Twitter has become meaningful enough to be habit forming. Future studies of this phenomenom are going to be a rough read…]

A system like Secure Scuttlebutt seems like it would be a far more “human” venue in which to interact. I still love the idea of staying in touch with a geographically dispersed circle of friends, with the possibility of serendipitous encounters of friends-of-friends, having friends help friends… but not all happening under the Unblinking Algorithmic Eye.

APT-E In the Wild?

Along with all the b&w negatives, and a few colour slides, there were a few strips of colour negative film. At first i couldn’t work out why they would have been taken. They are mostly out of focus, blurred by camera shake… and lets not talk about the weird colour casts!

Having actually scanned them, it eventually dawned on my that they might have been taken by my Grandfather at the behest of my Father, who was probably on the train that loosely features in the shots.

The other clue is that there is a picture of my Grandfathers Jaguar parked up by the side of the road. If there is one thing that i’ve learned from scanning all these old pictures it is that my Grandfather loved photographing his cars. We’ll come back to that!

135-Neg_00.jpg

More APT-E Photographs

More photographs from my Father’s archives. Earlier that the previous shots, maybe 1968 or so. No idea what i’m looking at… but presumably the shop in which the prototype was built, and maybe the parts he’d worked on?

Odd that there is nobody around.