RedShark Christmas Replay: This is a hot topic, so we're giving you another chance to read this: If we don't do something fast, all of our digital films and videos will disappear in just a few years
You might be forgiven for thinking that digital recordings are robust and can last, effectively, for ever. Storing video and audio as ones and noughts has advantages: you only have to tell the difference between "On" and "Off", rather than measure, accurately, a continuously varying voltage. What could be more robust than a string of pulses that could be represented by acorns or carrots or pits on an optical disk?
Except that it's not like that at all. The "1"s and "0"s are meaningless if we don't understand the way they're arranged. And this is the crux of the problem.
Operating systems exist to protect us
In a sense, operating systems exist to protect us from all of this. We can even select and play digital files now by talking to our computers: "Siri, play some Calvin Harris". That's pretty useful but also incredibly abstract, where "abstract" means "there is an awful lot hidden between our command and the digital content of the file".
Oil paintings last pretty well. So do photographs, and - as long as it's stored properly, film. Not so digital storage media, especially anything that moves or rotates.
For a long time after I bought my first digital SLR camera (a Canon 300D, in 2004) I made sure that all the pictures I wanted to keep were stored on multiple drives. Around that time external USB hard drives had become cheap and easy to use. So I backed up my rapidly growing collection of JPEGs to two separate external drives, leaving a third copy on whatever was my main work computer. Pretty safe, then.
I did this for several years, copying all the media to newer and newer drives, until, last year, when I was moving house, I tried to find a copy of the folder with all those pictures.
The two external drives just flatly refused to work, for different reasons. One was completely dead, and the other wasn't recognised as a drive by my computer. I tried alternative power supplies and cables. I tried plugging the one that did spin up into another computer. Nothing.
And the third copy?
That was on an old Macbook whose 60GB spinning drive had failed as well.
Luckily, I had most of the files also stored in DropBox, a paid-for cloud storage and synchronisation company that seems to be doing OK today but which might not be here at some point in the future.
You might rightly say that all of this was down to carelessness, and it was, but this was all about approximately 30GB of pretty much universally-readable JPEG files.
Several feature film's worth of raw video
Now imagine the sort of issues you might get storing several feature films' worth of raw video, or an entire digital film archive. This is not going to be easy.
Let's look at the problem in some more detail. And a good way to do this is to consider the question "how do you get a number down a wire". Seriously, this is a very abstract thing and it doesn't happen easily. Let's say you want send a 7 down a cable. 7 is a concept. A cable is a physical thing with no visible means of carrying precise numerical values.
It would be a lot easier if you had more than one wire. If you had ten, for example, you could send the numbers one to ten just by designating the wires as representing one to ten, and then applying a voltage to cable number seven.
This would work, but you'd have to have a way of specifying when one number had finished and another had started. And you'd also need an awful lot of wires, although you can reduce that number if you move towards a binary system where cable 1 represents 1s, cable 2 represents 2s, cable 3 represents 4s, and so on.
But digital media needs a a lot of big numbers, sent very quickly. On our "parallel" system described above, there's always a chance that the numbers won't arrive exactly together. The digits on some wires could be confused with the previous or next numbers. This would place a limit on the bandwidth of this arrangement. The longer the cables, the bigger the problem.
Perhaps counter-intuitively, you can achieve more with a single wire (a "serial" system, which, in practice, is almost never just a single wire), because numbers would be very precisely clocked, and there would be no room for confusion as to which number belongs to which chunk of data. You'd need to wrap up the numbers so that you would know whether they belonged to the right or left channel, with audio, or to R, G, B etc with video. You'd need all sorts of complicated and precise schemes to keep track of the numbers.
So that's how it is with signals. But we're concerned with storage in this article. How do you store these signals? These days, we store them as files.
We store signals as files
Now as we've already hinted, it's very easy to fall back into thinking that files are straightforward to get at. They're there in the file explorer on our computer: easy to see. You can even see what type they are: .jpg, .png, .mov etc. What's not immediately apparent is that at the level that we see them, they're incredibly abstract. Think about it like this: I can see in my Apple OS X finder (Apple's file explorer) files on my computer's hard disk (which is actually an SSD) some files on a 4 terabyte raid on my desk, some files in Dropbox, and some others on an SD card from my camera. These are all wildly different devices, in different places - and in the case of Dropbox I have absolutely no idea either about the nature of the storage device (in the cloud) or where the file is. It might be shredded between a hundred different places for all I know. It really doesn't make any difference.
In the case of my files in Dropbox and the 4TB raid, there's a measure of protection, through redundancy if an individual drive fails. If my computer's hard drive gives up, or my SD card fries itself, that's it. No more files.
Note that these are all devices that play nicely with OS X. It's the job of the operating system to slice through all these layers of abstraction and make my files visible as useful things that I can play and understand. But inside the individual disk drives - well, where do you start? How can you find a file if you don't have a computer to plug it into? If you take it apart all you will see is a bunch of metal plates with some vague pattern of magnetism. You won't even begin to be able to make sense of the magnetic variations on the disk.