Computers are supposed to make sense of this stuff
But why worry? Isn't that what computers are supposed to do? Make sense of this stuff? Yes, of course they do. Until they don't, that is. At some point in the history of any computer platform and operating system, it will stop supporting external and internal devices, either because the OS no longer supports the type of interface used by the external drive, or because there is no longer any software driver for that type of device. Do you remember the early days of SCSI ("Scuzzy") Drives, when they connected to your computer tower with a cable about as thick as a hosepipe? What could you do with one of those drives now? Nothing. There's nowhere that you can plug one into a modern computer and no-one makes the interface cards any more. What's more, these drives are so old that they're very likely to have seized on their spindles.
I have lost valuable material because storage devices have become obsolete. You have to be a certain age to remember Jazz drives. Made by Iomega, these were great. They were about a hundred times the capacity of a floppy drive, and were in a robust plastic cartridge. You needed a special reader for them but they worked and worked well enough to encourage you to put all your stuff on them. Which I did. And I now have a stack of these things and absolutely no way to access any of the information on them.
This lost data includes some MIDI files which were the result of some musical works that I was commissioned to compose. One of them was a ballet score (I know I don't look like I know anything about ballet, but ask me about it one day...) and there were other works that I spent weeks working on as a jobbing composer. At the time I thought I was being clever by copying all my eight inch and three and a half inch floppy drives onto this newer 100 MB format, but all I was doing was throwing them into an obscure dungeon whose keys would dissolve before my eyes.
Ebay is your friend
I still have a few of these works remaining. As a hissy audio recording on a Compact Cassette. Ironically, since I no longer have a working cassette recorder (Ebay is your friend here) I managed to transfer them to my computer where they're now stored in Dropbox as high bitrate MP3 files.
Absolutely the most important thing to remember here is that this can happen right under your nose without you realising it. It's like they way you forget things. When you lose something from the memory in your head, you don't get a notification saying "you've forgotten the name of that annoying kid in your primary school". All that happens is that the next time you try to remember it, you can't. That's it. That's the first time you find out.
Now, luckily, we can take control of this. We can have backup strategies. But that's clearly not enough. There's no point at all in backing up all your files so that they're stored on accessible error-free media, only to find that you don't have any applications to play them. Remember that this abstraction business keeps going all the way from analogue voltages (or analogue patches of polarised magnetism) up to the arcane way that highly compressed video is stored in files. And while we're here - compression is another thing. To play a compressed file, you have to not only have software that can decode the compression (a codec) but it has to understand the way the compressed file is stored. None of this is simple and it certainly isn't obvious.
I'm not the only one worried about this. I have been for some time. And everyone in the digital content creation business who has any time at all to reflect has these concerns as well.
The "father of the internet"
What prompted this article was that someone who really, really, knows what he's talking about, has just drawn attention to this issue, which doesn't just affect digital media files, but everything that exists in a digital form. It's Vint Cerf, a VP of Google and one of the very few people on the planet who can claim to have been a parent of the internet. Cerf was hugely influential in the development of TCP/IP, so he knows a thing or two about this stuff.
And what he's saying is very scary simply because it's clear what we have to do and even more clear that we don't have the means to do it right now. What has to happen, he's saying, is that we have to not only preserve the files, but the means to decode them as well. Unfortunately it's not as simple as storing the codec along with the file - although that will obviously help.
No, we also have to preserve a working copy of the operating system that can play back the media files, and because machines go out of date, we have to preserve a working copy of the machine. (In case you're wondering how you can "copy" a machine, it's fairly established technology. In fact, much of the World Wide Web is run from virtual machines - emulated computers running on real computers - and this allows it to be much more efficient, if somewhat slower, in individual machine performance terms.)
Once we have a virtual machine that is written in a "portable" language, like Java, perhaps, then it can be transported to run on anything that supports that portable language. This gets us a long way towards being able to resurrect "dead" machine/software combinations.
But it takes work. Lots of it. And because it takes work, to support the virtual machines and make them able to run on whatever is the current and near-future generations of technology, it will cost a lot of money. This, then is the point.
The point is that in future, we will no longer be able to store our creations on a shelf at zero cost. It will require real expenditure for our films and videos to be available in the future. This will not be a small cost. Probably the best we can hope for is that a big company, like Google, perhaps, or Dropbox, or very likely some company that doesn't exist yet, will set up an operation to do what is necessary on a very large scale that will reduce the costs to individual users. This will have to happen.
It will have to happen because if it doesn't, our films, videos, music tracks, personal memories, and in fact the whole of our recent (and future) history, will simply disappear.
Please understand that we are not taking here about data safety as it affects us now, today. We're not talking about RAID systems, which can absolutely be configured to make data incredibly safe. We're not talking about backups and archives, the theory and practise of which are very well known, if not always perfectly executed.
No, what we're talking about is that moment when you realise that you were just - only just - too slow in transferring your content to media that can still be read by today's machines. You don't get a warning when something is about to become obsolete or unreadable. You just get an error message bringing you the bad news, or the device doesn't show up in your file system explorer.
Data doesn't fade away gradually. It just becomes inaccessible. But when you step back and look at a mass of data from afar, the effect is that it gradually goes away. Welcome to the future of digital media.