Play your cards right

Play your cards right

Off the top of my head, I can think of quite a lot of different kinds of flash storage device that are used by at least one sort of motion picture recording technology: multi-manufacturer standards such as CompactFlash, SD, SSD, the proprietary types including P2, SxS and AXS (in all their varieties), and expensive device-specific formats such as SR Memory, RED's flash devices, and the packs made by companies such as Codex for their general purpose recorders

There's a fairly wide range of speed and capacity in devices on that list, something that's controlled by both the underlying technology and the way it's organised and interfaced to the outside world.
Even so, this gap has narrowed in the last few years to only a single order of magnitude, even when comparing the most and least capable types which range from a few tens to many hundred megabytes per second. With all this variety, one would think that there would be very little need for the development of new types, given the data rates typical of digital cinematography work. Relatively low-cost consumer solid state disks, capable of rates approaching one gigabyte per second, are capable of storing more or less the entire uncompressed output of any extant digital cinematography camera, and even the venerable CompactFlash has recently been updated (or, if you like, replaced) with the very capable CFast specification.

Benefits to Commoditised Technology

All of these things are intended for and used by both the consumer and professional markets, and benefit from commodity pricing, but there are more reasons than cost to like commoditised technology. For a start, it's available: in an emergency, I can buy a compactflash card or even a 2.5” SSD in a whole lot of places worldwide. What's more, they're all made by a wide variety of manufacturers. Now, it's true that regardless of the name on the packaging, a lot of the underlying flash memory technology is made in Asia in just a few factories, but this is still a better situation than the one that pertained with regard to HDCAM and related tape formats, all of which were made in a factory in eastern Japan that was destroyed in the 2011 tsunami. I've heard the opinion that the resulting shortage of Sony's premier tape format is probably one of the things that served to popularise data recording in the first place, although Sony tell me the format is still a big seller. And finally, as consumer or at the very least widely applied industrial devices, there's an amount of competition going on that encourages healthy competition and plentiful R&D.

Does Device-Specific Technology need to Exist?

So, commodity flash devices are cheap, capable, reliably available and can reasonably be expected to improve. Certainly had this circumstance not arisen, the technology required to create more device-specific technology probably wouldn't exist. The question for me is whether the more device-specific technology needs to exist at all: creating a device that needs storage, and creating a custom flash storage module for it, does nothing but nullify the advantages of commodity hardware.

Let's try to answer the question. In the case of exotic hardware such as Vision Research's Phantom high speed cameras, a high performance flash module is required that must store very large amounts of data, as the camera runs at both high resolution and high speed. In these niche applications, technology such as single-level-cell (SLC) flash becomes necessary, wherein each cell of the flash memory is used to store only one bit. This contrasts with the multi-level-cell (MLC) technology used in many flash devices, including those intended for professional use. In MLC flash chips, each cell, which is an analogue device at heart, is charged to one of several levels, which allows a single cell to represent several digital bits at once. SLC-equipped solid state disks are available, at some price premium, but would still struggle to contain the torrent of data from a high speed camera). Even so, while SLC memory is faster and more reliable, it's only an absolute technical necessity in edge cases where the very, very highest performance is required.

Physical Robustness

 Another commonly-raised issue is that of physical robustness, particularly with regard to the connectors used to allow high speed data buses to be conveniently connected and disconnected as flash devices are filled and replaced. This involves some rather specific engineering challenges, as the data involved is often at quite high speed, and liable to suffer at the hands of a less-than-ideal connector. The thing is, most of this engineering has been done for extant commodity flash card formats. Since Blackmagic released their admirably low-cost Hyperdeck Shuttle recorder (itself pretty sturdy, notwithstanding the power connector), solid-state disks based on the form factor intended for laptop computers have been commonly used to provide storage to video recorders. Blackmagic even made some plastic cases designed to store them, in the same style as Sony's well-known blue tape cases. When this design first became public, I was very concerned about the reliability of the SATA connector (and later the related design used on Red's flash cards). Most SATA connectors were, at the time, designed to stand something like 250 insertions, which would be more than ample for the semi-permanent installations typical of IT applications. It's not adequate for moviemaking, however, or any circumstance which treats an SSD as a giant removable flash card. The SSD industry has, to some extent, recognised this unanticipated use of SSDs, and newer connector designs (which is the only place the insertion/removal durability count seems to be stated) are often advertised as being good for 1000 cycles. As a practical matter, I don't hear a storm of protest over problems with Blackmagic's design, possibly because the approach of most video recorders is to accept the entire drive as a slide-in unit, as opposed to plugging in a connector, ensuring things slide home in an orientation that's already parallel and properly aligned. If you're still worried, a good solution to this is that developed by Sound Devices, whose Pix series of recorders take commodity SSDs mounted in a custom connector assembly which can be replaced without loss of data even if it does break.

Performance

The final concern over commodity devices is that of performance, especially where people want to build devices intended to record data at rates that are still quite trying, such as uncompressed 4K Bayer mosaics. There do exist consumer SSDs of moderate price that can do this, and they're readily identifiable from simple reviews and relatively uncomplicated testing, although there is a legitimate need for caution at this level of performance as nobody likes dropped frames. For most of the work that's actually being done in the world, however, even low-cost SSDs tend to be more than adequate. High-grade HD work at 1920 by 1080, in 10-bit RGB, equates to no more than 200 megabytes per second, which practically any current SSD will happily sustain until very nearly full.

There certainly were problems with the earliest designs, which failed to anticipate the issues caused by fragmentation and suffered strange, difficult-to-fix performance problems as a result. Properly designed recorders (those which issue the SATA TRIM command appropriately, the details of which are beyond the scope of this article) will not encounter this issue with any currently-available SSD. It is just barely possible to buy an SSD that will exhibit performance problems with the large, continuous writes, but avoiding the bottom third of the price bracket for a drive of any specific size will generally avoid the issue, and it's easy to test for in any case. For a consumer product, caution over this might be appropriate. For a device intended for professional use, it's reasonable to expect a degree of expertise from the user, especially when the advantages of commodity storage are so great.

No need for Development

So, I'm unconvinced that there's any reason for new flash storage formats to be developed, and there are several reasons that standardisation is helpful. Existing devices are adequate for current needs, and are likely to be developed at a rate that is at the very least sufficient to accommodate foreseeable increases in the storage required for digital cinematography. When Sony released the F5 and 55 cameras, questions were raised about the legitimate technical necessity of the new AXS cards in light of Sony's already bulging portfolio of flash formats. Even in the consumer world, with CompactFlash, the new CFast and various grades of SD card, we're clearly in the middle of a format war, and the last thing we need is more proprietary contenders as well.