Can you spot post colourisation?

Written by Phil Rhodes

Glenn MooreHand colourisation is highly skillful, but can AI ever match it?

A world of colour. The process of colourising photos has been around almost as long as photography itself. Done skillfully and the results can be amazing, but it is fraught with difficulties, and sometimes the results can be a mixed bag. Could artificial intelligence eventually automate the process?

It's becoming clear that appending the words “in color” to the title of a piece of motion picture footage has reached a level of popularity not seen since the 1950s, when it was necessary to remind monochromatic holdouts that they were missing something by failing to upgrade their equipment. That was understandable, but the phrase has become popular on YouTube mainly because of a surge of interest in early colour photography – although what's billed as early colour might, in many cases, actually be a rather more recent application of colourisation.

Before we jump into a discussion of colourisation, one thing this does make clear is that there's a very worthwhile public demand for seeing things in colour. Don't worry, this isn't going to turn into a manifesto on the subject of besmirching Citizen Kane with a bumper pack of magic markers. The public's love of colour largely seems to apply to historical material, the earlier the better, and often of very everyday scenes. The strange implication is that it should be surprising to discover that people strolled along the streets of the early twentieth century wearing clothes that weren't grey, like a sort of pre-Pleasantville monochromatic world suddenly turned to colour.

Part of the interest in old colour is the widely-held understanding that colourising photos represents a staggering amount of human effort (though more on that below.) There's also presumably some interest in the technical history of photography, although video titles such as “40 must see historic photos in color – this will change how you feel about the past” put the lie to the idea that all this is really all aimed at photo nerds. To indulge in a moment of complete subjectivity, there's something about seeing the past in colour which makes it more immediate, the humanity of the subjects less filtered by the obscuring screen of black and white photography. People like it.

Spotting the fakes

Much of it, though, is being promoted as original colour photography, which much of it is probably not. By the time we get to World War 2 In Color! we're probably looking at Kodachrome, which was launched in 1935 and (staggeringly) was manufactured until 2009. Although much black-and-white motion picture film was shot until the 1960s, colour photography had been quite everyday (and quite expensive) for decades prior. Before Kodachrome, though, it was pretty rare; the first thing called Kodachrome was actually a two-colour process in 1915. There were colour processes before then, but they were vanishingly rare.

It's usually quite clear whether a piece of black-and-white cinematography has been colourised. Modern approaches to colourising stills can be highly convincing, but generally rely on a degree of human intervention to impose varying hues on the picture. Someone's face, for instance, isn't generally a consistent hue on the brown-to-beige spectrum, but has redder or browner areas depending on the epidermal physiology of the person involved. It's possible to touch that sort of thing in by hand on a still, but tremendously difficult on a motion picture, where it's hard enough work to rotoscope enough different parts of the frame to create a convincing combination of colours in the first place. The giveaway on colourised film, usually, is that too many things have exactly the same hue.

As with so many things, all of this may be soon to change. Richard Zhang, Phillip Isola and Aleei A. Efros, from UC Berkeley, published a paper in October 2016 called “Colorful Image Colorization” in which they describe the use of a neural net to automatically add colour to images. It's a fundamentally impossible problem to solve if we demand accuracy, but Zhang et. al. targeted plausibility as opposed than the impossible task of reconstructing the original colour perfectly. In tests it fooled human observers a little more than a third of the time, which doesn't sound great but is considerably better than the prior art. They don't talk about moving pictures, but it's probably only a matter of time.

You can try their software here.

Title image by kind permission of Glenn Moore.

Tags: Post & VFX


Related Articles

27 July, 2020

LaCie 1big Dock SSD Pro and 1big Dock: Your storage media is now an important connectivity hub [Sponsored]

When is storage not simply storage? Today’s storage is now becoming a highly important connectivity hub to help speed up your workflows.

The LaCie...

Read Story

18 July, 2020

The Quantel name is legendary. This is its story, and ultimately what happened to it

The story of Quantel is a classic story of the rise and fall of a high-end manufacturer with a groundbreaking product based on very expensive,...

Read Story

16 July, 2020

Is colour grading essential? Perhaps we are too obsessed with it

The current consensus is that the best practice for shooting is to aim for the maximum options in post. So we shoot in log, or maybe RAW, to give us...

Read Story