Video as moving art

Still from NYC Flow by Danil Krivoruchko

Apparently, efforts to apply a convincing, organic painting style to moving images are much further along than we had thought, as evidenced by the following video.

Here's a video (kindly pointed out to us by RedShark commenter Liam Finn) that shows exactly what we predicted wasn't going to happen for some time.

It's some footage of New York that has been processed to look like a moving painting.

Deep technique

The filmmaker, Danil Krivoruchko, has processed his video using some software, which is open source and available on Github that applies a painting style to the moving images. I have to say that this is way more impressive than I was expecting at what is probably an early stage in the development of this technique.

This is far, far deeper than some of the painting, sketching and watercolour effects that have been around for about fifteen years. It's almost as if the video is being perceived in some way. No, I don't know what I mean by that either, but what I'm getting at, I think, is that this type of processing looks as though it's been done by a brain rather than a processor.

Much like us

In what little information is available, there are references to neural nets. These are programs that can be taught to recognise patterns and to give a stronger output when they do recognise something than when they don't. Most importantly, they don't need specific programming. In a sense, they learn by themselves through exposure to elements of the real world. Crucially, the technique here can separate content from pixels. It can extract the style of a painting and apply it to objects in a video.

This is definitely more analogous to the way our brains work than, say, our smartphones, although it's perfectly possible to write apps for smartphones that use neural net techniques.

This kind of technology is taking us in directions that we would probably not expect if we insist on being bound by the concept of pixels. The real world is not made from pixels and nor is the way we perceive the world. Ultimately, this could take us towards a form of video that deals with objects and impressions of those objects.