06 Jul 2015

This may be the weirdest video you have ever seen

  • Written by 
How a neural net sees the real world How a neural net sees the real world Memo Akten

This is odd. Very odd. But there's a simple - well, quite complicated, actually - explanation.

With apologies for a headline that sounds decidedly like clickbait, we really couldn't think of a more appropriate way to introduce this video. Because it is, 100%, weird.

If it looks a bit "trippy" there's a very good reason: it was made by an artificial neural network - probably the closest thing we have to the way a brain actually learns things.

Neural networks learn in layers. The lower layers look for edges and other primitive components of an image. As the image works it's way up through the hierarchy, the elements are put together into objects with ever increasing complexity and structure.

So at level two or three, a collection of circles and edges might become an eye. A couple of layers up from that, you might get a face. Or anything! The thing about this video is that the networks have been left to guess what they are seeing, and then make more guesses based on what they've already guessed.

This technique might lead to inaccurate and fantastical images, because it's reinforcing the mistakes, but in the long run, it should lead to better and better image recognition, because it allows us to see how the networks are "thinking".

A striking example of this is when a network was asked to show what it understood by a dumbbell. It dutifully drew a composite image of some dumbbells, but each one had a muscular human arm attached to it! In this case, the network had failed to accurately distil the "essence" of the concept of a dumbbell, probably because it had only ever been shown dumbbells being lifted by muscular arms.

 

From the Google Research Blog

You can imagine the sort of errors that might happen where the muddled "thinking" has less clear assumptions behind it than this one.

It gets even weirder than this. A week or so ago, the internet was abuzz with this image, and it's not surprising, because it looks really freaky.

This image was generated by a computer on its own (from a friend working on AI)

Thank you to rebrn.com for this image

Have a look at this article, which explains the whole process very nicely.

I can't help feeling that we can learn an awful lot about how we perceive things by looking at these pictures and videos.


David Shapton

David is the Editor In Chief of RedShark Publications. He's been a professional columnist and author since 1998, when he started writing for the European Music Technology magazine Sound on Sound. David has worked with professional digital audio and video for the last 25 years.

Twitter Feed