This incredible new technique allows us to see around corners in photographs!

 "Track 45 left. Stop. Enhance 15 to 23. Give me a hard copy right there." Deckard’s vocal instruction to the Esper machine to ‘see’ around the corner of a room in a photograph in the original Blade Runner looked unachievably cool in 1982 but could now be in reach.

Boffins at Boston University have devised a computer program that turns a normal digital camera into a periscope. They even call it “computational periscopy”.

There are systems that use lasers and sensors and a lot of hardware processing to achieve what’s called non-line-of-sight imaging but the breakthrough here has been made using a 4 megapixel digital camera and a mid-range laptop. The secret is in the algorithm used to decode the hidden object’s image and cunning use of information extracted from the object’s shadow.

“Basically, our technique allows you to see what’s around the corner by looking at a penumbra on a matte wall,” explained team member and associate professor of electrical and computer engineering, Vivek Goyal, in a paper published in Nature.

The original images displayed on a monitor (left) and the penumbra (centre) from which the computer makes its calculations, and the final resulting image (right). Image courtesy of the Goyal lab at Boston University

A penumbra is the partially shaded outer region of a shadow cast by an opaque object.

The researchers found that their algorithms worked best only when there’s a shadow of the object you want to reveal, so that requires something to at least partially block your view of it.

The resulting combination of light and shade at different points on the wall help the math reconstruct what lies around the corner. It does it in full-colour too.

In Goyal’s words, “Based on light ray optics, we can compute and understand which subsets of the scene’s appearance influence the camera pixels. In essence, computation can turn a matt wall into a mirror.”

In the test, they used an LCD screen to display several illuminated cartoon images. They then placed a sheet of black foamboard in front of the screen, casting a shadow onto a matte white wall. Off to the side was a digital camera, which couldn't see the LCD screen but could take pictures of the shadow on the wall.

While their program takes about 48 seconds to work out a hidden scene from a digital image, the researchers believe it could be could be done much faster with more computer power. Eventually, it may be fast enough to run on video footage, they reckon.

There are obvious applications for this sort of trick in autonomous vehicles to send alerts about pedestrians. The technology could also help police monitor buildings from the outside during hostage situations or first responders scout out collapsed buildings after disasters.

Since their Boston team’s research was funded by DARPA (Defense Advanced Research Projects Agency) the US military’s R&D wing, non-line-of-sight devices would also come in handy for spying, even for high-flying drone observation.

The technique could also be built into smartphones to allow you to take a picture of something that’s not even in the frame. Handy for paparazzi.
Perhaps the team’s greatest discovery is that when you realise how much light can be extracted you just can’t look at shadows the same way again.