NVIDIA has just made super slow motion possible with any camera

Written by Adrian Pennington

NVIDIA

A team of researchers claims to have cracked the secret of making videos shot at a lower frame rate look more fluid and less blurry when played back at a higher rate.

The difference between individual frames of video is pretty negligible, the result of either the camera moving or an object moving very slightly from frame to frame. Compression techniques exploit this fact by using mathematical algorithms to predict (interpolate) motion in a frame – given knowledge of previous or future frames – and to reduce the amount of data required.

However, unless you record a high enough number of frames, slowing down footage for a slo-mo replay can appear nigh on unwatchable. High-end pro cameras like the Phantom VEO 4K can capture full resolution 4K images at 1000 frames per second (fps) and do an unbelievably good job, but at $60,000 a pop, these are used only for top-end natural history, sports or corporate promo applications.

Unpredictable events

While it is possible to take 240-fps videos with a cellphone, many of the moments we would like to slow down are unpredictable – the first time a baby walks, a difficult skateboard trick, a dog catching a ball – and, as a result, are recorded at standard frame rates.

Likewise, recording everything at high frame rates is impractical for mobile devices – it requires much memory and is power intensive.

But could high-quality slow-motion video be generated from existing standard videos? A team of researchers claims to have cracked the code.

“Our method can generate multiple intermediate frames that are spatially and temporally coherent,” the researchers announced. “Our multi-frame approach consistently outperforms state-of-the-art single frame methods.”

Interpolation

For instance, generating 240-fps videos from standard sequences (30-fps) requires interpolating seven intermediate frames for every two consecutive frames. In order to generate high-quality results, the math not only has to correctly interpret the motion between two input images but also understand occlusions.

“Otherwise, it may result in severe artefacts, especially around motion boundaries, in the interpolated frames,” explain the researchers.

They used over 11000 YouTube clips with 240-fps, containing 300K individual video frames, to train the artificially intelligent system and have had their findings – published last November - officially backed by GPU-maker Nvidia this month.

The system makes use of Nvidia Tesla V100 GPUS and its cuDNN-accelerated PyTorch deep-learning framework.

Now you can take everyday videos of life’s most precious moments and slow them down to look like your favourite cinematic slow-motion scenes, adding suspense, emphasis, and anticipation,” suggests Nvidia.

Tags: Technology

Comments

Related Articles

31 July, 2020

This is how Netflix is adapting Anime to modern technology

The streaming service brings 4K and HDR to the classic Japanese artform in the latest example of its prototyping production techniques.

Netflix's...

Read Story

30 July, 2020

Gigabyte Aero 17 XA review: A competition beating powerhouse [sponsored]

The Gigabyte Aero 17 XA has some pretty nifty specs on paper. How does it stack up in the real world, and more importantly against the competition?

...

Read Story

30 July, 2020

For all film makers: How to avoid losing your stuff and where to put it

Replay: The technological revolution has created great opportunities for new film-makers everywhere, but has in its wake created a new challenge:...

Read Story