Video-based Facial Capture Gets Real

Written by Mark Ramshaw

ICT / Digital Human League / RedShark NewsICT and Digital Human League Facial Capture

The quest to develop a system for animating and rendering completely realistic human faces takes another step forward with the development of a new performance capture technique from the Digital Human League.

Under the guidance of Paul Debevec, the computer graphics researcher whose groundbreaking papers on image-based modelling and lighting helped create the visual effects in The Matrix, The University of Southern California's ICT Graphics Laboratory has arguably become the world's most respected pioneer in the field of photorealistic animation and rendering.

Its work on high dynamic range imaging has changed the way 3D artists capture and utilise real-world lighting information, while its Light Stage technology is now a staple tool for many visual effects studios when building digital doubles – even recently making an appearance at The White House for the creation of a 3D bust of Barack Obama.

Now, following the news that Debevec and a number of other VFX luminaries are working together under the Digital Human League banner to push forward realism for digital humans, comes a method for driving high resolution facial skins directly using video performance capture.

The process involves first creating a high resolution model of an actor's face, using one or more high-quality geometry and reflectance scans. This is then used to generate animation based on one or more video streams of a performance recorded in arbitrary environments. What's remarkable is just how accurately the system is able to map and recreate the subtleties of non-markered facial movement, without any geometry drift and without the need for any specialised rendering tools or tricks. The technique is due to be shown at this year's SIGGRAPH event, but in the meantime you can read more about the technique at the ICT webpage here, and see the technique in action and explained in the video below.

Tags: Technology


Related Articles

24 May, 2020

Sensors need better technology, not more resolution


The Sensor of the Future

Replay: This was first published in 2013, but the discussion about sensor technology is as relevant as ever. Just how can...

Read Story

27 April, 2020

Maxon Cinema 4D S22 first release from the new subscription model

Maxon has updated Cinema 4D, the company's acclaimed VFX, 3D modelling, animation and rendering software, to version S22. This is the first release...

Read Story

27 April, 2020

LaCie Rugged drives are the perfect storage you need when going gets really tough [Sponsored]

There can't be many more extreme environments to be in than the Arctic tundra during a full white out blizzard. This is the sort of environment that...

Read Story