From creepy to credible

If you saw Up last year, you’ll know that the animation isn’t designed to look completely realistic – you’d never mistake Carl and Russell for real humans. Ultra-realism isn’t exactly Pixar’s style, fortunately, because it’s still very difficult for CGI animators to make people look realistic. They usually end up looking creepy instead.

Animated or re-animated?

The problem is the face – people’s facial movements are incredibly subtle, and we’re all very good at picking up on them. If anything is amiss, an animated human looks kind of zombie-ish. That’s exactly what went wrong in 2004, when the Tom Hanks film The Polar Express came out. It was meant to be a charming Christmas film, but instead it got a reputation for creepiness – one CNN critic said it should have been subtitled 'The Night of the Living Dead'.

What went wrong? Well, the performances in The Polar Express came from real actors wearing motion-capture suits (Gollum in the Lord of the Rings films is the most famous example). The problem came because there were two crucial bits of the actors where the Polar Express animators couldn’t put sensors: the inside of the mouth and the eyes. Those areas had to be created from scratch by computer animation, and at the time, the technology just wasn’t good enough to get it right. The same CNN critic said that when the characters spoke, their tongues looked "like slabs of meat". Eww.

But there is hope. In 2008 the University of Southern California’s Graphics Lab partnered up with an animation company, Image Metrics, to try to produce a completely photorealistic animated face. They were going to try to break through the creepiness barrier.

Right in the face

First they chose an actor – Emily O’Brien, a soap opera star – to deliver a monologue about computer animation. Once they had their actor, they needed to gather all the information they could about her. Well, her face anyway. To do this, they put Emily into what’s called a light stage. It’s a huge spherical rig, with lights all around it. By turning various combinations of lights on and off, the lab researchers can record how an actor’s face looks under different lighting conditions. They can eliminate the natural reflection of the skin, so that the team were able to see Emily as evenly lit as you could possibly get.

Using the data from the light scans, the USC team could also build 3D maps of Emily’s face. One was a low-resolution map that showed her basic bone structure, and another was in such high detail that they could see the individual pores of Emily’s face. Next, the lab team took photos of Emily with 33 different facial expressions, which allowed them to capture the movement of her entire face, especially the trickiest parts to get right, her eyes and mouth.

Then the team analysed the pictures. With resolution that went down below a millimetre, they could even see how Emily’s pores got longer and shallower when she stretched her cheek. They also gathered information about the colour of her face at every point, and they even took exact models of her teeth! All of that data would be used to create the most realistic picture of a moving face they could get. All that was left was to animate the picture.

Making Emily move

The USC lab team handed their scans over to their partners the animators at Image Metrics, and within a few months they had made a fully movable model. It was so detailed that, unlike the Polar Express characters, the digital Emily even had fully modelled, moveable skin around her eyes. The next step was for the animators to record a video of the real Emily speaking the lines of her monologue. Then they pulled, stretched and shifted the muscles of Emily’s digital double so that they matched the real movement exactly.

In the final video of the monologue, the animators replaced Emily’s real face with the digital version. They did an amazing job: it’s practically impossible to tell the difference, and many people can’t even tell after they watch the video many times.

It may not be long until we see realistic digital humans in the movies. When we do, it’s entirely possible that animators will look back to Emily as the beginning of a new era: the first time animation left creepy, zombie-like humans back in the horror films in which they belong.