Your future check mate

Robots that play games with you need to be able to tell how you feel. When you’re playing with a friend they know what’s on your mind, no matter if you’re winning or losing. The expressions we make on our faces are a part of the social glue we call friendship. So if we want to build robots that can be our friends, they need to be able to read our faces.

The main features

People’s faces are interesting. Over the inflexible bone of our skull is stretched a layer of skin and muscles that make us look like the people we are. We can wrinkle our noses, we can flex an eyebrow or two, we can smile or frown. Our faces are wonderfully flexible. But this amazing flexibility is a real problem for computers. When is a smile a smile? What did that narrowing of the eyes really mean? To work with us, computers need to be able to give meaning to the subtle shapes we create with our faces. Researchers in the UK, Portugal and Poland working on a joint research project called LIREC have had to think of new ways to understand the expressions on faces. That’s because they are developing a robot companion to help school kids learn to play chess.

Your checkmate and its secrets

The body of the robot is called the iCat. Shaped like a cartoon cat, it can make sounds, and move its mouth and eyes to create expressions that the other human player can understand. But how does the robot know what face to pull? It needs to understand how you are feeling about the game. To do this the robot uses lots of different clues: it can detect how often you crouch over the board, which tells the robot how interested you are in what’s happening. Perhaps most importantly, though, the iCat has learned to recognise the expressions on children’s faces, which is particularly tough for a computer.

Kids vs adults

Adults in social situation tend to make fairly strong expressions with their faces. It’s often clear when an adult means to smile, but kids’ expressions tend to be more subtle. So the researchers recorded hours of video at a local kids’ chess club and went through it by hand, deciding when a smile or a frown was present. Then they noted the situations where these expressions occurred. Armed with the videos, they were able to train up a computer program to recognise how the various parts of the face moved when the kids were feeling in a particular mood.

Check your learning by example

Here’s how to train a computer to recognise a smile. The researchers gave the computer around 500 of the videos, and programmed it to automatically recognise where a face was and notice how the important parts of the face moved between a neutral expression and a smile. Each time someone in the videos smiled, it was up to the computer to notice. If the computer got it right, great! If it missed the smile, the program used this error to correct itself so it got it right in future. Just as though it were in school, it learned by example and by having a teacher telling it when it was right and wrong. After its schooling, the computer was able to recognise new smiles it hadn’t seen before at about 90% accuracy – it had learned what the smile on a young child’s face looks like.

This face reading ability was then combined with information on the state of the game, giving the robot the ability to ‘understand’ what the human player was feeling. It could constantly check the face of its human companion and make sure that it was playing at an appropriate level to help them improve their game.

Using this technology, based on our understanding of human faces, the companion robot became a checking mate rather than just getting checkmate!

Playing for real

How can the iCat computer play on a real chess set when most play on a screen? Partly it’s because the chess set contains a secret: there is a small microchip in each piece that lets the robot know how it’s being moved on the board. The robot also has a state- of-the-art chess program installed, so it can play an easy or a more challenging game as it sees fit. It then just tells you what move it wants to make.