To See or Not to See

CCTV cameras are now everywhere, silently monitoring the streets, shops and offices. With every street under the watchful eye of a camera, the police and security staff can now instantly spot crimes as they happen... At least, that's the idea!

The reality is that it is impossible to watch all those videos. Only a few will ever be viewed by a human. After a crime happens the police can collect and view the footage to help track down the criminals, but even that takes a vast amount of time, so will only be done for really serious crimes.

It is ever more important that potential incidents are flagged in time to allow the good guys to intervene immediately: investigating unattended bags as soon as they have been left, for example. It is just not possible for humans to do this without help. If only computers could "see", really see, and make sense of what they were seeing then they could help by spotting potential problems automatically and alerting security guards. It's not that easy though. In fact a massive research effort is being devoted to making computers "see".

One such project is run by Professor Sean Gong of Queen Mary, University of London. His team has developed new ways to process, interpret and manage the vast quantities of visual information captured by live CCTV cameras.

One importnat problem to solve concerns what is called "Zero Motion Detection". It's just what it sounds like - being able to do things like detect changes in a scene that could for example indicate the deployment of a bomb, or theft of a bag and the recognition of undesirable behaviour like trespassing or a pick-pocket in action.

The biggest job for a computer doing Zero Motion Detection is to work out what is in the background and what is in the foreground. Even with simple scenes that can be harder than it sounds - many optical illusions are the result of our own brains getting foreground and background wrong.

It gets even harder in real CCTV footage where the scene changes constantly as the light changes through the day, due to the activity of people, trees swaying in the wind and so on. Just because it is background, doesn't mean it doesn't change. By programming computer systems to adapt to slow changes over time, sudden changes in the scene can be detected.

For example if a person walks into a room and places a box, temporary changes like the person moving around are ignored, but after the box has been present for a while it is flagged as a new permanent object.

To recognise undesirable behaviour, the problem is to separate normal from odd behaviour. To do this, the computer is given lots of different footage that contains the "normal" if constantly changing state of affairs. The computer pulls out patterns in the data creating a "model", a representation, of what is OK based on this allowable behaviour. Then when the computer detects a variation that doesn't fit the patterns in the "model" it raises the alarm.

For example, if the computer has been trained repeatedly on sequences of a person entering a room and walking up and down, then it will thereafter ignore people entering and moving around because it is "normal". However if the person stops moving, that behaviour is detected as abnormal, as the training footage didn't include people stopping.

Work is in progress to add the technology developed in such research projects into existing security software, so it may not be long before the all-seeing computers are really seeing not just recording and so helping protecting a street or airport near you.