Umm, ahh... understanding

Video calls have finally made it. No longer a futuristic dream, anyone can now use Skype. Why bother with face-to-face meetings any more? Just stay in bed and teleconference in to work! Why can't life be that simple for once? It turns out that face-to-face is still better, at least part of the time - video streams are still missing something important.

Computer scientists once believed that if they designed better ways for humans to communicate with computers (good 'human-computer interaction') then they would make it easier for people to communicate with each other too (good 'human-human interaction'). But you don't necessarily make it easy for two people to talk just by making it easy for them both to talk to a computer. You need to understand what makes human conversation work and design for that.

Terrible texting

Text messaging is a good example of how things aren't always as obvious as you'd think. The original text messaging system was horrible human-computer interaction. It was intended just as a way for engineers to test the connections. You just wouldn't design a real system where people used a numeric keypad to type words: it's terrible! Despite that millions of people used it. In fact lots of people actually send more text messages than they have face-to-face conversations these days. Poor human-computer interaction made wonderful human-human interaction. Why? Because (by accident) texting has just the right properties to be great at what it's used for. It doesn't interrupt you like a phone or even face-to- face chat, so is great for keeping in touch while you are busy. You can also keep several different chat conversations going in parallel. Another plus is that texts are short. Talk to someone and you have to go through a lot of social niceties - you can say goodbye half a dozen times on the phone before you actually manage to put the phone down on a friend! Texts avoid all that.

Splurging sentences

So what is so subtle about human-human interaction? When we talk to people face to face, we don't think up whole messages and then just splurge out the whole thing (as we do in an email). Watch people chatting. Listen for all the umms and ahs, the pauses, the half said sentences, the corrections, the "ok"s and "ah ahh"s from the other person that keep the conversation going. Real conversations don't go...

"This trouble I had in understanding not only what others said to me, but also what I said to them"

They are more like:
You: "This prob...err...trouble I had..."
Listener: "uhu"
You: "in under umm in under..."
Listener: "understanding?"
You: "yes"
Listener: "go on"

Meaning in mess

It looks a mess, but it all actually matters to make conversations work. It's about being sure you understand each other. There's more too. Now look for the nods and smiles, the blank faces, turns of the head, the waving hands, the pointing that go with all the "umms" and "ahhs" and interruptions. The listener's interruptions show they are following. Subtle changes in their expression might make you rethink what you were saying. Some of that does come across on a small video image of a person's head and shoulders but not all.

Subtle space

One of the key things missing in a video conference call is 3-D space itself. When people are working together on a problem they use the space around them as part of this subtle communication. A person may briefly glance to another for confirmation: confirmation that comes with a tiny nod of the head. If all you have is a series of images lined up on your screen no-one can tell who is being glanced at in the way you can round a table. Similarly human gestures aren't just about sliding your fingers about on a touch-screen. In deep conversation we make shapes in the air around us with our hands as we talk. Point to that place later in the conversation and people will know what we are talking about, just as it can be clear we mean Jo not Laura when we just say 'she'.

Good, bad (and ugly?)

All this doesn't mean video calls are rubbish. It just means you need to understand what different technologies are good and bad at and use them appropriately. Email is great for sending information but terrible for anything with emotion - hence the flame wars that result. Phone calls are great for one-to- one conversations, but bad if you just want to communicate a quick message like "Sorry I'm running late". Video calls do let you see each other's expressions and the things you are talking about. They can be good for 2-way conversations. What we are still missing though is a technology that is as good as actually sitting round a table working as a team: a technology that doesn't lose all the messy subtlety of human conversations. Video calls aren't it. Maybe you can invent a technology that is.

This article is based on a keynote talk given by Pat Healey of Queen Mary, University of London, June 2013. (cs4fn was there watching it live!)