Evaluate your phone: Cognitive Walkthrough

Want some fun? Next time you are in a gadget shop thinking of buying something, ask the salesman (who will be full of specs and features) which of two mobile gadgets was easiest to use and which had the best usability evaluation results. Then watch him blag. He will probably have plenty to say but chances are he won't be answering the question other than something like "They are all really easy to use". Most gadgets are full of features - the more the better apparently, except most are then never used, because most people find it is too much hassle to work out how to use them.

We could just give up and accept that all the features are there to sell the thing not to actually be used by most people, or we could try and make things better. To do that we need ways to find out how easy the products we design are to use before they are released.

Users or Experts?

There are two general ways to evaluate a new interactive computer system: you can get lots of potential users to try it and see how well they get on. That can be expensive though and you already have the thing 'finished'. Can you afford to scrap it and redesign at that point just because people struggled to use it? Ideally you want to find problems earlier than that. Another way is to get experts to look for problems. They can evaluate the design before it's finished - perhaps based on a paper mock up of it like the storyboards used by film makers. Experts can just use gut instinct or they can follow a 'usability evaluation method'. That is just a sort of recipe to follow that is known to help spot problems.

One expert method that focusses on how easy devices are to learn to use on first sight, is known as 'Cognitive Walkthrough'. It is useful for everyday gadgets where people won't be trained first but just pick up and use them. A Cognitive Walkthrough can be done on paper prototypes or on the final device.

Let's try it on one of your gadgets like a phone or MP3 player (or better still a friend's you haven't used before). The hard thing as an 'expert' is actually to forget what you know about using such devices and put yourself in the position of a complete novice. You may know that the C symbol means 'cancel' but would everyone? And what is the difference between D for delete and C for cancel? You may know, but would everyone? How many problems can you you find?

How to do a Cognitive Walkthrough

To do a Cognitive Walkthrough, you first decide which task you want to evaluate (like "check who last called me and return to the home screen") You then write down the correct sequence of steps (like button presses) you have to follow to do it.

Next, for each step in turn, consider the following three questions:

Will a user who hasn't done this before definitely...

1. know what to try to do at this step?
2. see how to do it?
3. be able to tell they did the right thing?

For each question the challenge is to come up with a 'story' about how someone could get it wrong in a believable way.

The first two steps can be a bit confusing - the difference between them is about knowing what you are trying to do (like I have to get back to the previous screen next) and knowing how to do it (I have to press the button marked C to do that.)

An Example

Take my mobile phone and the final step after checking the last number dialed where you need to return to the home screen. Here it turns out there is a really bad problem the designers missed...Let's look at each question:

Will a user know what to try to do?

I can't see a real problem here - the person will know they have finished and know they need to be at the home screen to start the next thing like make a call. My mobile passes the test for this question: at least on this step of this task.

Will a user know how to do it?

This is where the problem is. It turns out that on all previous screens the right hand button directly below the screen and marked with a red bar is used to go back a screen. The word "Back" appears on the screen above it with "Select" appearing over the other button. This time though the word "Call" appears over the left button and "Options" over the right button. Neither are the correct thing to do. They could initially just be confused and believe it's not possible to go back or press "Options" in the hope that one of the options is to go back. None of the above work. For this screen only, to go back you have to press a completely different button (it has a permanent label marked C on it and does something else at other times). A novice who hasn't seen this before wouldn't have any reason to press that button as it is not labelled in any clear way.

Will a user know they have done the right thing?

If they do the right thing they will see the screen they were on previously so it will be clear they are moving back towards the start. If they do the wrong thing it will be clear as the wrong screen will appear. However, there could be confusion if they pressed Options, as they may then try scrolling through the options in the hope there is a quit option. They might not discover they were on the wrong track till several button presses later.

Problems, problems everywhere

With a bit of practice it is amazing how many problems you can find this way in everyday gadgets. Loads are released without having the problems found or fixed. Have some fun looking, and you will also start to see why so many functions never get used and why some people get to think they can't use modern gadgets at all. Whatever they think, it isn't that they are thick, it is just that the designers didn't try hard enough. Happy Hunting!