by Steve Krug
You’ll
need Internet access for screen sharing and to get to whatever you’re testing if it’s online.
The screen recording software is used to capture a record of what happens on the screen and what you and the participant say. The best
[ 65 ]
chapter 8
part of having a recording is that you don’t have to feel pressured to take detailed notes. By dragging the “thumb” on the scroll bar in the video player, you can usually find any point in the recording within seconds.1
The recordings can also come in very handy at the debriefing session in case there’s a disagreement about what the participant actually said or did.
In theory, people who can’t come to the sessions can watch the recordings later, but in the universe I live in, it almost never happens.2 I consider this fortunate since I’d much rather have people come to the tests.
I use Camtasia ($300 for the PC, $150 for the Mac). Even though there are less expensive screen recorders available (including one—CamStudio—
that’s open source), I don’t know of another one that has as many useful features as Camtasia, including a built-in video editor that makes it easy to extract clips, add titles, and much more. I’ve relied on it for years, and it has never let me down. (Looking around for some wood to knock on.) The screen sharing software allows the people in the observation room to see and hear the test. There are many options to choose from; some require a paid subscription and some are free, and almost all of them have free trials. If you work for a large company, they may already have a license for one of them. (WebEx seems to be a corporate favorite.) Personally, I use GoToMeeting. It’s extremely user-friendly and reliable, it works on Macs and PCs, and it has a lot of very useful features. It costs $49
a month for unlimited use, with up to 15 computers connected at a time.
And it has VOIP capability built in.
It’s also particularly good for doing remote testing (Chapter 14), which you’ll probably want to do eventually.
b) A monitor and a keyboard. If you’re using a laptop, you should arrange to have a monitor and an external keyboard. A 17" monitor will make it easy for you to see the screen without having to sit too close to the participant.
1 Believe it or not, this action actually has a name: “scrubbing” the video.
2 To quote Yogi Berra again, “In theory, there is no diff erence between theory and practice.
But, in practice, there is.”
[ 66 ]
mind reading made easy
The screen resolution should probably be 1024 x 768, unless you know that a large majority of your users will have it set higher or lower. (If the participant says something like “I have a lot more showing on my screen,”
you can increase it to a higher resolution, like 1280 x 1024.)
c) A “plain vanilla” mouse. Don’t make the participant use an exotic trackball, a laptop touchpad, or one of those eraser-things that stick out from the middle of the keyboard that don’t seem to have a name. Some people will find anything other than a mouse difficult to use.
d) A USB microphone. Getting good-quality audio into the observation room is crucial. Straining to hear what the user is saying can be very tiring, and eventually people will take out their BlackBerrys or just leave.
Good audio also allows you to hear the “body language” of the user’s voice so you can easily sense whether they’re feeling comfortable or frustrated, for instance.
I recommend using VOIP (voice over IP) instead of a speakerphone if possible, because the audio quality tends to be much better. GoToMeeting includes VOIP service, or you can use a service
like Skype.
You’ll need a microphone to use VOIP. My favorite is the
inexpensive (about $25) Logitech USB Desktop Microphone.
Even if the microphone built into your laptop is good, having
an external mike allows you to position it so the observers
can hear both the participant and the facilitator clearly.
e) A
speakerphone.
Even if you use VOIP, you should have speakerphones
available in both rooms as backup. (Make sure that you have the phone number for the observation room.)
[ 67 ]
chapter 8
Pre-Test Prep
Allow 60 Minutes
At the beginning of the day, you need to make sure that everything in the test room is ready. Try to start doing this an hour before the first participant arrives so you’ll have enough time to fix anything that’s not working, plus a few minutes to catch your breath when you’re done.
�Test the screen recorder. Make a short recording and play it back.
The microphone volume setting should be turned all the way up since you and the participant are going to be relatively far away from the mike. In my experience, turning the recording volume all the way up won’t distort the audio.
If you’re using a laptop that has a built-in microphone, make sure that you’re recording from the right source by tapping the external microphone during your test recording. (It should sound like you hit it with a hammer when you play it back.)
�Test the screen sharing. Ask someone to step into the observation room for a minute and then start a screen sharing session and make sure that they can hear you clearly and see the screen.
�Make the cursor larger than normal. This will make it easier for you and the observers to follow what the participant is doing.3
�Turn off any software that might interrupt the test. Email, instant messaging, calendar event reminders, and scheduled virus scans are the most likely culprits.
�Make sure you have bookmarks for any pages you’ll need to open during the test. You don’t want to spend valuable test time typing URLs.
3 I’ll give details about how to do this—and the rest of my standard settings for Camtasia and GoToMeeting—at the book Web site.
[ 68 ]
mind reading made easy
�Try using what you’re testing. It never hurts to have one last look at your site or prototype to make sure that your Internet access is working, your server hasn’t crashed, and some dedicated developer hasn’t done some last-minute tweaking without telling you. It’s much better to find out now than when the participant is in the room.
�Reset everything. If you’re using sample data, make sure that you’ve reloaded a clean set. And clear the browsing history in your Web browser, so visited links don’t give the user “clues.”
�Touch base with your Designated Greeter. Make sure the person you’ve asked to welcome the participants when they arrive is ready for them.
[ 69 ]
chapter 8
Welcome
4 Minutes
You begin each session by
reading the first part of the
Hi, ___________. My name is ___________, and I’m going
script,4 which explains how
to be walking you through this session today.
the test is going to work.
Before we begin, I have some information for you,
Some people like
and I’m going to read it to make sure that I cover
to improvise these
everything.
instructions from an
You probably already have a good idea of why we
outline so it will sound
asked you here, but let me go over it again briefly.
more natural, but I
We’re asking people to try using a Web site that
recommend using the
we’re working on so we can see whether it works
script and reading it exactly
as intended. The session should take about
as written. Even though
an hour.
I’ve been doing testing for
twenty years, every time
The first thing I want to make clear right away is
/>
I give in to the temptation
that we’re testing the site, not you. You can’t do
to depart from the script,
anything wrong here. In fact, this is probably the
odds are 50-50 I’ll say
one place today where you don’t have to worry
something that gives the
about making mistakes.
participant the wrong
As you use the site, I’m going to ask you as much
idea (using a word like
as possible to try to think out loud: to say what
“opinions” or “feedback,”
you’re looking at, what you’re trying to do, and
for instance). Don’t
what you’re thinking. This will be a big help to us.
improvise.
Also, please don’t worry that you’re going to hurt
our feelings. We’re doing this to improve the site,
so we need to hear your honest reactions.
4 You’ll fi nd the entire script—complete with “stage directions”—on pages 147–152, and you can also download it from the book Web site.
[ 70 ]
mind reading made easy
Read the script exactly as
If you have any questions as we go along, just
written.5
ask them. I may not be able to answer them right
Reading it may feel a little
away, since we’re interested in how people do
uncomfortable because
when they don’t have someone sitting next to
we don’t often read things
them to help. But if you still have any questions
aloud—at least not to other
when we’re done I’ll try to try to answer them
adults. But no one will
then. And if you need to take a break at any
mind because the script
point, just let me know.
explains why you’re doing
You may have noticed the microphone. With
it, it’s only about three
your permission, we’re going to record what
minutes long, and they’re
happens on the screen and our conversation. The
probably curious about
recording will only be used to help us figure out
what’s going to happen.
how to improve the site, and it won’t be seen by
anyone except the people working on this project.
And it helps me, because I don’t have to take as
many notes.
Also, there are a few people from the Web design
team observing this session in another room.
(They can’t see us, just the screen.)
If you would, I’m going to ask you to sign a
simple permission form for us. It just says that
we have your permission to record you, and that
the recording will only be seen by the people
working on the project.
Do you have any questions so far?
5 If you fi nd that each time you read it you’re stumbling over something that doesn’t sound natural to you, you can make very minor changes that don’t aff ect the meaning. For instance, if you think “If you have any questions during this session” sounds more natural to you than “If you have any questions as we go along,” then edit your copy of the script so you say it the same way every time.
[ 71 ]
chapter 8
Just try to relax while you read it and connect with the participant: Make eye contact. Print the script out in large type so you can read it without having to stare too intently at the page, and try to look at the participant after every few sentences.
Don’t mumble. The participant needs to hear what you’re saying.
Don’t race through it, but don’t drag it out.
Don’t read it in a monotone or a sing-song voice. Try to put a little life into it but don’t make it into “The Midnight Ride of Paul Revere.”
[ 72 ]
mind reading made easy
The Questions
2 Minutes
Usability test participants
are often asked a series of
OK. Before we look at the site, I’d like to ask you
pre-test questions at the
just a few quick questions.
beginning of the test and
First, what’s your occupation? What do you do
post-test questions at the
all day?
end.
Now, roughly how many hours a week
I only ask a few simple
altogether—just a ballpark estimate—would you
questions, and they serve
say you spend using the Internet, including Web
three functions:
browsing and email, at work and at home?
Get the participants
And what’s the split between email and
comfortable talking.
browsing—a rough percentage?
Everyone can come up
with answers to these
What kinds of sites are you looking at when you
questions, so it gets
browse the Web?
them started talking
Do you have any favorite Web sites?
about themselves. This
makes it easier when they have to start thinking aloud.
Show them that you’re going to be listening to what they say.
Knowing that you’re actually listening and not just getting what you need to fill in a form tends to make the participants more invested in the process and increase their comfort level. But to have this effect, you need to actually listen.
[ 73 ]
chapter 8
Feel free to ask them follow-up questions. I usually ask at least one question about their job, like what their title means or what their company does. And if you don’t understand something they say (“We broker
irreducible energy credits”), don’t pretend that you do. Ask them to explain it.
Get the information you need to grade on a curve. By the time the participant has answered these questions, you’ll have a pretty clear idea of (a) what they do for a living and (b) how computer-savvy and Web-savvy they are. This—plus the sense of the extent of their domain knowledge that you get from their reaction to the Home page (coming up next)—is usually all you need to decide how this person compares to your target audience.
[ 74 ]
mind reading made easy
The Home Page Tour
3 Minutes
I always start a Web
OK, great. We’re done with the questions, and we
site test by having the
can start looking at things.
participant look around
the Home page and tell
First, I’m going to ask you to look at this page and
me briefly what they
tell me what you make of it: what strikes you about
make of it.
it, whose site you think it is, what you can do here,
and what it’s for. Just look around and do a little
The point is to see if the
narrative.
nature of the site is clear:
Can users figure out what
You can scroll if you want to, but don’t click on
this thing is? As I’ll explain
anything yet.
later,6 more often than
you’d expect, the answer is a surprising—and revealing—“No.”
Having them do this “narrative” also gives you some idea of what they already know about the site, the organization behind it, and the subject matter—their domain knowledge.
Note that you’re not asking them for their opinion of the Home page. The script doesn’t say “Look around the Home page and tell me what you t
hink of it.”
The instruction is carefully worded so they actually have a specific task to do: Figure out what this site is. This is a realistic (and important) task, one that people do on their own whenever they come to a new site. You’re just asking them to verbalize it.
And it doesn’t use much time, since most people will run out of things to say in two or three minutes. You don’t want to let it go on longer than three minutes anyway.
The script tells them that they can scroll, but they shouldn’t click on anything yet. If they click on a link anyway—and some people will—step in right away and ask them to go back to the Home page. Just say, “For right now, I just want you to stick to the Home page. Could you go back?”
6 The Big Bang Theory of Web Usability on page 122.
[ 75 ]
chapter 8
The Tasks
35 Minutes
The tasks are the “meat”
Now I’m going to ask you to try doing some specific
of the test.
tasks. I’m going to read each one out loud and give
At the beginning of each
you a printed copy.
task, you’ll hand the
I’m also going to ask you to do these tasks without
participant a copy of the
using Search. We’ll learn a lot more about how well
scenario and then read
the site works that way.
it aloud, word-for-word,
exactly the way you
And again, as much as possible, it will help us
wrote it.
if you can try to think out loud as you go along.
Why not just let the participant read it? If you do, some people won’t read it carefully enough and they’ll end up wasting time based on some misunderstanding of the task. If you read it to them, at least you’ll know that they’ve heard every word of it.
Once they start a task, try not to interrupt any more than necessary. Basically, just keep them focused on the task and thinking aloud until it’s time to move on to the next task.
How do you decide it’s time to move on?
Have they completed the task? If they have, hand them the next scenario and start the next task. (If they think they have, but haven’t really, you can ask if they could try doing it again another way, which will usually lead them to realize their mistake.)
[ 76 ]
mind reading made easy
