by Steve Krug
75 feet away.
[ 125 ]
chapter 12
Here’s another example, from a site the Usability Professionals Association built a few years ago for World Usability Day. As you might imagine, it was quite…usable. The top of the page looked like this:
But if they actually wanted people to notice and use the excellent navigation system they’d built, the visual cues that would let you know it was there needed to be less subtle. Here’s a version I doctored to show what I mean: If you want people to use something you’ve built, they have to notice it first. I think it’s always possible to maintain visual appeal—and even sophisticated design—and still direct people’s attention to where it needs to be.
[ 126 ]
the usual suspects
FAQ
Why so much focus on the Home page? Hasn’t Google made it irrelevant?
There’s no question that nowadays most of us live in Google. Almost everything I do starts with a Google search. Or a Wikipedia search.
In fact, I even use Google to do my Wikipedia searches. For instance, I type
“Hasselhoff wiki” in Google, and sure enough, the first entry on the Google results page is the Wikipedia page for David Hasselhoff. Nine times out of ten it’s what I was looking for.
As a result many (or most) of the people who come to your site don’t enter via your Home page anymore. They search for something in Google and go directly to some lower-level page in your site.
A lot of people think this means that the Home page is no longer
important, but they’re wrong.
If people land on an interior page of a site and it’s not exactly what they’re looking for, very often the next thing they’ll do is look around for a Home link so they can bob up to the surface and get their bearings. What is this site all about? Who are these people? What else do they have to offer?
Are they credible? Very often, their next click after that is on an About Us link, and hopefully, the About Us page starts with a clear, simple, brief explanation of who the publisher of the site is and what they do—and not a mission statement.
The Home page still matters and its job is to clarify who you are and what you do quickly, so people who teleported in via Google can decide whether your site is worth further exploration.
[ 127 ]
chapter
chapter13
13
Making sure
life actually
improves
the art of playing nicely with others
[ 128 ]
Interviewer: But do you feel you’ve learned
from your mistakes?
Sir Arthur Streeb-Greebling: I think I have, yes.
I’m sure I could repeat them exactly.
—PETER COOKE AND DUDLEY MOORE,
IN THEIR “FROG AND PEACH” SKETCH
While writing this book, I looked back through a lot of reports
from tests I facilitated years ago for clients. I used to do very nice versions (if I do say so myself ) of the Big Honkin’ Report, complete with screenshots illustrating the problems and sometimes even some doctored screenshots showing possible solutions. They were very clear and easy to read (I was told), and my clients seemed to agree strongly with my conclusions. In fact, they were usually very enthusiastic about the process and jazzed about the idea of improving their product.
But then I’d watch their sites for signs of the fixes being implemented. And a disappointing number of times, nothing happened. Three months. Six months. A year. Nothing.
Even though my reports typically mentioned dozens of problems, I always tried to make a clear list of the most serious ten or fifteen and emphasize that these should be given top priority.
Some of these problems involved relatively easy to implement fixes that I thought would make a huge improvement in the site’s customer experience, and quite probably in its profitability. And the people I delivered my findings to (often fairly high up the organizational food chain) agreed that the changes were important and valuable. Even though the problems I was talking about usually weren’t news to the people who brought me into the project, they were news to those higher up, and they seemed clearly committed to making the fixes soon.
I’ve seen this happen many times in all kinds of organizations, and other usability professionals have told me that they’ve often had similar experiences.
[ 129 ]
chapter 13
Why things don’t get fixed
So what’s going on? If people understand what the serious problems are and how to fix them and they have the requisite clout to make sure it gets done—
and in many cases the problems aren’t even very hard to fix—why don’t things improve?
Why does this happen? And more important, how can you make sure it doesn’t happen to you? Here are some of the usual reasons why things end up not getting fixed:
Change of management, change of direction, or both.
Putting things off. If fixing a problem turns out to be more work than anticipated, the easiest solution is to just say “That’ll have to wait for our next redesign.” (Translation: the check is in the mail.)
Lack of sufficient buy-in from all the right people.
Sabotage. Believe it or not, team members and stakeholders who feel they didn’t have a voice in deciding what to fix have sometimes been known to drag their feet.
Your eyes were bigger than your stomach. In their enthusiasm, teams often put an unrealistic number of problems on their plate.
Problems turn out to have deep roots. When you go to fix some usability problems, it quickly becomes clear that they’re actually a symptom of some much larger unresolved conflict—about the site’s purpose or the company’s mission, for instance.
And above all,
Life intervenes. For whatever reason it turns out that you just don’t have the time, resources, or commitment to follow through.
To survive all of these and end up with real improvement, you need serious, durable commitment from everyone involved: management, your team, and the stakeholders.
[ 130 ]
making sure life actually improves
It helps to have friends in high places
People often ask me about the best way to sell usability to
management.
One obvious way is to be persuasive: understand management’s
goals and figure out how usability can advance them, learn to
speak their language, make frequent presentations about your
testing efforts, and so on. Certainly a good idea.
You can also make the argument for the return on investment
(ROI) for money spent on usability. There’s even an excellent
book on the subject: Cost-Justifying Usability (2nd Edition, 2005, edited by Randolph Bias and Deborah Mayhew). ROI case
studies can be very convincing,1 but they also tend to be very
time-consuming and expensive to create.
And even if you succeed in convincing management, when money is tight, as the new kid on the block usability is likely to be one of the first things thrown overboard (following the last-hired/first-fired principle).
Usability testing (and user-centered design in general) is slowly (very slowly, I think) becoming a “must have” for some enlightened organizations. But in tough times, it’s still not on the “indispensable” list.
And when it comes down to getting products out the door sooner or getting them out the door and usable a little later, usability will often lose. Management knows that users won’t be able to do anything if you haven’t got code and content written, but it’s easy to assume that even if the thing is confusing or hard to use people will still manage to use it somehow.
1 …particularly when you’re working on an intranet, where you can quantify the payoff .
(“Our tests show that with a new design employees can save 15 minutes a week in the time they spend looking people up in ou
r corporate directory. At an average salary of 35 cents a minute for 1,000 employees, it amounts to a savings of over $200,000 per year. Our testing and redesign cost $10,000. Net savings: $190,000.”)
[ 131 ]
chapter 13
Personally, I’m not a big fan of using ROI arguments. I think most companies that need ROI-style proof to convince them to “do usability” probably aren’t going to do great work anyway. That requires more than the sense that it’s profitable—it requires a passion to do it right. Buy-in is OK in flush times, but when resources are short, you need people who are fanatics—who can’t imagine not spending time and money on creating a top-notch user experience.
So what works?
Fortunately, you have at your disposal a mechanism for generating conversion experiences, which is what it takes to make a fanatic.
Rather than engage in arguments about the value of testing, I’d rather rely on demonstration. Don’t try to make converts: let the seeing-is-believing effect of watching usability testing make converts for you. I tend to think it’s much easier and longer lasting and more recession-proof to have the boss and everyone on the team become true believers.
[ 132 ]
This page intentionally left blank
chapter
chapter14
14
Teleportation
made easy
remote testing: fast, cheap, and slightly
out of control
[ 134 ]
What am I working on?
Uhh.... I’m working on something that will
change the world, and human life as we know it.
—SETH BRUNDLE (JEFF GOLDBLUM), IN THE FLY
Remote testing is a simple idea: instead of bringing the users to you, you go to them—electronically. Instead of looking at the screen over the participant’s shoulder, you use screen sharing. And instead of conversing face to face, you talk on the phone (or via VOIP).
I first did remote testing 15 years ago. With no screen sharing software, I had to imagine what the user was doing—based on what he said while thinking aloud—and try to duplicate his actions on my computer so I could follow along. As you might imagine, I spent a lot of time asking “What screen are you on now?”
Today, though, with robust screen sharing software and broadband access, the experience is a lot like looking over the participant’s shoulder.
Why do it?
One word: convenience. Remote testing has several significant advantages: Easier recruiting. Your pool of potential participants widens from
“people who live or work near where you’re testing” to “anybody with a fast internet connection.” This is particularly helpful when you’re looking for a particular type of user.
No travel required. For the participant this means the whole thing takes an hour of their time, not two. This is very helpful when recruiting people who have very little free time.
Easier scheduling. You can do tests at almost any time of day. For hard to find people who are only available at 11 pm, you can run sessions at 11 pm.
Produces [almost] the same results. Remote testing is very likely to uncover the same kinds and the same amount of problems as testing in person.
[ 135 ]
chapter 14
If it’s so great, why not do all tests
remotely?
Overall, I’d say remote testing gives you about 80% of the benefits of a live test with about 70% of the effort.1
You do lose 20 percent of something not being in the same room as the participant. The in-person experience is just richer, somehow. It’s just a little harder to know exactly what they’re thinking.
And having a layer of technology between you can lead to misunderstandings.
It’s about the same as the difference between having a conversation with someone on the phone as compared to having the same conversation in person. You usually have to spend more time clarifying what the participant said and meant.
You also have a lot less control of the session. For instance, if someone walks into the participant’s office, or the participant decides to take a phone call, there’s comparatively little you can do about it. And it can be particularly harder to rein in tough customers when you’re not in the same room, because you can’t use your body language to indicate that you really mean it’s time to get back on track.
How do you do it?
Almost everything about remote testing is the same as testing in person: you choose what to test, write scenarios, follow the script, ask them to think aloud, probe, and so on. You can test anything you can display on the screen. You may have to make minor modifications to the script and you’ll have to mail them their incentive check or email them an Amazon gift certificate.
You always need to do a quick test of the screen sharing before the test session.
You can do this when you call to confirm their test appointment.
You have to decide whose screen is going to be shared: yours or theirs. It’s best to let them access what you’re testing from their computer while you watch via screen sharing so they won’t be affected by the inevitable (but usually slight 1 Yes, that’s a guesstimate. As Jared Spool like to say in presentations, “74% of all statistics in presentations are made up on the spot.”
[ 136 ]
teleportation made easy
lag time). If you’re testing something that is only installed on your computer, you can give them control of your screen.
(If they’re sharing their screen, be sure to tell them to hide anything they don’t want you to see, like email.)
As I mentioned in Chapter 8, you have a number of options for screen sharing software. The most important factor in choosing one for remote testing is ease of use for the participants. You want something that (a) requires as little setup time for the participant as possible (preferably less than a minute), (b) will work through corporate firewalls, if necessary, and (c) doesn’t require installing an actual application, which many corporate IT departments won’t allow.
Again, I prefer GoToMeeting for screen sharing, and I have nothing but good things to say about it. The installation for the participant is a simple automatic download that takes about 30 seconds, and I have yet to encounter a participant who couldn’t use it. It tends to refresh the screen quickly so there’s little or no lag between what the participant sees and what you see.
It also does an excellent job of screen resizing (since your screen may not be the same size or resolution as the participant’s), and it makes it very easy to switch whose screen is being shared. 2
For audio, you can use GoToMeeting’s conference calling service (included in the subscription price, but each caller pays their own toll call charges), or you can use VOIP if the participant has a microphone connected to his computer.
If you’re not using VOIP the participant should have a speakerphone if possible, so they don’t have to hold a phone to their ear for 50 minutes.
Ask them to turn on call waiting if they have it and agree to try to keep interruptions to a minimum. Since they’ll be at home or at work, though, you have to be prepared for interruptions.
You can record the entire session by running a screen recorder on your computer, positioning the microphone near the speakerphone.
2 I know, I know. “If you love GoToMeeting so much why don’t you marry it?” But it’s so well designed that I really do enjoy using it. I rarely do any kind of conference call anymore without using screen sharing.
[ 137 ]
chapter 14
Faster, cheaper, and even more
out of control
While we’re talking about remote testing, there’s one other option you should be aware of: unmoderated remote testing.3
Consider a service like Usertesting.com.4 Here’s how it works:
You give them a URL for what you want to
test and a task (or maybe two short tasks).
Then you tell them how many participants
&nb
sp; you want and specify a few preferences
like gender, age, income, and computer
experience.
They post the request online to their pool
of testers, who then sign up to do it. Each
tester goes to the URL and spends about
15 minutes doing the task(s), while
thinking aloud. When they’re done, you
get a link to a screen recording of their
session.
Obviously, it’s not the same as sitting
down with a user, because you can’t ask
questions and you can’t probe. But given
the limitations, the recordings can be
surprisingly helpful. (The participants
have been screened to ensure that they’re
good at thinking aloud, and they tend to
put a fair amount of effort
into it.)
3 Tom Tullis, Bill Albert, and Donna Tedesco will be publishing a book on the topic (Beyond the Usability Lab) in 2010.
4 There are several of them now with the same business model. Usertesting.com was one of the fi rst and it’s the one I’m most familiar with.
[ 138 ]
teleportation made easy
The beauty of it is that it’s inexpensive ($29 per user), requires very little effort (you just have to come up with the task), and fast (you can often get results back the next day).
The quality is not going to be the same as doing a moderated test, but I’ve been favorably impressed.
For the price, I think it’s an excellent thing to have in your toolbox. It’s perfect for getting a quick-and-dirty take on some question that’s not worth including in your monthly testing, or that just can’t wait for it. It’s also very handy for doing a quick retest after you’ve fixed a problem you found in monthly testing, since you already have the task written.
FAQ
Why is this chapter at the end of the book?
An excellent question. It does seem like it might make more sense to have this in the “Finding Problems” section of the book. But there’s a very good reason why I put it back here:
You shouldn’t try remote testing until you have
some in-person tests under your belt.
