FAQ
- What kinds of resources do you study?
- So, what's usability testing, anyway?
- How do usability projects work?
- What test tools do you develop and use?
- How do task-based exercises work?
- How large of a sample do you recommend?
What kinds of resources do you study?
We have conducted usability studies about newspaper Web sites, public library sites, corporate research intranets, and a leading federal Web site on cancer research. Lately, we've been conducting user-centered research for an extranet that lets foundation donors manage their philanthropic funds. Often, our own usability findings surprise us and that's one reason we enjoy our work.
Although we do a lot of usability research about online products, we study tangible, physical information resources, too. Not long ago, we conducted a market research study about who buys newspapers from racks and retail outlets. We made recommendations about how the newspaper company could improve services and increase sales.
So, what's usability testing anyway?
Usability testing measures the quality of a user's experience when interacting with an interface. The interface could be a Web site, a software application, or for that matter, any user-operated device. We think an interface should be easy to learn, easy to remember, satisfying to use, and cause few errors and/or distractions.
In our work, we evaluate an information resource in three primary areas:
- Task support-does the interface support the work users need to get done and the goals they need to fulfill?
- Aesthetics-do the design elements (font, size, layout, color, etc.) help users interact with an interface?
- Usability-does the interface have an ease of use, "memorability," and "learnability"?
How do usability projects work?
We generally divide our usability projects into three phases: (1) Developing test tools and recruiting the sample, (2) gathering quantitative and qualitative data through one-on-one sessions with participants, and (3) interpreting the results, making design recommendations, and writing the final usability report.
Our work usually occurs over two to four months. If we come back and test a newly redesigned site or feature down the way, the project can take a little longer. We prefer to work collaboratively with a client and have project benchmarks along with client-sign offs during each phase.
What test tools do you develop and use?
We use surveys and task-based exercises. We develop surveys for gathering demographic data about participants and to find out their preferences and attitudes.
We spend most of the time in test sessions with participants completing task-based exercises. We develop the exercises for collecting behavioral data and measuring how participants interact with a resource.
How do task-based exercises work?
The exercises are administered in one-on-one sessions with participants. A test administrator works from a script with the participant. A data logger records participant's comments, error rates, and navigational click streams.
Often our tasks are sequential, with one building upon another and focusing on frequent tasks users conduct and the goals they need to fulfill.
Here's an simplified version from a public library's site test we conducted:
- Find a specific library book.
- Find if a local branch has an available copy.
- Place a hold on the book.
- Find directions to the library to pick the up the book.
- Check hours when the library is open.
For this study, we collected quantitative usability measures, such as how many participants successfully completed a task, how many clicks it took to complete a task, what kinds of errors were made, and how many participants abandoned the task and went somewhere else for a solution (i.e., telephone, Amazon, or Google).
In our final report to the client, we reported the measures and findings about the site's layout, labels, functionality, navigation, and search, as well as design recommendations for improving the site.
How large of a sample do you use recommend?
Conventional wisdom says this: (1) observing four or five participants will allow a usability practitioner to discover 80 percent of a product's usability problems, (2) observing additional participants will reveal fewer and fewer new usability problems, and (3) more severe usability problems are easier to detect with the first few participants.
Still, other usability practitioners have questioned this prescriptive answer. In their own studies they have found that five participants only revealed 35 percent of the usability problems.
Suffice it to say, there is no magical number for usability samples. We think, however, that three other variables related to a project must be considered in determining a sample:
- The diversity of the people who use the product/site and the site's mission (e.g., public library users are a diverse group vs. software engineers for a given product for producing prototypes),
- The complexity of tasks the product/site supports (e.g., an ATM is a lot more simple to use than using Fidelity's site for tracking your portfolio and managing your funds),
- The kind of usability test being run (e.g., a hands-on assessment test of an existing site with a goal of improving its current functionality/usability is different than a validation test at the end of a cycle, certifying that the site works).
Considering all of this, we think determining a sample for any project requires a closer look at the research questions and goals of the whole project.
Why is usability important?
People using the Web clearly have great difficulty finding the information they need for living, functioning, learning, and surviving in an increasingly digitized world.
Usability research offers design recommendations for making sites easier to use, so that people can carry out the work they need to get done with a site.
The Pew Internet and the American Life Project found that 46 percent of their sample said they encountered problems using governmental sites they studied.
Research by User Interface Engineering shows that people cannot find the information they seek on Web sites about 60 percent of the time. This can lead to wasted time, reduced productivity, increased frustration, and loss of repeat visits and money.
According to usability guru Jakob Nielsen, "Studies of user behavior on the Web find a low tolerance for difficult designs or slow sites. People don't want to wait. And they don't want to learn how to use a home page. There's no such thing as a training class or a manual for a Web site. People have to be able to grasp the functioning of the site immediately after scanning the home page-for a few seconds at most."