Outlook
Beyond the Firewall: Assessing Corporate Intranets
An Interview with Alison Head
by Nikki Poling
This interview appeared in Information Outlook, Vol. 6, No. 6, June 2002. Copyright 2003 Special Libraries Association. All rights reserved.
Alison Head entered the field of library and information science with an interest in information-seeking behavior. She wondered how people formulated research queries and sought answers how they found the information they needed in life. That was her focus when she was a graduate doctoral student at the University of California at Berkeley in the 1980s.
Later, as a visiting scholar at Stanford University, Head became more involved with the Web. Her interest in human-computer interaction was piqued when she realized how closely the fields of information-seeking behavior and human-computer interaction were related. With this focus in mind, she wrote a book, Design Wise, and started her own company, Alison J. Head & Associates, in1998. Hewlett Packard was her first client.
Most recently, Head finished a study (with Shannon Staley) analyzing the usability of intranets in seven different companies: Bechtel Corporation, Chevron Corporation (now ChevronTexaco Corporation), Fireman's Fund Insurance Company, Gale Group, Gilead Sciences, Sun Microsystems and Synopsys. The 110-page study, On-the-Job Research: How Usable are Corporate Research Intranets?, presents 32 usability findings and makes 24 design recommendations. It is now available through the Special Libraries Association. Head will also speak about her findings during a Hot Topic presentation at SLA's 2002 Annual Conference in Los Angeles.
"I thought it was important for a professional organization like SLA to have some visibility and presence in the assessment of research intranets," says Head. "The organization needs to move into this area and talk about how these new tools work, with the hope of creating more usable sites."
Information Outlook Assistant Editor Nikki Poling presents Head's thoughts on the study and its applications in the following interview. Afterward, you may need to ask yourself, "How do I begin the process of finding out how usable my corporate intranet is or is not?"
Nikki Poling (NP): What is a research intranet?
Alison Head (AH): There's really not a formal definition out there. For purposes of our study, we defined a research intranet as a collection of different online research resources. It includes both commercial databases and links on the Web, along with internal information-rich content, like company news and company reports. Research intranets aren't just confined to department sites. They are company-wide sites that are available to all employees. And they are, of course, secure and internally firewalled.
According to this definition, we found that there were different types of research intranets at companies. At some companies, we found huge gateway sites. At other companies, they were precision research tools with a collection of very expensive-to-use commercial databases and resources for market research. But actually, we found that most research intranets were a hybrid of these two main types.
The Why's and How's of the Research Study
NP: Why did you decide to do this particular usability study?
AH: My company had been doing a lot of work on information resources for the Web design evaluation and usability testing. The work was fascinating but I was limited by how much of these findings I could share with the rest of the information-professional and usability communities. The findings were proprietary and confidential they belonged to the company, not to me. Some companies were generous and let me share findings in conference presentations, but usually I only had enough time to present part of the picture. I thought, 'Wouldn't it be great to do a study where you could talk openly about impact usability beyond the company level.
So that was the origin of the idea. I also thought it would be great to see what different companies were doing with their intranets (intranet designs are usually safely guarded behind a company's firewall.) Intranets had been around since the mid-'90s. By 1996, two-thirds of companies already had sites up. And here it was five years later. I wanted to find out what those sites looked like and how they were actually working.
NP: Who participated in your study and why?
AH: The research design of our study required that we gain access to companies and then test with employees, using their actual firewalled research sites. No small feat. We started the recruiting phase by posting a general call for participation on the SLA listserv for the San Francisco Bay Area and other information professional listservs. Part of my thinking was that I had some visibility within the library community and there was some trust already established. I had taught at San Jose State's library school at one point, for about nine years and had done a lot of public speaking when my book, Design Wise, came out.
I also thought that information professionals and corporate librarians were the people most likely to be interested in our project. Librarians would find a way to get us inside their companies. Managers weren't going to let us in. Usability specialists weren't going to let us in. And developers probably weren't going to, either. But corporate librarians, because of the nature of the work they do and the focus of our research project, ended up being very open to the idea. We also directly recruited some corporate librarians, who worked at companies we wanted to include in our sample.
Even though our plans for recruiting were going well, we still had some hurdles to clear. What would the participating companies and the librarians who had volunteered as liaisons, as we call them in the study, get in return for letting us in? How could we thank them in a meaningful way? We ended up generating a separate usability report for each company, reporting findings about their site, in particular. So the liaisons ended up getting some unique research as part of the deal.
I do have to add that the liaisons made this project. They were committed, responsive and incredibly supportive. As liaisons, they had to recruit five people at their company a mixture of administrative assistants, managers and researchers to come in for testing. They also had to put their sites on the line. The understanding was that we were going to come in, evaluate the site and then publish generalized findings about the good, the bad and the ugly of corporate research sites.
One great irony of the project was that the last full day of testing was September 10th. Shannon, who worked on the project and did the data logging during the testing phase, walked away from that last day elated and exhausted. The next day, of course, the world changed forever. We finished just in time. I don't think this project could happen again not with the security in companies, not with the economy, not with how busy people are trying to keep their lives and businesses together these days.
The Intranet Evaluation Process
NP: How do companies normally evaluate their intranets? Do they bring in outside sources or do they evaluate in-house?
AH: It really varies. Let me run through some typical scenarios. In some cases, companies decide to conduct usability testing using someone from inside. Sometimes it's even the person who developed the site. This raises some issues, though. If someone has worked on the site, they may be too vested and too close to a site to objectively identify key usability problems and workable solutions.
In larger companies, like Sun and HP, employees have the luxury of contracting with their own company's usability department. But it doesn't always work that way. Sometimes these companies hire an outside usability expert because of time constraints, specialized expertise or a professional preference for an outside expert they already know.
Then there are a lot of companies that don't have usability resources, and they really don't have an expertise in usability, themselves. They often hire an outside expert like me. This has a couple of advantages. First, I am going to look at issues that are grounded in my expertise (information retrieval and information-seeking behavior). A client might not think to look at these kinds of issues. Second, users tend to be more candid with me, as an outsider, than they might be with a co-worker who was testing with them.
Finally, there are companies that want to do some usability testing but are not sure how to integrate usability into the development process. So they don't test at all. That's too bad. When this happens, the company is less likely to end up with a site that's in touch with users' needs. Someone at these companies may still do a lot of legwork and be devoted to collecting usage statistics and such, but the advantage of usability testing is that it can reap behavioral findings about what's really happening when a user sits down and tries to find something on a site. It can be very powerful stuff.
NP: How often do you think a company should evaluate a site?
AH: It really depends on the site and what the focus is, what the funding is, and what the needs are. If a company is gearing up for a redesign, then they should probably conduct a usability study of the old site to find out what works and what doesn't. If they have gone through a redesign in the last six months, they probably want to come back at some point and do more informal usability testing to find out how users are reacting to the new design as it starts to take hold.
I think the key point to remember is that usability evaluation needs to be iterative. You want it to be a continuing process. You don't want to conduct a usability test once and say, "We paid that usability expert to come in and now we're set. We never need to do change over time, technology changes and users' behaviors can change and fluctuate, too. So ongoing testing and evaluation are essential.
Why Intranets Fail
NP: You found that sites often fail. Why?
AH: Overall, research intranets fail on two counts. On one count, sites fail because of the design. On the other, they fail because the content offered doesn't support what users need and how they conduct research.
Let's start off with the design. Many of the sites we tested had an ineffective design for research work. For instance, the layout on some of the sites was cluttered with too many resources on one page. It was too overwhelming for users to absorb and process all of what they see. One participant in our testing sessions put it well. She said, "All of the information on this page just looks like a 'list of lists.' I know it's here, but I don't know where to start."
Another design weakness was labeling. The labeling on many research sites wasn't informative or intuitive. The buckets, or the categories, that resources were divided into often had overlapping labels that were too general or too vague for users to be able to distinguish between the different groupings. Also, research sites often used labels that were really library jargon. Predictably, few users understand these labels.
The other reason research intranets fail is that users lack the experience with using the research resources that sites feature. A lot of the sites suffered from overkill. In a way, a site with this kind of design is saying, "Come on down, we've got it all here for you Hoovers, Dun and Bradstreet, and access to Standard and Poor's Register on Dialog." Instead, users are thinking, "Boy, I just wanted to know how much one of our competitors made last year!"
In general, we found most users don't have the experience to efficiently use the different tools presented to them on sites. Or if they do, they don't want to waste too much time trying to get the answer. On top of being overwhelmed by a plethora of available resources to choose from, most users (other than librarians) are looking for a quick answer. So it follows that passwords and instructions on how to use different resources on research sites is death to any interaction. When users needed to put in a password for a resource in our sessions, most closed down the site and typed in a URL out on the Web so they could complete a task.
NP: What about graphics on intranets? Can they make a site easier to use?
AH: You have to remember that intranet users, even more than Web users, are focused on how an intranet can make them work better, smarter and faster. We found that there is not a lot of patience for graphics on intranets. Users have no appreciation for images that take a long time to download and that often don't impart any valuable information.
One of my favorite stories is about a test session I did several years ago on an intranet at HP. While one of the test participants and I waited for a page to load, she stopped, pulled her hands from the keyboard, looked at me, and said, "You know, if I get out of here in 10 minutes I can go watch my son's baseball game." And I thought, "Boy, what's that have to do with anything here?" And she said, "That's what I think of an intranet if it can save me 10 minutes, it means I have more time for my family."
The story's pretty insightful. Intranets are fundamentally work productivity tools. So a good design guideline to remember is that most intranet users don't want to be dazzled. They don't want to be entertained, they don't want to read promotional hype about the company they already work for, and they clearly don't want to see how effectively the designer can put together a three-minute flash introduction. The patience threshold for intranet users is pretty darn low. Think about it, though. Intranet users are also workers who have all these other things going at the same time they are traversing some company intranet. Users have ringing phones, e-mail coming in, deadlines, meetings in 10 minutes, interrupting co-workers, pending lunch plans and, in the case of the woman at HP, what they want to do after leaving work, too.
Common Problems with Labeling
NP: You said earlier that labeling was a real problem on the sites you tested. Can you expand on that a bit more?
AH: Labeling is always a key issue on any site, especially on information-rich sites. Labels need to match the way that users think a site should be divided up and what those buckets and the resources in them should be called. To create good labels, you've got to have a strong appreciation for what primary users need and want from a site, and for what kinds of tasks, and how they view the world.
Predictably, in our study, we found that labeling was a big usability problem on research intranets. One problem is that information professionals, who either build or do labeling as part of a development team, often label resources with library jargon without meaning to do this, of course. They tend to choose labels for resources based on who the producer or the author of the resource was. For most information professionals, the difference between Forrester and Gartner makes a certain amount of sense they are both market research resources with different areas of focus. But for the average user, this can be a no-man's land, a world that is unfamiliar to them. Users who run across this on a site are likely to be thinking, "Not this! All I wanted was a label that says 'find a market research report,' or even something like 'market research reports information appliance trends.' Do I have to read each one of these companies' reports now to find what I need?"
In his book, Don't Make Me Think, Steve Krug talks about how people make choices about where to click on a site. What he says applies to our study. Krug argues that users don't sit there weighing every possible option, and then make and choose the most optimal click. They glance, they second-guess and they take a chance on the first realistic option. They "satisfice," making a choice halfway between total satisfaction and being just adequate.
Does Usability Testing Really Occur in Companies?
NP: Did you find the usability of sites to be better for larger companies, since they often have more resources for conducting evaluation?
AH: Your question makes me think of Microsoft. They do more usability testing on their products than any other software company out there. But, of course, we know the end result for Microsoft hasn't necessarily led to incredibly usable products!
One company in our study, which we ended up profiling in the report, did some pretty rigorous information architecture research to establish the labeling system for their site. A corporate librarian at the company did card sorting to find out the level of granularity for labels, how labels would roll out from page to page, what sub-labels would correspond to what, and how new content would be worked in over time. She did the research on her own, without a usability group or anything too fancy. I've got to tell you, their site did really well in our testing, especially when it came to how users made their navigational choices. The labels were intuitive and robust.
NP: Did you find users were pretty vocal about the difficulties they experience with intranets?
AH: Yes. Most workers are very vocal about intranets when they don't work. And I don't mean vocal in a horrible, confrontational way but vocal in an honest, concerned and constructive way. Actually, I find that when users aren't as vocal during testing and have trouble completing the tasks we give them, they tend to be apologists for the site. They think it's not the site, it must be them and they just aren't good at using the Web. It's too bad. Apologists are never going to advance Web usability.
Overall, though, intranet users are vocal because they demand and expect a lot from research sites. Developers need to remember the consequences of that kind of demand, too. One of our findings was that intranet users will jump ship if a site doesn't seem to be working, and they think they won't be able to get what they need in a relatively short period of time. They will go to something that has worked for them in the past. Sometimes this is another site that will allow them to start searching all over again, like Google or Ask Jeeves. A lot of times, users told us they get very practical they will pick up a phone or lean over their carrel and ask a co-worker for the answer they need!
Putting the Usability Practices to Work
NP: What are the simple starting steps for people who may want to evaluate their intranet?
AH: I like to tell people about the 80/20 Rule. This means, quite simply, that 80 percent of the time, only 20 percent of a site is used. This rule turns a lot of heads around, because even though you realize all the content on a site may be important and dear to your own heart, in reality only about 20 percent of a site's content will be used with any regularity. That is a startling piece of information. Jared Spool has recently argued that it's probably even less than 20 percent. The upshot is, a very small part of the site is accessed over and over again for key tasks. In our report, we stress that intranet developers need to find out what part of the site that is, and ask whether it's easily accessed and whether it actually satisfies and meets the needs of users.
The 80/20 Rule is a fairly straightforward exercise and a good diagnostic. Usually, a truthful assessment will lead to several things that will impact usability. In particular, the site is likely to end up with far better information architecture, more user-centered labeling, a clearer layout and a better system for providing resources with the 20 percent that users actually need and use. This exercise can also be a foundation for developing a usability test tool. You start looking at a site differently when you start taking a task-particular focus and a user-centered focus, as opposed to a focus that is design-centric or totally content-focused.
On the Job Research: How Usable Are Research Corporate Intranets? by Alison J. Head with Shannon Staley, Alison J. Head & Associates, is available through the Special Libraries Association headquarters. The report may be ordered by fax or email. Read more about the study in the overview of major findings.