Card Sorting: A Tool to Reorganize Information-Rich Web Sites

By TMCnet Special Guest

Michael Hawley, User Experience Director, Mad*Pow

 

If your company designed an information-rich Web site such as an intranet or content portal a few years ago, chances are the site could be bursting at the seams. The types of information or messages that get put on the Web site have likely changed and increased in number. Also, the deployment of content management systems may have opened up content creation to a wide team of content providers. Initially, the information hierarchy, or organization, developed for the site might have accommodated the additional content, but content can quickly outgrow the existing structure as new initiatives and information needs are added. The result is a Web site that is overcrowded with new information organized in an old way. Frequently, users of these sites will complain that the site is out of date, and information they need is hard to find.

 

Improving the search capability on the site is one way to help visitors find the information they need. Reviewing search logs, improving search engine relevancy and configuring narrowing and filtering options on the search results page can help those visitors who search for keywords that they know. However, the information hierarchy or table of contents of the site is also important. First, some visitors don’t necessarily know exactly what they are looking for and rely on the information hierarchy to guide them. Second, the mere presence of the information hierarchy can help orient all users to the content and purpose on the site.

 

Organizing all of the new content into an intuitive information hierarchy can be a daunting task. Potentially, hundreds or thousands of pages need to be organized and labeled in such a way that different types of people can find information. Fortunately, several user-centered design activities can help this process. Of these activities, card sorting is an appealing option.

 

Card sorting gets its name from the cards that are used to conduct an in-person sort. In a research setting, research participants are presented a series of cards on a table and asked to sort them into piles or groups. The cards include a title or description of the content to be sorted, whether it is products, functions on a Web site, brands, etc. The process can be conducted in two ways. First, participants can group cards as they see fit, with no guidelines as to the number of groups. Statistical methods such as cluster analysis are then applied to the results of multiple participants and viewed in spreadsheet or tree diagram form. This process, called open card sorting, reveals insights into relationships between the cards. Based on this information, a proposed information hierarchy is developed.

 

Second, in a process called closed card sorting, participants are asked to sort the cards into a set of pre-defined groups based on the proposed hierarchy. Analysis is then conducted to determine the percentage of time participants placed a card in the target group. Closed card sorting is often used after open sorting to validate that proposed groups work as intended.

 

Part of the appeal of card sorting now is that both open and closed card sorting can be conducted online. The advantage of this is that a large number of participants can complete the exercise lending additional statistical weight to the findings. In some business cultures, the large sample size of online card sorting is helpful when dealing with decision makers. Several vendors offer online applications that allow you to create conduct and analyze online card sort studies for a very low cost.

 

Sounds easy, right? Just load your content into an online card sort application, have participants complete the exercise, and the right organization for your site will be revealed! Unfortunately, it is not necessarily that easy. It is very easy to be overwhelmed and led down the wrong path if the study is not conducted correctly. Based on my experience with card sorting, the following are several points that you should consider:

 

If you are re-organizing a current site, chances are there are several levels of content to organize. Select the levels that you want to focus on, and then choose content from one level below to include in the card sort. When you choose content from one level, the resulting groups from an open card sort will likely reveal the structure for the level above. For a particular study, choose all content from the same level so items are sorted consistently.

 

Generally for reorganizing large sites, conducting one card sort study is not sufficient. There may be several levels of content to organize and several sections with a large amount of content. It would be nice to load all of the content to organize into one study and have participants group it. However, participating in a card sort study takes time and thought. Additionally, there is a limit to how many pieces of content or cards a participant can manage in the organization exercise. The guideline per study is 60–75 cards. Anything over 80–100 cards is overwhelming for participants, and the reliability of the data would be questionable. Choose representative content from the site that reflects the areas you need to organize, but be prepared to run multiple studies so you can cover all of the sections and levels of the site.

 

The two card sorting methods, open and closed card sorting, produce different types of data about the content you are organizing, and both are valuable. The open card sorting should be conducted first to develop an understanding of the relationship between different content and inform a proposed grouping. It is open-ended and you may see some findings that surprise you. After analyzing the open sort and developing a proposed grouping, you can include an interim step that includes a survey to ask participants for names of your proposed groups. Following the naming survey, apply group names from the survey to proposed groups and run a closed card sort exercise to validate that users consistently interpret the group names and groupings as you intended. As you can start to see, the process of card sorting to re-organize a large site can include studies, and require access to the user base on a frequent basis.

 

There are a number of ways to analyze card sort data. Specific documentation is available from vendors or from researchers who have experimented with different techniques. When you start the data analysis, realize that the results will not be black-and-white. There is a fair amount of subjectivity in interpreting the data — domain knowledge and previous experience with analyzing card sorts is helpful. The best way to deal with this is to plan to spend time with the data and employ several of the analysis methods to give you several perspectives on the results.

 

At the outset of a re-organization process, project teams are generally excited about the potential for card sorting. It is a research method that is seemingly designed for exactly that purpose. However, it is important to remember some of the limitations of card sorting.

 

First, communicating the results of a study is difficult. Cluster analysis tree diagrams and large spreadsheets are not easily summarized in an executive summary. It is best to know your audience and consider a presentation of the final results. Second, when card sort studies are conducted online, you don’t get the advantage of verbal feedback from the participant. Often, users of the site can describe their ideal organization for the site better than they can represent it in a structured card sorting exercise. Finally, and perhaps most importantly, it is important to remember that the analysis of both open and closed card sort results represents the aggregate of all the data collected — it does not necessarily reflect the results or organizing pattern of any one individual. The aggregate or average grouping is important to know, but you need to be careful that individual differences in approaches to content organization are not lost.

 

Card sorting is an appealing research technique to assist in the organization or re-organization of an information-rich Web site. But conducting the sorts correctly requires thoughtful planning and can demand a fair amount of resources. By employing card sorting techniques along with other user-centered design methods such as usability testing, contextual inquires and surveys, you can reveal insights to help you develop an intuitive information hierarchy for your site.