When developing hierarchies for information-rich Web sites, designers and usability researchers often turn to card sorting for help in making design decisions. Card sorting offers a systematic and statistically significant process for answering questions about hierarchy design. However, those of us who have run card sorts know there is an art to conducting successful card sort studies, and there are many variables that can affect the usefulness of results. In this column, I’ll discuss the challenges and limitations of card sorting and review alternative and complementary techniques that designers can leverage when developing an information hierarchy for a large-scale Web site.
Challenges of Card Sorting
Part of the appeal of card sorting today is that researchers have the option of conducting studies either online or in person. When using an online tool, large numbers of participants can complete the exercise, lending additional statistical weight to the findings. In some organizations, the large sample size and statistical basis of online card sorting is helpful in dealing with decision makers. In-person card sorts let researchers interact with participants and ask probing questions to determine their organizational strategies, as well as other follow-up questions. A number of resources are available online that provide detailed steps on running and analyzing studies in both contexts.
Sounds easy, right? Just load your content into a card sort application or create actual cards, have participants complete the exercise, and the right organization for your site will be revealed! Unfortunately, it is not usually quite that easy. As Sam Ng wrote in an article on UXmatters, there are a number of issues to consider when conducting a study, from the timing of a card sort to setting proper expectations with stakeholders. Two of the points Sam mentioned in his article ring true and concur with my experience:
Run multiple studies. Designers can expect participants to sort a maximum of 80–100 cards during any one study. For sites with a large volume of content—such as intranets, retail stores, or research portals—such a limited number of cards may not accurately reflect the full breadth of content on the site. For such a site, you’ll need multiple studies to understand each individual level or section of the site. Running multiple studies adds significantly to the time and effort completing the process requires, and you should account for this when planning. Count on doing some subjective analysis. Although you can apply statistical methods to results, you’ll still need a certain level of subjective analysis to interpret the intent of card sort participants, as well as the consequences of content relationships. As a result of this necessary subjectivity, designers may have different interpretations of the same results, depending on their depth of domain knowledge and previous experience with the content. User researchers have proposed several analysis templates, involving spreadsheets and visualizations in place of dendrograms, but their analysis still involves a level of subjectivity. In addition to the points Sam Ng raised, I’ve found some other challenges to conducting successful card sort studies.
selecting and naming cards—Creating the cards for an online sort is very difficult in some domains. For physical objects, the process is easier. Choosing objects that have universally recognized names—and, potentially, using images—helps ensure respondents will understand the cards. But on an informational Web site, complex pieces of content are more difficult to describe, so you must make significant effort to ensure respondents will interpret the names of cards consistently, without introducing bias to the groupings participants will create. labeling groups—One of the most important elements of a site hierarchy is the labeling for each category or menu item. While open card sorting can provide insights to users’ view of content relationships, designers should not necessarily expect open card sorting to provide useful solutions for the names or labels of the resulting content groups. During a card-sorting exercise, participants give names to the groups they create, but each participant may create groups of varying sizes, with different intents. As a result, looking for trends among group names from different participants is difficult and may not yield helpful insights. While closed card sorting lets you test group labels, you can test only one set of labels at a time, and it does not allow any interplay between the labels for multiple levels of the hierarchy.
Color theorists and designers in fashion or computer graphics have coined phrases based around what colors shouldn't go together. A recent forum post, Red and Green Should Not Be Seen? discusses two of the sayings ("Red and Green Should Not Be Seen" and "Blue and Green Should Never Be Seen Without Something in Between"), and some lovers have even responded in protest, showing how there is no 'wrong' in love. Here are some palettes and applications of the forbidden colors that really work.
