Cognitive walkthrough

Last updated

The cognitive walkthrough method is a usability inspection method used to identify usability issues in interactive systems, focusing on how easy it is for new users to accomplish tasks with the system. A cognitive walkthrough is task-specific, whereas heuristic evaluation takes a holistic view to catch problems not caught by this and other usability inspection methods. The method is rooted in the notion that users typically prefer to learn a system by using it to accomplish tasks, rather than, for example, studying a manual. The method is prized for its ability to generate results quickly with low cost, especially when compared to usability testing, as well as the ability to apply the method early in the design phases before coding even begins (which happens less often with usability testing).

Contents

Introduction

A cognitive walkthrough starts with a task analysis that specifies the sequence of steps or actions required by a user to accomplish a task, and the system responses to those actions. The designers and developers of the software then walk through the steps as a group, asking themselves a set of questions at each step. Data is gathered during the walkthrough, and afterwards a report of potential issues is compiled. Finally the software is redesigned to address the issues identified.

The effectiveness of methods such as cognitive walkthroughs is hard to measure in applied settings, as there is very limited opportunity for controlled experiments while developing software. Typically measurements involve comparing the number of usability problems found by applying different methods. However, Gray and Salzman called into question the validity of those studies in their dramatic 1998 paper "Damaged Merchandise", demonstrating how very difficult it is to measure the effectiveness of usability inspection methods. The consensus in the usability community is that the cognitive walkthrough method works well in a variety of settings and applications.

Streamlined cognitive walkthrough procedure

After the task analysis has been made, the participants perform the walkthrough: [1]

  1. Define inputs to the walkthrough: a usability specialist lays out the scenarios and produces an analysis of said scenarios through explanation of the actions required to accomplish the task.
    1. Identify users
    2. Create a sample task for evaluation
    3. Create action sequences for completing the tasks
    4. Implementation of interface
  2. Convene the walkthrough:
    1. What are the goals of the walkthrough?
    2. What will be done during the walkthrough
    3. What will not be done during the walkthrough
    4. Post ground rules
      1. Some common ground rules
        1. No designing
        2. No defending a design
        3. No debating cognitive theory
        4. The usability specialist is the leader of the session
    5. Assign roles
    6. Appeal for submission to leadership
  3. Walk through the action sequences for each task
    1. Participants perform the walkthrough by asking themselves a set of questions for each subtask. Typically four questions are as
      • Will the user try to achieve the effect that the subtask has? E.g. Does the user understand that this subtask is needed to reach the user's goal
      • Will the user notice that the correct action is available? E.g. is the button visible?
      • Will the user understand that the wanted subtask can be achieved by the action? E.g. the right button is visible but the user does not understand the text and will therefore not click on it.
      • Does the user get appropriate feedback? Will the user know that they have done the right thing after performing the action?
    2. By answering the questions for each subtask usability problems will be noticed.
  4. Record any important information
    1. Learnability problems
    2. Design ideas and gaps
    3. Problems with analysis of the task
  5. Revise the interface using what was learned in the walkthrough to improve the problems.

The CW method does not take several social attributes into account. The method can only be successful if the usability specialist takes care to prepare the team for all possibilities during the cognitive walkthrough. This tends to enhance the ground rules and avoid the pitfalls that come with an ill-prepared team.

Common shortcomings

In teaching people to use the walkthrough method, Lewis & Rieman have found that there are two common misunderstandings: [2]

  1. The evaluator doesn't know how to perform the task themself, so they stumble through the interface trying to discover the correct sequence of actions—and then they evaluate the stumbling process. (The user should identify and perform the optimal action sequence.)
  2. The walkthrough method does not test real users on the system. The walkthrough will often identify many more problems than you would find with a single, unique user in a single test session

There are social constraints that inhibit the cognitive walkthrough process. These include time pressure, lengthy design discussions and design defensiveness. Time pressure is caused when design iterations occur late in the development process, when a development team usually feels considerable pressure to actually implement specifications, and may not think they have the time to evaluate them properly. Many developers feel that CW's are not efficient because of the amount of time they take and the time pressures that they are facing. A design team spends their time trying to resolve the problem, during the CW instead of after the results have been formulated. Evaluation time is spent re-designing, this inhibits the effectiveness of the walkthrough and leads to lengthy design discussions. Many times, designers may feel personally offended that their work is even being evaluated. Due to the fact that a walk-through would likely lead to more work on a project that they already are under pressure to complete in the allowed time, designers will over-defend their design during the walkthrough. They are more likely to be argumentative and reject changes that seem obvious.

History

The method was developed in the early nineties by Wharton, et al., and reached a large usability audience when it was published as a chapter in Jakob Nielsen's seminal book on usability, "Usability Inspection Methods". [3] The Wharton, et al. method required asking four questions at each step, along with extensive documentation of the analysis. In 2000 there was a resurgence in interest in the method in response to a CHI paper by Spencer who described modifications to the method to make it effective in a real software development setting. Spencer's streamlined method required asking only two questions at each step, and involved creating less documentation. Spencer's paper followed the example set by Rowley, et al. who described the modifications to the method that they made based on their experience applying the methods in their 1992 CHI paper "The Cognitive Jogthrough". [4]

Originally designed as a tool to evaluate interactive systems, such as postal kiosks, automated teller machines (ATMs), and interactive museum exhibits, where users would have little to no experience with using this new technology. However, since its creation, the method has been applied with success to complex systems like CAD software and some software development tools to understand the first experience of new users.

See also

Related Research Articles

Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.

Usability engineering is a professional discipline that focuses on improving the usability of interactive systems. It draws on theories from computer science and psychology to define problems that occur during the use of such a system. Usability Engineering involves the testing of designs at various stages of the development process, with users or with usability experts. The history of usability engineering in this context dates back to the 1980s. In 1988, authors John Whiteside and John Bennett—of Digital Equipment Corporation and IBM, respectively—published material on the subject, isolating the early setting of goals, iterative evaluation, and prototyping as key activities. The usability expert Jakob Nielsen is a leader in the field of usability engineering. In his 1993 book Usability Engineering, Nielsen describes methods to use throughout a product development process—so designers can ensure they take into account the most important barriers to learnability, efficiency, memorability, error-free use, and subjective satisfaction before implementing the product. Nielsen’s work describes how to perform usability tests and how to use usability heuristics in the usability engineering lifecycle. Ensuring good usability via this process prevents problems in product adoption after release. Rather than focusing on finding solutions for usability problems—which is the focus of a UX or interaction designer—a usability engineer mainly concentrates on the research phase. In this sense, it is not strictly a design role, and many usability engineers have a background in computer science because of this. Despite this point, its connection to the design trade is absolutely crucial, not least as it delivers the framework by which designers can work so as to be sure that their products will connect properly with their target usership.

A think-aloudprotocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences.

<span class="mw-page-title-main">Usability</span> Capacity of a system for its users to perform tasks

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles. These evaluation methods are now widely taught and practiced in the new media sector, where user interfaces are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing.

User-centered design (UCD) or user-driven development (UDD) is a framework of process in which usability goals, user characteristics, environment, tasks and workflow of a product, service or process are given extensive attention at each stage of the design process. These tests are conducted with/without actual users during each stage of the process from requirements, pre-production models and post production, completing a circle of proof back to and ensuring that "development proceeds with the user as the center of focus." Such testing is necessary as it is often very difficult for the designers of a product to understand intuitively the first-time users of their design experiences, and what each user's learning curve may look like. User-centered design is based on the understanding of a user, their demands, priorities and experiences and when used, is known to lead to an increased product usefulness and usability as it delivers satisfaction to the user. User-centered design applies cognitive science principles to create intuitive, efficient products by understanding users' mental processes, behaviors, and needs.

Interaction design, often abbreviated as IxD, is "the practice of designing interactive digital products, environments, systems, and services." While interaction design has an interest in form, its main area of focus rests on behavior. Rather than analyzing how things are, interaction design synthesizes and imagines things as they could be. This element of interaction design is what characterizes IxD as a design field, as opposed to a science or engineering field.

The following outline is provided as an overview of and topical guide to human–computer interaction:

Task analysis is a fundamental tool of human factors engineering. It entails analyzing how a task is accomplished, including a detailed description of both manual and mental activities, task and element durations, task frequency, task allocation, task complexity, environmental conditions, necessary clothing and equipment, and any other unique factors involved in or required for one or more people to perform a given task.

GOMS is a specialized human information processor model for human-computer interaction observation that describes a user's cognitive structure on four components. In the book The Psychology of Human Computer Interaction. written in 1983 by Stuart K. Card, Thomas P. Moran and Allen Newell, the authors introduce: "a set of Goals, a set of Operators, a set of Methods for achieving the goals, and a set of Selections rules for choosing among competing methods for goals." GOMS is a widely used method by usability specialists for computer system designers because it produces quantitative and qualitative predictions of how people will use a proposed system.

<span class="mw-page-title-main">User interface design</span> Planned operator–machine interaction

User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and the user experience. In computer or software design, user interface (UI) design primarily focuses on information architecture. It is the process of building interfaces that clearly communicate to the user what's important. UI design refers to graphical user interfaces and other forms of interface design. The goal of user interface design is to make the user's interaction as simple and efficient as possible, in terms of accomplishing user goals.

Cognitive ergonomics is a scientific discipline that studies, evaluates, and designs tasks, jobs, products, environments and systems and how they interact with humans and their cognitive abilities. It is defined by the International Ergonomics Association as "concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Cognitive ergonomics is responsible for how work is done in the mind, meaning, the quality of work is dependent on the persons understanding of situations. Situations could include the goals, means, and constraints of work. The relevant topics include mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress and training as these may relate to human-system design." Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance. It is a subset of the larger field of human factors and ergonomics.

Human processor model or MHP is a cognitive modeling method developed by Stuart K. Card, Thomas P. Moran, & Allen Newell (1983) used to calculate how long it takes to perform a certain task. Other cognitive modeling methods include parallel design, GOMS, and keystroke-level model (KLM).

Software security assurance is a process that helps design and implement software that protects the data and resources contained in and controlled by that software. Software is itself a resource and thus must be afforded appropriate security.

Cognitive dimensions or cognitive dimensions of notations are design principles for notations, user interfaces and programming languages, described by researcher Thomas R.G. Green and further researched with Marian Petre. The dimensions can be used to evaluate the usability of an existing information artifact, or as heuristics to guide the design of a new one, and are useful in Human-Computer Interaction design.

User experience design defines the experience a user would go through when interacting with a company, its services, and its products. User experience design is a user centered design approach because it considers the user's experience when using a product or platform. Research, data analysis, and test results drive design decisions in UX design rather than aesthetic preferences and opinions. Unlike user interface design, which focuses solely on the design of a computer interface, UX design encompasses all aspects of a user's perceived experience with a product or website, such as its usability, usefulness, desirability, brand perception, and overall performance. UX design is also an element of the customer experience (CX), and encompasses all aspects and stages of a customer's experience and interaction with a company.

The pluralistic walkthrough is a usability inspection method used to identify usability issues in a piece of software or website in an effort to create a maximally usable human-computer interface. The method centers on recruiting a group of users, developers and usability professionals to step through a task scenario, discussing usability issues associated with dialog elements involved in the scenario steps. The group of experts used is asked to assume the role of typical users in the testing. The method is prized for its ability to be utilized at the earliest design stages, enabling the resolution of usability issues quickly and early in the design process. The method also allows for the detection of a greater number of usability problems to be found at one time due to the interaction of multiple types of participants. This type of usability inspection method has the additional objective of increasing developers’ sensitivity to users’ concerns about the product design.

Usability inspection is the name for a set of methods where an evaluator inspects a user interface. This is in contrast to usability testing where the usability of the interface is evaluated by testing it on real users. Usability inspections can generally be used early in the development process by evaluating prototypes or specifications for the system that can't be tested on users. Usability inspection methods are generally considered to be less costly to implement than testing on users.

<span class="mw-page-title-main">Interaction technique</span>

An interaction technique, user interface technique or input technique is a combination of hardware and software elements that provides a way for computer users to accomplish a single task. For example, one can go back to the previously visited page on a Web browser by either clicking a button, pressing a key, performing a mouse gesture or uttering a speech command. It is a widely used term in human-computer interaction. In particular, the term "new interaction technique" is frequently used to introduce a novel user interface design idea.

Tools, devices or software must be evaluated before their release on the market from different points of view such as their technical properties or their usability. Usability evaluation allows assessing whether the product under evaluation is efficient enough, effective enough and sufficiently satisfactory for the users. For this assessment to be objective, there is a need for measurable goals that the system must achieve. That kind of goal is called a usability goal. They are objective criteria against which the results of the usability evaluation are compared to assess the usability of the product under evaluation.

References

  1. Spencer, Rick (2000). "The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company". Proceedings of the SIGCHI conference on Human Factors in Computing Systems. The Hague, The Netherlands: ACM Press. pp. 353–359. doi:10.1145/332040.332456. ISBN   978-1-58113-216-8. S2CID   1157974.
  2. Lewis, Clayton; Rieman, John (1994). "Section 4.1: Cognitive Walkthroughs". Task-Centered User Interface Design: A Practical Introduction. pp. 46–54. Retrieved April 10, 2019.
  3. Wharton, Cathleen; Riemann, John; Lewis, Clayton; Poison, Peter (June 1994). "The cognitive walkthrough method: a practitioner's guide". In Nielsen, Jakob; Mack, Robert L. (eds.). Usability inspection methods. John Wiley & Sons. pp. 105–140. ISBN   978-0-471-01877-3 . Retrieved 2020-02-11.{{cite book}}: |website= ignored (help)
  4. Rowley, David E; Rhoades, David G (1992). "The cognitive jogthrough: A fast-paced user interface evaluation procedure". Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '92. pp. 389–395. doi:10.1145/142750.142869. ISBN   0897915135. S2CID   15888065.

Further reading