Cognitive walkthrough

Last updated

The cognitive walkthrough method is a usability inspection method used to identify usability issues in interactive systems, focusing on how easy it is for new users to accomplish tasks with the system. A cognitive walkthrough is task-specific, whereas heuristic evaluation takes a holistic view to catch problems not caught by this and other usability inspection methods. The method is rooted in the notion that users typically prefer to learn a system by using it to accomplish tasks, rather than, for example, studying a manual. The method is prized for its ability to generate results quickly with low cost, especially when compared to usability testing, as well as the ability to apply the method early in the design phases before coding even begins (which happens less often with usability testing).

Contents

Introduction

A cognitive walkthrough starts with a task analysis that specifies the sequence of steps or actions required by a user to accomplish a task, and the system responses to those actions. The designers and developers of the software then walk through the steps as a group, asking themselves a set of questions at each step. Data is gathered during the walkthrough, and afterwards a report of potential issues is compiled. Finally the software is redesigned to address the issues identified.

The effectiveness of methods such as cognitive walkthroughs is hard to measure in applied settings, as there is very limited opportunity for controlled experiments while developing software. Typically measurements involve comparing the number of usability problems found by applying different methods. However, Gray and Salzman called into question the validity of those studies in their dramatic 1998 paper "Damaged Merchandise", demonstrating how very difficult it is to measure the effectiveness of usability inspection methods. The consensus in the usability community is that the cognitive walkthrough method works well in a variety of settings and applications.

Streamlined cognitive walkthrough procedure

After the task analysis has been made, the participants perform the walkthrough: [1]

  1. Define inputs to the walkthrough: a usability specialist lays out the scenarios and produces an analysis of said scenarios through explanation of the actions required to accomplish the task.
    1. Identify users
    2. Create a sample task for evaluation
    3. Create action sequences for completing the tasks
    4. Implementation of interface
  2. Convene the walkthrough:
    1. What are the goals of the walkthrough?
    2. What will be done during the walkthrough
    3. What will not be done during the walkthrough
    4. Post ground rules
      1. Some common ground rules
        1. No designing
        2. No defending a design
        3. No debating cognitive theory
        4. The usability specialist is the leader of the session
    5. Assign roles
    6. Appeal for submission to leadership
  3. Walk through the action sequences for each task
    1. Participants perform the walkthrough by asking themselves a set of questions for each subtask. Typically four questions are as
      • Will the user try to achieve the effect that the subtask has? E.g. Does the user understand that this subtask is needed to reach the user's goal
      • Will the user notice that the correct action is available? E.g. is the button visible?
      • Will the user understand that the wanted subtask can be achieved by the action? E.g. the right button is visible but the user does not understand the text and will therefore not click on it.
      • Does the user get appropriate feedback? Will the user know that they have done the right thing after performing the action?
    2. By answering the questions for each subtask usability problems will be noticed.
  4. Record any important information
    1. Learnability problems
    2. Design ideas and gaps
    3. Problems with analysis of the task
  5. Revise the interface using what was learned in the walkthrough to improve the problems.

The CW method does not take several social attributes into account. The method can only be successful if the usability specialist takes care to prepare the team for all possibilities during the cognitive walkthrough. This tends to enhance the ground rules and avoid the pitfalls that come with an ill-prepared team.

Common shortcomings

In teaching people to use the walkthrough method, Lewis & Rieman have found that there are two common misunderstandings: [2]

  1. The evaluator doesn't know how to perform the task themself, so they stumble through the interface trying to discover the correct sequence of actions—and then they evaluate the stumbling process. (The user should identify and perform the optimal action sequence.)
  2. The walkthrough method does not test real users on the system. The walkthrough will often identify many more problems than you would find with a single, unique user in a single test session

There are social constraints that inhibit the cognitive walkthrough process. These include time pressure, lengthy design discussions and design defensiveness. Time pressure is caused when design iterations occur late in the development process, when a development team usually feels considerable pressure to actually implement specifications, and may not think they have the time to evaluate them properly. Many developers feel that CW's are not efficient because of the amount of time they take and the time pressures that they are facing. A design team spends their time trying to resolve the problem, during the CW instead of after the results have been formulated. Evaluation time is spent re-designing, this inhibits the effectiveness of the walkthrough and leads to lengthy design discussions. Many times, designers may feel personally offended that their work is even being evaluated. Due to the fact that a walk-through would likely lead to more work on a project that they already are under pressure to complete in the allowed time, designers will over-defend their design during the walkthrough. They are more likely to be argumentative and reject changes that seem obvious.

History

The method was developed in the early nineties by Wharton, et al., and reached a large usability audience when it was published as a chapter in Jakob Nielsen's seminal book on usability, "Usability Inspection Methods". [3] The Wharton, et al. method required asking four questions at each step, along with extensive documentation of the analysis. In 2000 there was a resurgence in interest in the method in response to a CHI paper by Spencer who described modifications to the method to make it effective in a real software development setting. Spencer's streamlined method required asking only two questions at each step, and involved creating less documentation. Spencer's paper followed the example set by Rowley, et al. who described the modifications to the method that they made based on their experience applying the methods in their 1992 CHI paper "The Cognitive Jogthrough". [4]

Originally designed as a tool to evaluate interactive systems, such as postal kiosks, automated teller machines (ATMs), and interactive museum exhibits, where users would have little to no experience with using this new technology. However, since its creation, the method has been applied with success to complex systems like CAD software and some software development tools to understand the first experience of new users.

See also

Related Research Articles

Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.

Usability engineering is a field that is concerned generally with human–computer interaction and specifically with devising human–computer interfaces that have high usability or user friendliness. It provides structured methods for achieving efficiency and elegance in interface design.

A think-aloudprotocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences.

<span class="mw-page-title-main">Usability</span> Capacity of a system for its users to perform tasks

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles. These evaluation methods are now widely taught and practiced in the new media sector, where user interfaces are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing.

Interaction design, often abbreviated as IxD, is "the practice of designing interactive digital products, environments, systems, and services." While interaction design has an interest in form, its main area of focus rests on behavior. Rather than analyzing how things are, interaction design synthesizes and imagines things as they could be. This element of interaction design is what characterizes IxD as a design field, as opposed to a science or engineering field.

The following outline is provided as an overview of and topical guide to human–computer interaction:

Task analysis is a fundamental tool of human factors engineering. It entails analyzing how a task is accomplished, including a detailed description of both manual and mental activities, task and element durations, task frequency, task allocation, task complexity, environmental conditions, necessary clothing and equipment, and any other unique factors involved in or required for one or more people to perform a given task.

GOMS is a specialized human information processor model for human-computer interaction observation that describes a user's cognitive structure on four components. In the book The Psychology of Human Computer Interaction. written in 1983 by Stuart K. Card, Thomas P. Moran and Allen Newell, the authors introduce: "a set of Goals, a set of Operators, a set of Methods for achieving the goals, and a set of Selections rules for choosing among competing methods for goals." GOMS is a widely used method by usability specialists for computer system designers because it produces quantitative and qualitative predictions of how people will use a proposed system.

<span class="mw-page-title-main">User interface design</span> Planned operator–machine interaction

User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and the user experience. In computer or software design, user interface (UI) design primarily focuses on information architecture. It is the process of building interfaces that clearly communicates to the user what's important. UI design refers to graphical user interfaces and other forms of interface design. The goal of user interface design is to make the user's interaction as simple and efficient as possible, in terms of accomplishing user goals.

Cognitive ergonomics is a scientific discipline that studies, evaluates, and designs tasks, jobs, products, environments and systems and how they interact with humans and their cognitive abilities. It is defined by the International Ergonomics Association as "concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Cognitive ergonomics is responsible for how work is done in the mind, meaning, the quality of work is dependent on the persons understanding of situations. Situations could include the goals, means, and constraints of work. The relevant topics include mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress and training as these may relate to human-system design." Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance. It is a subset of the larger field of human factors and ergonomics.

Exploratory search is a specialization of information exploration which represents the activities carried out by searchers who are:

In software engineering, a walkthrough or walk-through is a form of software peer review "in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems". The reviews are also performed by assessors, specialists, etc. and are suggested or mandatory as required by norms and standards.

<span class="mw-page-title-main">Software security assurance</span>

Software security assurance is a process that helps design and implement software that protects the data and resources contained in and controlled by that software. Software is itself a resource and thus must be afforded appropriate security.

Cognitive dimensions or cognitive dimensions of notations are design principles for notations, user interfaces and programming languages, described by researcher Thomas R.G. Green and further researched with Marian Petre. The dimensions can be used to evaluate the usability of an existing information artifact, or as heuristics to guide the design of a new one, and are useful in Human-Computer Interaction design.

The pluralistic walkthrough is a usability inspection method used to identify usability issues in a piece of software or website in an effort to create a maximally usable human-computer interface. The method centers on recruiting a group of users, developers and usability professionals to step through a task scenario, discussing usability issues associated with dialog elements involved in the scenario steps. The group of experts used is asked to assume the role of typical users in the testing. The method is prized for its ability to be utilized at the earliest design stages, enabling the resolution of usability issues quickly and early in the design process. The method also allows for the detection of a greater number of usability problems to be found at one time due to the interaction of multiple types of participants. This type of usability inspection method has the additional objective of increasing developers’ sensitivity to users’ concerns about the product design.

Usability inspection is the name for a set of methods where an evaluator inspects a user interface. This is in contrast to usability testing where the usability of the interface is evaluated by testing it on real users. Usability inspections can generally be used early in the development process by evaluating prototypes or specifications for the system that can't be tested on users. Usability inspection methods are generally considered to be less costly to implement than testing on users.

Tools, devices or software must be evaluated before their release on the market from different points of view such as their technical properties or their usability. Usability evaluation allows assessing whether the product under evaluation is efficient enough, effective enough and sufficiently satisfactory for the users. For this assessment to be objective, there is a need for measurable goals that the system must achieve. That kind of goal is called a usability goal. They are objective criteria against which the results of the usability evaluation are compared to assess the usability of the product under evaluation.

<span class="mw-page-title-main">Human factors and ergonomics</span> Designing systems to suit their users

Human factors and ergonomics is the application of psychological and physiological principles to the engineering and design of products, processes, and systems. Primary goals of human factors engineering are to reduce human error, increase productivity and system availability, and enhance safety, health and comfort with a specific focus on the interaction between the human and equipment.

<span class="mw-page-title-main">Hardware interface design</span>

Hardware interface design (HID) is a cross-disciplinary design field that shapes the physical connection between people and technology in order to create new hardware interfaces that transform purely digital processes into analog methods of interaction. It employs a combination of filmmaking tools, software prototyping, and electronics breadboarding.

References

  1. ↑ Spencer, Rick (2000). "The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company". Proceedings of the SIGCHI conference on Human Factors in Computing Systems. The Hague, The Netherlands: ACM Press. pp. 353–359. doi:10.1145/332040.332456. ISBN   978-1-58113-216-8. S2CID   1157974.
  2. ↑ Lewis, Clayton; Rieman, John (1994). "Section 4.1: Cognitive Walkthroughs". Task-Centered User Interface Design: A Practical Introduction. pp. 46–54. Retrieved April 10, 2019.
  3. ↑ Wharton, Cathleen; Riemann, John; Lewis, Clayton; Poison, Peter (June 1994). "The cognitive walkthrough method: a practitioner's guide". In Nielsen, Jakob; Mack, Robert L. (eds.). Usability inspection methods. dl.acm.org. John Wiley & Sons. pp. 105–140. ISBN   978-0-471-01877-3 . Retrieved 2020-02-11.
  4. ↑ Rowley, David E; Rhoades, David G (1992). "The cognitive jogthrough: a fast-paced user interface evaluation procedure". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: 389–395. doi:10.1145/142750.142869. S2CID   15888065.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.