A think-aloud (or thinking aloud) protocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences (e.g., reading, writing, translation research, decision making, and process tracing).
Think-aloud protocols involve participants thinking aloud as they are performing a set of specified tasks. Participants are asked to say whatever comes into their mind as they complete the task. This might include what they are looking at, thinking, doing, and feeling. This gives observers insight into the participant's cognitive processes (rather than only their final product), to make thought processes as explicit as possible during task performance. In a formal research protocol, all verbalizations are transcribed and then analyzed. In a usability testing context, observers are asked to take notes of what participants say and do, without attempting to interpret their actions and words, and especially noting places where they encounter difficulty. Test sessions may be completed on participants own devices or in a more controlled setting. [1] Sessions are often audio- and video-recorded so that developers can go back and refer to what participants did and how they reacted. [2]
The think-aloud method was introduced in the usability field by Clayton Lewis [3] while he was at IBM, and is explained in Task-Centered User Interface Design: A Practical Introduction by Lewis and John Rieman. [4] The method was developed based on the techniques of protocol analysis by K. Ericsson and H. Simon. [5] [6] [7] However, there are some significant differences between the way Ericsson and Simon propose that protocols be conducted and how they are actually conducted by usability practitioners, as noted by Ted Boren and Judith Ramey. [8] These differences arise from the specific needs and context of usability testing; practitioners should be aware of these differences and adjust their method to meet their needs while still collecting valid data. For example, they may need to prompt for additional information more often than Ericsson and Simon would allow, but should take care not to influence what participants say and do.
A typical procedure of think-aloud protocols would include:
As Kuusela and Paul [10] state, the think-aloud protocol can be distinguished into two different types of experimental procedures. The first is the concurrent think-aloud protocol, collected during the task. The second is the retrospective think-aloud protocol, gathered after the task as the participant walks back through the steps they took previously, often prompted by a video recording of themselves. There are benefits and drawbacks to each approach, but in general a concurrent protocol may be more complete, while a retrospective protocol has less chance to interfere with task performance. Nonetheless, some concurrent protocols have not produced such interference effects, [11] suggesting that it may be possible to optimize both completeness and authenticity of verbal reports.
The think-aloud method allows researchers to discover what users genuinely think of your design. [12]
A related but slightly different data-gathering method is the talk-aloud protocol. This involves participants only describing their actions but not other thoughts. This method is thought to be more objective in that participants merely report how they go about completing a task rather than interpreting or justifying their actions (see the standard works[ which? ] by Ericsson & Simon).[ citation needed ]
Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.
Usability engineering is a professional discipline that focuses on improving the usability of interactive systems. It draws on theories from computer science and psychology to define problems that occur during the use of such a system. Usability Engineering involves the testing of designs at various stages of the development process, with users or with usability experts. The history of usability engineering in this context dates back to the 1980s. In 1988, authors John Whiteside and John Bennett—of Digital Equipment Corporation and IBM, respectively—published material on the subject, isolating the early setting of goals, iterative evaluation, and prototyping as key activities. The usability expert Jakob Nielsen is a leader in the field of usability engineering. In his 1993 book Usability Engineering, Nielsen describes methods to use throughout a product development process—so designers can ensure they take into account the most important barriers to learnability, efficiency, memorability, error-free use, and subjective satisfaction before implementing the product. Nielsen’s work describes how to perform usability tests and how to use usability heuristics in the usability engineering lifecycle. Ensuring good usability via this process prevents problems in product adoption after release. Rather than focusing on finding solutions for usability problems—which is the focus of a UX or interaction designer—a usability engineer mainly concentrates on the research phase. In this sense, it is not strictly a design role, and many usability engineers have a background in computer science because of this. Despite this point, its connection to the design trade is absolutely crucial, not least as it delivers the framework by which designers can work so as to be sure that their products will connect properly with their target usership.
Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subject.
Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.
User-centered design (UCD) or user-driven development (UDD) is a framework of process in which usability goals, user characteristics, environment, tasks and workflow of a product, service or process are given extensive attention at each stage of the design process. These tests are conducted with/without actual users during each stage of the process from requirements, pre-production models and post production, completing a circle of proof back to and ensuring that "development proceeds with the user as the center of focus." Such testing is necessary as it is often very difficult for the designers of a product to understand intuitively the first-time users of their design experiences, and what each user's learning curve may look like. User-centered design is based on the understanding of a user, their demands, priorities and experiences and when used, is known to lead to an increased product usefulness and usability as it delivers satisfaction to the user.
A usability lab is a place where usability testing is done. It is an environment where users are studied interacting with a system for the sake of evaluating the system's usability.
The cognitive walkthrough method is a usability inspection method used to identify usability issues in interactive systems, focusing on how easy it is for new users to accomplish tasks with the system. A cognitive walkthrough is task-specific, whereas heuristic evaluation takes a holistic view to catch problems not caught by this and other usability inspection methods. The method is rooted in the notion that users typically prefer to learn a system by using it to accomplish tasks, rather than, for example, studying a manual. The method is prized for its ability to generate results quickly with low cost, especially when compared to usability testing, as well as the ability to apply the method early in the design phases before coding even begins.
Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people.
RITE Method, for Rapid Iterative Testing and Evaluation, typically referred to as "RITE" testing, is an iterative usability method. It was defined by Michael Medlock, Dennis Wixon, Bill Fulton, Mark Terrano and Ramon Romero. It has been publicly championed by Dennis Wixon while working in the games space for Microsoft.
Protocol analysis is a psychological research method that elicits verbal reports from research participants. Protocol analysis is used to study thinking in cognitive psychology, cognitive science, and behavior analysis. It has found further application in the design of surveys and interviews, usability testing, educational psychology and design research. With the introduction of video- and audio-based based surveys, the scale and scope of verbal report collection is increased dramatically compared to in-person verbal report recording.
The pluralistic walkthrough is a usability inspection method used to identify usability issues in a piece of software or website in an effort to create a maximally usable human-computer interface. The method centers on recruiting a group of users, developers and usability professionals to step through a task scenario, discussing usability issues associated with dialog elements involved in the scenario steps. The group of experts used is asked to assume the role of typical users in the testing. The method is prized for its ability to be utilized at the earliest design stages, enabling the resolution of usability issues quickly and early in the design process. The method also allows for the detection of a greater number of usability problems to be found at one time due to the interaction of multiple types of participants. This type of usability inspection method has the additional objective of increasing developers’ sensitivity to users’ concerns about the product design.
Usability testing methods aim to evaluate the ease of use of a software product by its users. As existing methods are subjective and open to interpretation, scholars have been studying the efficacy of each method and their adequacy to different subjects, comparing which one may be the most appropriate in fields like e-learning, e-commerce, or mobile applications.
Card sorting is a technique in user experience design in which a person tests a group of subject experts or users to generate a dendrogram or folksonomy. It is a useful approach for designing information architecture, workflows, menu structure, or web site navigation paths.
The Torrance Tests of Creative Thinking is a test of creativity.
In software engineering, rubber duck debugging is a method of debugging code by articulating a problem in spoken or written natural language. The name is a reference to a story in the book The Pragmatic Programmer in which a programmer would carry around a rubber duck and debug their code by forcing themselves to explain it, line by line, to the duck. Many other terms exist for this technique, often involving different (usually) inanimate objects, or pets such as a dog or a cat. Teddy bears are also widely used.
Partial concurrent thinking aloud is a method used to gather data in usability testing with screen reader users. It is a particular kind of think aloud protocol created by Stefano Federici and Simone Borsci at the Interuniversity Center for Research on Cognitive Processing in Natural and Artificial Systems of University of Rome "La Sapienza". The partial concurrent thinking aloud is built up in order to create a specific usability assessment technique for blind users, eligible to maintain the advantages of concurrent and retrospective thinking aloud while overcoming their limits. Using PCTA blind users' verbalizations of problems could be more pertinent and comparable to those given by sighted people who use a concurrent protocol. In the usability evaluation with blind people, the retrospective thinking aloud is often adopted as a functional solution to overcome the structural interference due to thinking aloud and hearing the screen reader imposed by the classic thinking aloud technique; such a solution has yet a relapse in the evaluation method, because the concurrent and the retrospective protocols measure usability from different points of view, one mediated by navigation experience (retrospective) one more direct and pertinent (concurrent). The use of PCTA could be widened to both summative and formative usability evaluations with mixed panels of users, thus extending the number of problems' verbalizations according to disabled users' divergent navigation processes and problem solving strategies.
User experience evaluation (UXE) or user experience assessment (UXA) refers to a collection of methods, skills and tools utilized to uncover how a person perceives a system before, during and after interacting with it. It is non-trivial to assess user experience since user experience is subjective, context-dependent and dynamic over time. For a UXA study to be successful, the researcher has to select the right dimensions, constructs, and methods and target the research for the specific area of interest such as game, transportation, mobile, etc.
Retrospective think aloud protocol is a technique used in usability, and eye tracking in particular, to gather qualitative information on the user intents and reasoning during a test. It's a form of think aloud protocol performed after the user testing session activities, instead of during them. Fairly often the retrospective protocol is stimulated by using a visual reminder such as a video replay. In writing studies, the visual reminder may be the writing produced during the think-aloud session.
Ergonomics, also known as human factors or human factors engineering (HFE), is the application of psychological and physiological principles to the engineering and design of products, processes, and systems. Primary goals of human factors engineering are to reduce human error, increase productivity and system availability, and enhance safety, health and comfort with a specific focus on the interaction between the human and equipment.
Cognitive discourse analysis (CODA) is a research method which examines natural language data in order to gain insights into patterns in (verbalisable) thought. The term was coined by Thora Tenbrink to describe a kind of discourse analysis that had been carried out by researchers in linguistics and other fields. As it is limited to examining verbalisable thought, CODA studies are often triangulated against other research methods. The method is theoretically neutral, and can therefore be used alongside a range of different models of cognition and grammar.
{{cite journal}}: CS1 maint: multiple names: authors list (link){{cite book}}: CS1 maint: multiple names: authors list (link){{cite book}}: CS1 maint: multiple names: authors list (link){{cite journal}}: CS1 maint: multiple names: authors list (link){{cite web}}: |first= has generic name (help)