Partial concurrent thinking aloud

Last updated

Partial concurrent thinking aloud (or partial concurrent think-aloud, or PCTA) is a method used to gather data in usability testing with screen reader users. It is a particular kind of think aloud protocol (or TAP) created by Stefano Federici and Simone Borsci [1] at the Interuniversity Center for Research on Cognitive Processing in Natural and Artificial Systems [2] of University of Rome "La Sapienza". The partial concurrent thinking aloud is built up in order to create a specific usability assessment technique for blind users, eligible to maintain the advantages of concurrent and retrospective thinking aloud while overcoming their limits. Using PCTA blind users' verbalizations of problems could be more pertinent and comparable to those given by sighted people who use a concurrent protocol. In the usability evaluation with blind people, the retrospective thinking aloud is often adopted as a functional solution to overcome the structural interference due to thinking aloud and hearing the screen reader imposed by the classic thinking aloud technique; such a solution has yet a relapse in the evaluation method, because the concurrent and the retrospective protocols measure usability from different points of view, one mediated by navigation experience (retrospective) one more direct and pertinent (concurrent). [3] The use of PCTA could be widened to both summative and formative usability evaluations with mixed panels of users, thus extending the number of problems' verbalizations according to disabled users' divergent navigation processes and problem solving strategies.

Contents

Cognitive assumptions

In general, in the usability evaluation both retrospective and concurrent TAP could be used according to the aims and goals of the study. Nevertheless, when a usability evaluation is carried out with blind people several studies propose to use the retrospective TAP: indeed, using a screen reader and talking about the way of interacting with the computer implies a structural interference between action and verbalization. Undoubtedly, cognitive studies provided a lot of evidence supporting the idea that individuals can listen, verbalize, or manipulate, and rescue information in multiple task condition. As Colin Cherry [4] showed, subjects, when listening to two different messages from a single loudspeaker, can separate sounds from background noise, recognize the gender of the speaker, the direction, and the pitch (cocktail party effect). At the same time, subjects that must verbalize the content of a message (attended message) listening to two different message simultaneously (attended and unattended message) have a reduced ability to report the content of the attended massage, while they are unable to report the content of the unattended message. Moreover, K. Anders Ericsson and Walter Kintsch [5] showed that, in a multiple task condition, subjects' ability of rescuing information is not compromised by an interruption of the action flow (as it happens in the concurrent thinking aloud technique), thanks to the “Long Term Working Memory mechanism” of information retrieval (Working Memory section Ericsson and Kintsch). Even if users can listen, recognize, and verbalize multiple messages in a multiple task condition and they can stop and restart actions without losing any information, other cognitive studies underlined that the overlap of activities in a multiple task condition have an effect on the goal achievement: Kemper, Herman and Lian, [6] analysing the users' abilities to verbalize actions in a multiple task condition, showed that the fluency of a user's conversation is influenced by the overlap of actions. Adults are likely to continue to talk as they navigate in a complex physical environment. However, the fluency of their conversation is likely to change: Older adults are likely to speak more slowly than they would if resting; Young adults continue to speak just as rapidly while walking as while resting, but they adopt a further set of speech accommodations, reducing sentence length, grammatical complexity, and propositional density. Just by reducing length, complexity, and propositional density adults free up working memory resources. We do not know how and how much the content of verbalizations could be influenced by the strategy of verbalization (i.e. the modification of fluency and the complexity in a multiple task condition). Anyway, we well know that users in the concurrent thinking aloud verbalize the problems in a more accurate and pertinent way (i.e. more focused on the problems directly perceived during the interaction) then in the retrospective one. [7] [8] The pertinence is granted to the user by the proximity of action-verbalization-next action; this multiple task proximity compels the subject to apply a strategy of verbalization that reduce the overload of the working memory. However, for blind users this time proximity between action and verbalization is lost: the use of the screen reader, in fact, increase the time for verbalization (i.e. in order to verbalize, blind users must first stop the [screen reader] and then restart it).

Protocol

PCTA method is composed of two sections, one concurrent and one retrospective:

The first section is a modified concurrent protocol built up according to the three concurrent verbal protocols criteria described by K. Anders Ericsson and Herbert A. Simon: [9] [10]

The first criterion
Subjects should be talking about the task at hand, not about an unrelated issue. In order to respect this rule, the time between problem retrieval, thinking and verbalization must be minimized to avoid the influence of a long perceptual reworking and the consequent verbalization of unrelated issues. Blind participants, using a screen reader, increase the time latency between identification and verbalization of a problem. To minimize this latency, users are trained to ring a desk-bell that stops both time and navigation. During this suspension, users can create a memory sign (i.e. ring the bell) and restart immediately the navigation. This setting modification allows to avoid the cognitive limitation problem and the influence of perceptual reworking, also creating a memory sign for the retrospective analysis.
The second criterion
To be pertinent, verbalizations should be logically consistent with the verbalizations that just preceded them. For any kind of user it is hard to be pertinent and consistent in a concurrent verbal protocol. Therefore, the practitioners could generally interrupt the navigation and ask for a clarification or stimulate the users to verbalize in a pertinent way. In order to do so and stop navigation to screen reader users, we propose to negotiate a specific physical sign with them: The practitioner, sitting behind the user, will put his hand on the user's shoulder. This physical sign grants the verbalization pertinence and consistence.
The third criterion
A subset of the information needed during the task performance should be remembered. The concurrent model is based on the link between working memory and time latency. The proximity between the occurrence of a thought and its verbal report allows users to verbalize on the basis of their working memory.

The second PCTA section is a retrospective one in which users analyse those problems previously verbalized in a concurrent way. The memory signs, created by users ringing the desk-bell, overcome the limits of classic retrospective analysis; indeed, these signs allow the users to be pertinent and consistent with their concurrent verbalization, thus avoiding the influence of long term memory and perceptual reworking.

See also

Related Research Articles

<span class="mw-page-title-main">Erlang (programming language)</span> Programming language

Erlang is a general-purpose, concurrent, functional high-level programming language, and a garbage-collected runtime system. The term Erlang is used interchangeably with Erlang/OTP, or Open Telecom Platform (OTP), which consists of the Erlang runtime system, several ready-to-use components (OTP) mainly written in Erlang, and a set of design principles for Erlang programs.

Visual thinking, also called visual or spatial learning or picture thinking, is the phenomenon of thinking through visual processing. Visual thinking has been described as seeing words as a series of pictures. It is common in approximately 60–65% of the general population. "Real picture thinkers", those who use visual thinking almost to the exclusion of other kinds of thinking, make up a smaller percentage of the population. Research by child development theorist Linda Kreger Silverman suggests that less than 30% of the population strongly uses visual/spatial thinking, another 45% uses both visual/spatial thinking and thinking in the form of words, and 25% thinks exclusively in words. According to Kreger Silverman, of the 30% of the general population who use visual/spatial thinking, only a small percentage would use this style over and above all other forms of thinking, and can be said to be true "picture thinkers".

In computer science, a lock or mutex is a synchronization primitive: a mechanism that enforces limits on access to a resource when there are many threads of execution. A lock is designed to enforce a mutual exclusion concurrency control policy, and with a variety of possible methods there exists multiple unique implementations for different applications.

A think-aloudprotocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences.

<span class="mw-page-title-main">Usability</span> Capacity of a system for its users to perform tasks

Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.

A heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface (UI) design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles. These evaluation methods are now widely taught and practiced in the new media sector, where UIs are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing.

<span class="mw-page-title-main">Screen reader</span> Assistive technology that converts text or images to speech or Braille

A screen reader is a form of assistive technology (AT) that renders text and image content as speech or braille output. Screen readers are essential to people who are blind, and are useful to people who are visually impaired, illiterate, or have a learning disability. Screen readers are software applications that attempt to convey what people with normal eyesight see on a display to their users via non-visual means, like text-to-speech, sound icons, or a braille device. They do this by applying a wide variety of techniques that include, for example, interacting with dedicated accessibility APIs, using various operating system features, and employing hooking techniques.

<span class="mw-page-title-main">Sony Ericsson P910</span> Smartphone model

The Sony Ericsson P910 is a smartphone by Sony Ericsson introduced in 2004 and the successor of the Sony Ericsson P900. The P910 has a full QWERTY keyboard on the back of the flip. The biggest change from the P900 to the P910 is that the P910 supports Memory Stick PRO Duo and the phone's internal memory has been upped from 16 MB to 64 MB. Although Memory Stick PRO Duo comes in larger capacities, the maximum supported by the P910i is 2 GB. It is powered by an ARM9 processor clocked at 156 MHz and runs the Symbian OS with the UIQ graphical user interface. Also, the touchscreen displays 262,144 colours, as opposed to the P900's 65,536 (16-bit). It comes in three versions:

<span class="mw-page-title-main">Subvocalization</span> Internal process while reading

Subvocalization, or silent speech, is the internal speech typically made when reading; it provides the sound of the word as it is read. This is a natural process when reading and it helps the mind to access meanings to comprehend and remember what is read, potentially reducing cognitive load.

Cognitive ergonomics is a scientific discipline that studies, evaluates, and designs tasks, jobs, products, environments and systems and how they interact with humans and their cognitive abilities. It is defined by the International Ergonomics Association as "concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system. Cognitive ergonomics is responsible for how work is done in the mind, meaning, the quality of work is dependent on the persons understanding of situations. Situations could include the goals, means, and constraints of work. The relevant topics include mental workload, decision-making, skilled performance, human-computer interaction, human reliability, work stress and training as these may relate to human-system design." Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance. It is a subset of the larger field of human factors and ergonomics.

Protocol analysis is a psychological research method that elicits verbal reports from research participants. Protocol analysis is used to study thinking in cognitive psychology, cognitive science, and behavior analysis. It has found further application in the design of surveys and interviews, usability testing, educational psychology and design research.

The pluralistic walkthrough is a usability inspection method used to identify usability issues in a piece of software or website in an effort to create a maximally usable human-computer interface. The method centers on recruiting a group of users, developers and usability professionals to step through a task scenario, discussing usability issues associated with dialog elements involved in the scenario steps. The group of experts used is asked to assume the role of typical users in the testing. The method is prized for its ability to be utilized at the earliest design stages, enabling the resolution of usability issues quickly and early in the design process. The method also allows for the detection of a greater number of usability problems to be found at one time due to the interaction of multiple types of participants. This type of usability inspection method has the additional objective of increasing developers’ sensitivity to users’ concerns about the product design.

K. Anders Ericsson was a Swedish psychologist and Conradi Eminent Scholar and Professor of Psychology at Florida State University who was internationally recognized as a researcher in the psychological nature of expertise and human performance.

<span class="mw-page-title-main">Sony Ericsson P1</span>

The Sony Ericsson P1 is a smartphone and the successor of the P990. It was the last of the Sony Ericsson "P" Smartphone series, introduced in 2002 with the Sony Ericsson P800 and it integrates many of the hardware features of its predecessor the P990 in the form factor of the M600. It was announced on 8 May 2007. There is a Chinese version of P1 called P1c. Compare with P1/ P1i, P1c lacks of 3G, thereby using EDGE which is much slower but more available especially in the US and parts of Europe.

Usability testing methods aim to evaluate the ease of use of a software product by its users. As existing methods are subjective and open to interpretation, scholars have been studying the efficacy of each method and their adequacy to different subjects, comparing which one may be the most appropriate in fields like e-learning, e-commerce, or mobile applications.

A verbal fluency test is a kind of psychological test in which a participant is asked to produce as many words as possible from a category in a given time. This category can be semantic, including objects such as animals or fruits, or phonemic, including words beginning with a specified letter, such as p, for example. The semantic fluency test is sometimes described as the category fluency test or simply as "freelisting", while letter fluency is also referred to as phonemic test fluency. The Controlled Oral Word Association Test (COWAT) is the most employed phonemic variant. Although the most common performance measure is the total number of words, other analyses such as number of repetitions, number and length of clusters of words from the same semantic or phonemic subcategory, or number of switches to other categories can be carried out.

Retrospective memory is the memory of people, words, and events encountered or experienced in the past. It includes all other types of memory including episodic, semantic and procedural. It can be either implicit or explicit. In contrast, prospective memory involves remembering something or remembering to do something after a delay, such as buying groceries on the way home from work. However, it is very closely linked to retrospective memory, since certain aspects of retrospective memory are required for prospective memory.

Metamemory or Socratic awareness, a type of metacognition, is both the introspective knowledge of one's own memory capabilities and the processes involved in memory self-monitoring. This self-awareness of memory has important implications for how people learn and use memories. When studying, for example, students make judgments of whether they have successfully learned the assigned material and use these decisions, known as "judgments of learning", to allocate study time.

Time-based prospective memory is a type of prospective memory in which remembrance is triggered by a time-related cue that indicates that a given action needs to be performed. An example is remembering to watch a television program at 3 p.m. In contrast to time-based prospective memory, event-based prospective memory is triggered by an environmental cue that indicates that an action needs to be performed. An example is remembering to send a letter after seeing a mailbox. While event-based memory is dependent on the environment, time-based prospective memory is self-initiated; one must specifically monitor the passage of time.

Verbal overshadowing is a phenomenon where giving a verbal description of sensory input impairs formation of memories of that input. This was first reported by Schooler and Engstler-Schooler (1990) where it was shown that the effects can be observed across multiple domains of cognition which are known to rely on non-verbal knowledge and perceptual expertise. One example of this is memory, which has been known to be influenced by language. Seminal work by Carmichael and collaborators (1932) demonstrated that when verbal labels are connected to non-verbal forms during an individual's encoding process, it could potentially bias the way those forms are reproduced. Because of this, memory performance relying on reportable aspects of memory that encode visual forms should be vulnerable to the effects of verbalization.

References

  1. ↑ Borsci, S., & Federici, S. (2009). "The Partial Concurrent Thinking Aloud: A New Usability Evaluation Technique for Blind Users". In P. L. Emiliani; L. Burzagli; A. Como; F. Gabbanini; A. L. Salminen (eds.). Assistive technology from adapted equipment to inclusive environments. Vol. 25. IOS Press. pp. 421–425.{{cite book}}: CS1 maint: multiple names: authors list (link)
  2. ↑ "ECoNA - Home Page". Archived from the original on 2010-02-10. Retrieved 2010-02-04.
  3. ↑ Federici, S., Borsci, S., & Stamerra, G. (November 2009). "Web usability evaluation with screen reader users: implementation of the partial concurrent thinking aloud technique". Cognitive Processing. 11 (3): 263–72. doi:10.1007/s10339-009-0347-y. PMID   19916036. S2CID   2155123.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  4. ↑ Cherry, E.C. (1953). "Some experiments on the recognition of speech, with one and with two ears". Journal of the Acoustical Society of America. 25 (5): 975–979. Bibcode:1953ASAJ...25..975C. doi:10.1121/1.1907229. hdl: 11858/00-001M-0000-002A-F750-3 .
  5. ↑ Ericsson, K.A., Kintsch, W. (1995). "Long-Term Working Memory". Psychological Review. 102 (2): 211–245. doi:10.1037/0033-295X.102.2.211. PMID   7740089.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  6. ↑ Kemper, S., Herman, R.E., & Lian, C.H.T. (2003). "The Costs of Doing Two Things at Once for Young and Older Adults: Talking While Walking, Finger Tapping, and Ignoring Speech or Noise". Psychology and Aging. 18 (2): 181–192. doi:10.1037/0882-7974.18.2.181. hdl: 1808/8613 . PMID   12825768.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  7. ↑ Bowers, V.A & Snyder, H.L. (2003). Concurrent versus retrospective verbal protocols for comparing window usability. Human Factors Society 34th Meeting, 8–12 October 1990 HFES, Santa Monica. pp. 1270–1274.
  8. ↑ Van den Haak, M.J. & De Jong, M.D.T. (2003). Exploring Two Methods of Usability Testing: Concurrent versus Retrospective Think-Aloud Protocols. IEEE International Professional Communication Conference Proceedings Piscataway, New Jersey.
  9. ↑ Ericsson, K.A., Simon, H.A. (1980). "Verbal reports as data". Psychological Review. 87 (3): 215–251. doi:10.1037/0033-295X.87.3.215.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  10. ↑ Ericsson, K.A., Simon, H.A. (1993). Protocol analysis: Verbal reports as data (Revised ed.). MIT Press Cambridge.{{cite book}}: CS1 maint: multiple names: authors list (link)
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.