Programme for the International Assessment of Adult Competencies

Last updated

The Programme for the International Assessment of Adult Competencies (PIAAC) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) in 24 countries of cognitive and workplace skills. The main aim is to be able to assess the skills of literacy, numeracy and problem solving in technology-rich environments, and use the collected information to help countries develop ways to further improve these skills. The focus is on the working-age population (between the ages of 16 and 65). The first data was released on October 8, 2013. [1] [2] A new PIAAC survey is expected to be published in 2021/2022. [3]

Contents

Background

Since the early 1990s the need for assessing literary skills in developed countries has been addressed by two large international surveys. The first was the International Adult Literacy Survey (IALS) which was implemented in 1994, 1996, and 1998. The second was the International Adult Literacy and Life Skills Survey carried out in 2003, and between 2006 and 2008. [4]

Basic skills assessed

Three central basic skills are assessed in PIAAC: literacy, numeracy and problem solving in technology-rich environments. [5]

Literacy

Literacy encompasses the ability to understand, use and interpret written texts. Literacy is a prerequisite for developing one's knowledge and potential and participating in society. The literacy domain in PIAAC includes tasks such as reading and understanding a drug label or a brief newspaper article. In addition, there are tasks that involve digital media, such as reading an online job posting.

PIAAC Proficiency Levels for Literacy Achievement level and score range Task descriptions

Below Level 1 – 0 – 175 The tasks at this level require the respondent to read brief texts on familiar topics to locate a single piece of specific information. There is seldom any competing information in the text and the requested information is identical in form to information in the question or directive. The respondent may be required to locate information in short continuous texts. However, in this case, the information can be located as if the text were non-continuous in format. Only basic vocabulary knowledge is required, and the reader is not required to understand the structure of sentences or paragraphs or make use of other text features. Tasks below Level 1 do not make use of any features specific to digital texts. [6]

Level 1 – 76 – 225 Most of the tasks at this level require the respondent to read relatively short digital or print continuous, non-continuous, or mixed texts to locate a single piece of information that is identical to or synonymous with the information given in the question or directive. Some tasks, such as those involving non-continuous texts, may require the respondent to enter personal information onto a document. Little, if any, competing information is present. Some tasks may require simple cycling through more than one piece of information. Knowledge and skill in recognizing basic vocabulary determining the meaning of sentences, and reading paragraphs of text is expected. [6]

Level 2 – 226 – 275 At this level, the medium of texts may be digital or printed, and texts may comprise continuous, non-continuous, or mixed types. Tasks at this level require respondents to make matches between the text and information and may require paraphrasing or low-level inferences. Some competing pieces of information may be present. Some tasks require the respondent to

  • cycle through or integrate two or more pieces of information based on criteria;
  • compare and contrast or reason about information requested in the question; or
  • navigate within digital texts to access and identify information from various parts of a document. [6]

Level 3 – 276 – 325 Texts at this level are often dense or lengthy, and include continuous, non-continuous, mixed, or multiple pages of text. Understanding text and rhetorical structures become more central to successfully completing tasks, especially navigating complex digital texts. Tasks require the respondent to identify, interpret, or evaluate one or more pieces of information, and often require varying levels of inference. Many tasks require the respondent to construct meaning across larger chunks of text or perform multi-step operations in order to identify and formulate responses. Often tasks also demand that the respondent disregard irrelevant or inappropriate content to answer accurately. Competing information is often present, but it is not more prominent than the correct information. [6]

Level 4 – 326 – 375 Tasks at this level often require respondents to perform multiple-step operations to integrate, interpret, or synthesize information from complex or lengthy continuous, non-continuous, mixed, or multiple type texts. Complex inferences and application of background knowledge may be needed to perform the task successfully. Many tasks require identifying and understanding one or more specific, non-central idea(s) in the text in order to interpret or evaluate subtle evidence-claim or persuasive discourse relationships. Conditional information is frequently present in tasks at this level and must be taken into consideration by the respondent. Competing information is present and sometimes seemingly as prominent as correct information. [6]

Level 5 – 376 – 500 At this level, tasks may require the respondent to search for and integrate information across multiple, dense texts; construct syntheses of similar and contrasting ideas or points of view; or evaluate evidence based arguments. Application and evaluation of logical and conceptual models of ideas may be required to accomplish tasks. Evaluating reliability of evidentiary sources and selecting key information is frequently a requirement. Tasks often require respondents to be aware of subtle, rhetorical cues and to make high-level inferences or use specialized background knowledge. [6]

Numeracy

Numeracy refers to the ability to access, use and interpret everyday mathematical information in order to manage mathematical demands in daily life. This is measured, for example, with items involving the evaluation of a special offer or the interpretation of numerical information in figures and tables.

Description of PIAAC numeracy discrete achievement levels

Below Level 1 0 – 175 Tasks at this level require the respondents to carry out simple processes such as counting, sorting, performing basic arithmetic operations with whole numbers or money, or recognizing common spatial representations in concrete, familiar contexts where the mathematical content is explicit with little or no text or distractors. [6]

Level 1 176 – 225 Tasks at this level require the respondent to carry out basic mathematical processes in common, concrete contexts where the mathematical content is explicit with little text and minimal distractors. Tasks usually require one-step or simple processes involving counting, sorting, performing basic arithmetic operations, understanding simple percent's such as 50%, and locating and identifying elements of simple or common graphical or spatial representations. [6]

Level 2 226 – 275 Tasks at this level require the respondent to identify and act on mathematical information and ideas embedded in a range of common contexts where the mathematical content is fairly explicit or visual with relatively few distractors. Tasks tend to require the application of two or more steps or processes involving calculation with whole numbers and common decimals, percents and fractions; simple measurement and spatial representation; estimation; and interpretation of relatively simple data and statistics in texts, tables and graphs. [6]

Level 3 – 276 – 325 Tasks at this level require the respondent to understand mathematical information that may be less explicit, embedded in contexts that are not always familiar and represented in more complex ways. Tasks require several steps and may involve the choice of problem-solving strategies and relevant processes. Tasks tend to require the application of number sense and spatial sense; recognizing and working with mathematical relationships, patterns, and proportions expressed in verbal or numerical form; and interpretation and basic analysis of data and statistics in texts, tables and graphs. [6]

Level 4 326 – 375 Tasks at this level require the respondent to understand a broad range of mathematical information that may be complex, abstract or embedded in unfamiliar contexts. These tasks involve undertaking multiple steps and choosing relevant problem-solving strategies and processes. Tasks tend to require analysis and more complex reasoning about quantities and data; statistics and chance; spatial relationships; and change, proportions and formulas. Tasks at this level may also require understanding arguments or communicating well-reasoned explanations for answers or choices. [6]

Level 5 – 376 – 500 Tasks at this level require the respondent to understand complex representations and abstract and formal mathematical and statistical ideas, possibly embedded in complex texts. Respondents may have to integrate multiple types of mathematical information where considerable translation or interpretation is required; draw inferences; develop or work with mathematical arguments or models; and justify, evaluate and critically reflect upon solutions or choices. [6]

Problem solving in technology-rich environments

PIAAC is the first international survey to implement problem solving in technology-rich environments (ICT). This key skill is defined as the ability to successfully use digital technologies, communication tools and networks to search for, communicate and interpret information. The first wave of PIAAC focuses on how persons access and make use of information in a computer-based environment. Items include sorting and sending e-mails, filling out digital forms, and evaluating the informational content and credibility of different websites.

Level 1 – 241 – 290 At this level, tasks typically require the use of widely available and familiar technology applications, such as e-mail software or a web browser. There is little or no navigation required to access the information or commands required to solve the problem. The problem may be solved regardless of the respondent's awareness and use of specific tools and functions (e.g. a sort function). The tasks involve few steps and a minimal number of operators. At the cognitive level, the respondent can readily infer the goal from the task statement; problem resolution requires the respondent to apply explicit criteria; and there are few monitoring demands (e.g. the respondent does not have to check whether he or she has used the appropriate procedure or made progress towards the solution). Identifying content and operators can be done through simple match. Only simple forms of reasoning, such as assigning items to categories, are required; there is no need to contrast or integrate information. [6]

Level 2 – 291 – 340 At this level, tasks typically require the use of both generic and more specific technology applications. For instance, the respondent may have to make use of a novel online form. Some navigation across pages and applications is required to solve the problem. The use of tools (e.g. a sort function) can facilitate the resolution of the problem. The task may involve multiple steps and operators. The goal of the problem may have to be defined by the respondent, though the criteria to be met are explicit. There are higher monitoring demands. Some unexpected outcomes or impasses may appear. The task may require evaluating the relevance of a set of items to discard distractors. Some integration and inferential reasoning may be needed. [6]

Level 3 – 341 – 500 At this level, tasks typically require the use of both generic and more specific technology applications. Some navigation across pages and applications is required to solve the problem. The use of tools (e.g. a sort function) is required to make progress towards the solution. The task may involve multiple steps and operators. The goal of the problem may have to be defined by the respondent, and the criteria to be met may or may not be explicit. There are typically high monitoring demands. Unexpected outcomes and impasses are likely to occur. The task may require evaluating the relevance and reliability of information in order to discard distractors. Integration and inferential reasoning may be needed to a large extent. [6]

Survey design

PIAAC was initiated by the OECD member states in 2008 and, like PISA, it is designed as a multi-cycle programme. Round 1 took place in 2008–13 (main study in 2011 [4] ), supplementary Round 2 in 2012–16, and Round 3 in 2014–18 (main study in 2016–17). [7] Subsequent cycles will allow future changes in adult skills to be monitored and analysed and will provide first indications of where improvements have been achieved and deficits persist. Twenty-four countries participated in PIAAC Round I. Nine additional countries also expressed interest in participating in PIAAC. At least 5 000 randomly selected respondents between the ages of 16 and 65 were interviewed and assessed in each participating country. The survey was carried out as a personal interview comprising a questionnaire followed by a skills assessment, a computer- or paper-based version of which was independently completed by the respondent in the presence of the interviewer; the entire interview (including the assessment) took between 1 1/2 and 2 hours to administer. [5]

Results

The results were published in 2013, together with summaries in 25 languages. [8]

CountryLiteracyNumeracyProblem solving in technology-rich environmentsMean difference, men vs. womenNative vs. immigrant
Mean score % Non- starters % < Level1Mean score% < Level1Mean score%No ICT % < Level1LiteracyNumeracyLiteracy
Australia280.41.93.1267.65.7227.121.29.24.413.736.9
Austria269.51.82.52753.4222.624.99.92.513.231.4
Canada273.50.93.8265.55.9222.616.714.84.414.633
Czech Republic2740.61.5275.71.7229.324.612.94.693.5
Denmark270.80.43.8278.33.4222.514.113.93.610.342.7
Estonia275.90.42273.12.4229.129.113.82.6615.5
Finland287.502.7282.23.1234.518.4112.310.253.7
France262.10.85.3254.29.128.1210.835.4
Germany269.81.53.3271.74.5219.417.714.45.217.331
Ireland266.50.54.3255.67.1226.732.212.65.311.929
Italy250.50.75.5247.1841.50.410.729.2
Japan296.21.20.6288.21.2255.236.87.62.312.3
Korea272.60.32.2263.44.2236.5309.85.810.354
Netherlands2842.32.6280.33.5227.411.212.5416.740.4
Norway278.42.23278.34.3224.513.511.46.814.843.7
Poland266.903.9259.85.9236.549.812−1.81.9
Slovak Republic273.80.31.9275.83.523836.48.9−1.82.4−1.8
Spain251.80.87.2245.89.533.96.812.534.2
Sweden279.203.7279.14.4227.812.113.15.413.652.9
United States269.84.23.9252.89.1224.815.615.82.414.130.8
Flanders (Belgium)275.55.22.7280.43227.815.614.86.61648.4
England/N. Ireland (UK)272.51.43.3261.76.3222.914.615.12.714.334.3
Cyprus268.817.71.6264.63.438.3−0.97.326
Russian Federation275.201.6269.92234.433.614.9
Average272.81.23.3268.75229.224.412.33.511.733.8

Notes:

  1. Sub-national entities (Belgium and UK) are placed at the end followed by two partner countries (Cyprus and Russia) who took part. The figures for Russia are preliminary and do not include Moscow or any items in the 'Missing' category.
  2. Participants were ranked at 5 levels (3 in problem solving). Level 1 corresponds to 176 points. Those who scored less than level 1 are listed above as well as those marked as 'Non-starters' above or 'Missing' in the report, referring to 'literacy-related non-responses' due to mental or learning disabilities or language difficulties. These were marked at 85 (out of 500) in calculating the mean figures.
  3. The last three columns show the extra points scored on average by the first group compared with the second. The last compares native-born native language speakers with foreign-born foreign language speakers.
  4. France, Italy, Spain and Cyprus did not participate in the Problem Solving test, which is officially described as 'Problem Solving in technology-rich environments'. The 'No ICT' column includes those who had no computer experience, opted out or failed a basic IT competence test.
  5. Cyprus only refers to that part under control of the Republic of Cyprus.

Competence groups

For details of the numeracy and ICT tests see OECD 2013.

2020 Gallup impact study of PIAAC

Gallup principal economist Jonathan Rothwell assessed results collected by PIAAC during 2012 - 2017 for a 2020 economic impact analysis [9] commissioned by the Barbara Bush Foundation for Family Literacy, [10] surmising that, of the 33 OECD nations surveyed, the U.S. had placed sixteenth for literacy, with about half of Americans surveyed, aged 16 to 74, demonstrating a below sixth-grade reading level. [9]

See also

Programme for International Student Assessment

Related Research Articles

<span class="mw-page-title-main">Literacy</span> Ability to read and write

Literacy in its broadest sense describes "particular ways of thinking about and doing reading and writing" with the purpose of understanding or expressing thoughts or ideas in written form in some specific context of use. In other words, humans in literate societies have sets of practices for producing and consuming writing, and they also have beliefs about these practices. Reading, in this view, is always reading something for some purpose; writing is always writing something for someone for some particular end. Beliefs about reading and writing and their value for society and for the individual always influence the ways literacy is taught, learned, and practiced over the lifespan.

Computer science is the study of the theoretical foundations of information and computation and their implementation and application in computer systems. One well known subject classification system for computer science is the ACM Computing Classification System devised by the Association for Computing Machinery.

Constraint satisfaction problems (CSPs) are mathematical questions defined as a set of objects whose state must satisfy a number of constraints or limitations. CSPs represent the entities in a problem as a homogeneous collection of finite constraints over variables, which is solved by constraint satisfaction methods. CSPs are the subject of research in both artificial intelligence and operations research, since the regularity in their formulation provides a common basis to analyze and solve problems of many seemingly unrelated families. CSPs often exhibit high complexity, requiring a combination of heuristics and combinatorial search methods to be solved in a reasonable time. Constraint programming (CP) is the field of research that specifically focuses on tackling these kinds of problems. Additionally, Boolean satisfiability problem (SAT), the satisfiability modulo theories (SMT), mixed integer programming (MIP) and answer set programming (ASP) are all fields of research focusing on the resolution of particular forms of the constraint satisfaction problem.

<span class="mw-page-title-main">Mathematics education</span> Mathematics teaching, learning and scholarly research

In contemporary education, mathematics education—known in Europe as the didactics or pedagogy of mathematics—is the practice of teaching, learning, and carrying out scholarly research into the transfer of mathematical knowledge.

<span class="mw-page-title-main">Numeracy</span> Ability to apply numerical concepts

Numeracy is the ability to understand, reason with, and to apply simple numerical concepts. The charity National Numeracy states: "Numeracy means understanding how mathematics is used in the real world and being able to apply it to make the best possible decisions...It’s as much about thinking and reasoning as about 'doing sums'". Basic numeracy skills consist of comprehending fundamental arithmetical operations like addition, subtraction, multiplication, and division. For example, if one can understand simple mathematical equations such as 2 + 2 = 4, then one would be considered to possess at least basic numeric knowledge. Substantial aspects of numeracy also include number sense, operation sense, computation, measurement, geometry, probability and statistics. A numerically literate person can manage and respond to the mathematical demands of life.

In artificial intelligence (AI), commonsense reasoning is a human-like ability to make presumptions about the type and essence of ordinary situations humans encounter every day. These assumptions include judgments about the nature of physical objects, taxonomic properties, and peoples' intentions. A device that exhibits commonsense reasoning might be capable of drawing conclusions that are similar to humans' folk psychology and naive physics.

Functional illiteracy consists of reading and writing skills that are inadequate "to manage daily living and employment tasks that require reading skills beyond a basic level". Those who read and write only in a language other than the predominant language of their environs may also be considered functionally illiterate. Functional illiteracy is contrasted with illiteracy in the strict sense, meaning the inability to read or write simple sentences in any language.

Reading comprehension is the ability to process written text, understand its meaning, and to integrate with what the reader already knows. Reading comprehension relies on two abilities that are connected to each other: word reading and language comprehension. Comprehension specifically is a "creative, multifaceted process" dependent upon four language skills: phonology, syntax, semantics, and pragmatics.

The Key Skills Qualification is a frequently required component of 14-20 education in England, Northern Ireland and Wales. The aim of Key Skills is to encourage learners to develop and demonstrate their skills as well as learn how to select and apply skills in ways that are appropriate to their particular context.

The Iowa Tests of Educational Development (ITED) are a set of standardized tests given annually to high school students in many schools in the United States, covering Grades 9 to 12. The tests were created by the University of Iowa's College of Education in 1942, as part of a program to develop a series of nationally accepted standardized achievement tests. The primary goal of the ITED is to provide information to assist educators in improving teaching.

Skills for Life was the national strategy in England for improving adult literacy, language (ESOL) and numeracy skills and was established as part of the wider national Skills Strategy to increase the numbers of young people and adults with adequate skills. The strategy was launched by the Prime Minister, Tony Blair, in March 2001.

Choice modelling attempts to model the decision process of an individual or segment via revealed preferences or stated preferences made in a particular context or contexts. Typically, it attempts to use discrete choices in order to infer positions of the items on some relevant latent scale. Indeed many alternative models exist in econometrics, marketing, sociometrics and other fields, including utility maximization, optimization applied to consumer theory, and a plethora of other identification strategies which may be more or less accurate depending on the data, sample, hypothesis and the particular decision being modelled. In addition, choice modelling is regarded as the most suitable method for estimating consumers' willingness to pay for quality improvements in multiple dimensions.

<span class="mw-page-title-main">Literacy in the United States</span> Overview of literacy in the United States

Literacy in the United States was categorized by the National Center for Education Statistics into different literacy levels, with 92% of American adults having at least "Level 1" literacy in 2019. Nationally, over 20% of adult Americans have a literacy proficiency at or below Level 1. Adults in this range have difficulty using or understanding print materials. Those on the higher end of this category can perform simple tasks based on the information they read, but adults below Level 1 may only understand very basic vocabulary or be functionally illiterate. According to a 2020 report by the U.S. Department of Education, 54% of adults in the United States have English prose literacy below the 6th-grade level.

<span class="mw-page-title-main">Analytical skill</span> Crucial skill in all different fields of work and life

Analytical skill is the ability to deconstruct information into smaller categories in order to draw conclusions. Analytical skill consists of categories that include logical reasoning, critical thinking, communication, research, data analysis and creativity. Analytical skill is taught in contemporary education with the intention of fostering the appropriate practises for future professions. The professions that adopt analytical skill include educational institutions, public institutions, community organisations and industry.

The apprentice perspective is an educational theory of apprenticeship concerning the process of learning through physical integration into the practices associated with the subject, such as workplace training. By developing similar performance to other practitioners, an apprentice will come to understand the tacit duties of the position. In the process of creating this awareness, the learner also affect their environment; as they are accepted by master practitioners, their specific talents and contributions within the field are taken into account and integrated into the overall practice.

<span class="mw-page-title-main">Neo-Piagetian theories of cognitive development</span>

Neo-Piagetian theories of cognitive development criticize and build upon Jean Piaget's theory of cognitive development.

In mathematics education, a representation is a way of encoding an idea or a relationship, and can be both internal and external. Thus multiple representations are ways to symbolize, to describe and to refer to the same mathematical entity. They are used to understand, to develop, and to communicate different mathematical features of the same object or operation, as well as connections between different properties. Multiple representations include graphs and diagrams, tables and grids, formulas, symbols, words, gestures, software code, videos, concrete models, physical and virtual manipulatives, pictures, and sounds. Representations are thinking tools for doing mathematics.

<span class="mw-page-title-main">21st century skills</span> Skills identified as being required for success in the 21st century

21st century skills comprise skills, abilities, and learning dispositions that have been identified as being required for success in 21st century society and workplaces by educators, business leaders, academics, and governmental agencies. This is part of a growing international movement focusing on the skills required for students to master in preparation for success in a rapidly changing, digital society. Many of these skills are also associated with deeper learning, which is based on mastering skills such as analytic reasoning, complex problem solving, and teamwork. These skills differ from traditional academic skills in that they are not primarily content knowledge-based.

<span class="mw-page-title-main">Glossary of artificial intelligence</span> List of definitions of terms and concepts commonly used in the study of artificial intelligence

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence, its sub-disciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

<span class="mw-page-title-main">Kindergarten readiness</span>

Kindergarten readiness refers to the developmental domains that contribute to children's ability to adapt to the kindergarten classroom, which is often a new and unfamiliar environment. There is no single agreed upon definition of Kindergarten readiness. The domains often included in the definition comprise: academic skills, social-emotional development, and physical development. In addition to these competencies, the child's environment and opportunities for learning should also be considered. This includes the child's home environment and their interactions with teachers and peers.

References

  1. OECD (2013). OECD Skills Outlook 2013.
  2. Thorn, William (2009). "International Adult Literacy and Basic Skills Surveys in the OECD Region". OECD Education Working Papers. OECD Publishing. 26. Retrieved 2013-04-26.
  3. "New PIAAC study coming up – to measure abilities among adults — Nordic Labour Journal".
  4. 1 2 "Adult Literacy and Lifeskills (ALL) - Overview".
  5. 1 2 Beatrice Rammstedt; Daniela Ackermann; Susanne Helmschrott; Anja Klaukien; Débora B. Maehler; Silke Martin; Natascha Massing; Anouk Zabal (2013). PIAAC 2012: Overview of the Main Results (PDF).
  6. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 "Highlights from the U.S. PIAAC Survey of Incarcerated Adults: Their Skills, Work Experience, Education, and Training Program for the International Assessment of Adult Competencies: 2014" (PDF). US Department of Education. November 2016. Retrieved 5 August 2021.PD-icon.svg This article incorporates text from this source, which is in the public domain .
  7. "PIAAC - Organisation for Economic Co-operation and Development". www.oecd.org. Archived from the original on 2013-08-09.
  8. OECD 2013
  9. 1 2 SCHMIDT, Emily (16 March 2022). "READING THE NUMBERS: 130 MILLION AMERICAN ADULTS HAVE LOW LITERACY SKILLS, BUT FUNDING DIFFERS DRASTICALLY BY STATE". apmresearchlab.org. American Public Media. Retrieved 12 August 2022.
  10. ROTHWELL, PH.D., JONATHAN (8 September 2020). "Assessing the Economic Gains of Eradicating Illiteracy Nationally and Regionally in the United States" (PDF). barbarabush.org. Gallup. Retrieved 4 August 2022.