Large-scale learning assessments

Last updated

Large-scale learning assessments (LSLAs) is defined as a form of national or cross-national standardized testing that provide a snapshot of learning achievement for a group of learners in a given year and in a limited number of learning domains. [1] [2]

Contents

The use of these assessments has been increasing around the globe and have also broadened in scope. LSLAs go beyond measuring reading and mathematics, and target a greater number of domains, including digital skills, computer and information literacy, socio-emotional skills, or the understanding of concepts and issues related to civics and citizenship. [3]

Total spending and distribution of total spending on education, by country income group and financing source Total spending and distribution of total spending on education, by country income group and financing source.svg
Total spending and distribution of total spending on education, by country income group and financing source

LSLAs have gained central debates education at both local and global levels. [1] This was encouraged by emphasis on equitable, effective and relevant learning for all inherent to the 2030 Agenda for Sustainable Development and the focus on using data to improve policies and strategies. LSLAs are incorporated into the work programmes of international and regional organizations, and are supported by donor agencies through financial and technical assistance. [1]

Principles and implementations

LSLAs are a specific subset of learning assessment systems. [1] They are system-level assessments that provide information of learning achievement for a given group of learners (based on age or grade) in a given year and in a limited number of domains. They are often categorized as national or cross-national (regional/international) assessments. [1] [4]

LSLAs are uniform and standardized in content, administration process, timing and scoring. [1] They are frequently referred to as standardized tests, particularly within Anglo-Saxon countries and literature. They are generally sample-based, but over the last decades, some countries have adopted a census-based approach. [5] They can be school or household-based, curriculum-based or not. [3]

LSLAs are generally used by education authorities to determine learners’ overall achievement levels. They help governments monitor changes in learning outcomes over time and highlight inequalities in learning achievement among population groups. [1] By identifying correlates of learning outcomes and by providing a deeper insight on how a range of variables interact, LSLA data also provide better understanding of the dynamics behind the performance of education systems. They inform the design of policies and strategies aimed at improving student knowledge and competences as well as equity in learning. [1] [6]

LSLAs are designed and implemented to meet a certain level of standards. Despite there being no vetted international standard for the characteristics that define robust LSLAs (i.e. that yield reliable data), there is agreement among test developers, statisticians and psychometricians on the technical requirements of such assessments. [7]

LSLAs are developed and implemented based on at least three principles: [1]

  1. Technically sound. They are assessment methodologies, analysis and interpretation of data that follow scientific principles;
  2. Following standardized field operations;
  3. Designed to be ethical, fair and inclusive of the target population. [1]

Perceived benefits

Developing a national learning assessment or participating in cross-national initiatives are multiple and driven by interconnected factors. [1]

Four main factors that enhance the use of LSLAs are: the growing number of perceived benefits, an evolving global culture of evaluation, a shift in the focus of global education policy, and priorities and demands of development donors. [1] [8]

Data analysis

Data from LSLAs give governments evidence to address system inefficiencies by providing answers to key questions, such as who is learning what and who is not, where, when and why. [9] Learning achievement scores and information from the background questionnaires are generally used by experts and researchers to describe the knowledge and skills of a target population. [10] This involves several types of analyses. [1] [11] First, understanding the factors that influence learning achievement (e.g. home and school context and practices), and if these are changing over time. [12] Second, identifying general trends in learning achievements and evaluating progress towards specific targets using a set of indicators. Third, highlighting disparities in cognitive abilities among sub-populations of learners by relevant dimension, including socio-economic, regional, gender, migration status and mother tongue. [7]

Policy-makers use the results or evidence from LSLAs for many purposes including: [1] [7]

Limitations

LSLAs generally focus on a limited range of learning dimensions and address a defined number of the multiple purposes of education. [13] It may not measure other variables such as classroom and school settings. According to a 2019 UNESCO publication three main limitations arise: its under-use, over-use and the combination with (or subordinate to) accountability measures. [1]

See also

Sources

Definition of Free Cultural Works logo notext.svg  This article incorporates text from a free content work. Licensed under CC BY-SA 3.0 IGO. Text taken from The promise of large-scale learning assessments: acknowledging limits to unlock opportunities , UNESCO, UNESCO. UNESCO.

Related Research Articles

Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally covers specialized fields within psychology and education devoted to testing, measurement, assessment, and related activities. Psychometrics is concerned with the objective measurement of latent constructs that cannot be directly observed. Examples of latent constructs include intelligence, introversion, mental disorders, and educational achievement. The levels of individuals on nonobservable latent variables are inferred through mathematical modeling based on what is observed from individuals' responses to items on tests and scales.

Psychological testing refers to the administration of psychological tests. Psychological tests are administered or scored by trained evaluators. A person's responses are evaluated according to carefully prescribed guidelines. Scores are thought to reflect individual or group differences in the construct the test purports to measure. The science behind psychological testing is psychometrics.

<span class="mw-page-title-main">Adult education</span> Any form of learning adults engage in beyond traditional schooling

Adult education, distinct from child education, is a practice in which adults engage in systematic and sustained self-educating activities in order to gain new forms of knowledge, skills, attitudes, or values. It can mean any form of learning adults engage in beyond traditional schooling, encompassing basic literacy to personal fulfillment as a lifelong learner, and to ensure the fulfillment of an individual.

<span class="mw-page-title-main">Standardized test</span> Test administered and scored in a predetermined, standard manner

A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions and interpretations are consistent and are administered and scored in a predetermined, standard manner.

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, aptitude and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests. Assessment can focus on the individual learner, the learning community, a course, an academic program, the institution, or the educational system as a whole. The word 'assessment' came into use in an educational context after the Second World War.

<span class="mw-page-title-main">No Child Left Behind Act</span> 2002 United States education reform law; repealed 2015

The No Child Left Behind Act of 2001 (NCLB) was a U.S. Act of Congress that reauthorized the Elementary and Secondary Education Act; it included Title I provisions applying to disadvantaged students. It supported standards-based education reform based on the premise that setting high standards and establishing measurable goals could improve individual outcomes in education. The Act required states to develop assessments in basic skills. To receive federal school funding, states had to give these assessments to all students at select grade levels.

<span class="mw-page-title-main">Programme for International Student Assessment</span> Scholastic performance study by the OECD

The Programme for International Student Assessment (PISA) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) in member and non-member nations intended to evaluate educational systems by measuring 15-year-old school pupils' scholastic performance on mathematics, science, and reading. It was first performed in 2000 and then repeated every three years. Its aim is to provide comparable data with a view to enabling countries to improve their education policies and outcomes. It measures problem solving and cognition.

Education policy consists of the principles and policy decisions that influence the field of education, as well as the collection of laws and rules that govern the operation of education systems. Education governance may be shared between the local, state, and federal government at varying levels. Some analysts see education policy in terms of social engineering.

<span class="mw-page-title-main">Elementary and Secondary Education Act</span> 1965 US law, part of Johnsons War on Poverty

The Elementary and Secondary Education Act (ESEA) was passed by the 89th United States Congress and signed into law by President Lyndon B. Johnson on April 11, 1965. Part of Johnson's "War on Poverty", the act has been one of the most far-reaching pieces of federal legislation affecting education ever passed by the United States Congress, and was further emphasized and reinvented by its modern, revised No Child Left Behind Act.

The Education Quality and Accountability Office (EQAO) is a Crown agency of the Government of Ontario in Canada. It was legislated into creation in 1996 in response to recommendations made by the Royal Commission on Learning in February 1995.

In US education terminology, a rubric is "a scoring guide used to evaluate the quality of students' constructed responses". Put simply, it is a set of criteria for grading assignments. Rubrics usually contain evaluative criteria, quality definitions for those criteria at particular levels of achievement, and a scoring strategy. They are often presented in table format and can be used by teachers when marking, and by students when planning their work. In UK education, the rubric is the set of instructions at the head of an examination paper.

Formative assessment, formative evaluation, formative feedback, or assessment for learning, including diagnostic testing, is a range of formal and informal assessment procedures conducted by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. The goal of a formative assessment is to monitor student learning to provide ongoing feedback that can help students identify their strengths and weaknesses and target areas that need work. It also helps faculty recognize where students are struggling and address problems immediately. It typically involves qualitative feedback for both student and teacher that focuses on the details of content and performance. It is commonly contrasted with summative assessment, which seeks to monitor educational outcomes, often for purposes of external accountability.

Data based decision making or data driven decision making refers to educator’s ongoing process of collecting and analyzing different types of data, including demographic, student achievement test, satisfaction, process data to guide decisions towards improvement of educational process. DDDM becomes more important in education since federal and state test-based accountability policies. No Child Left Behind Act opens broader opportunities and incentives in using data by educational organizations by requiring schools and districts to analyze additional components of data, as well as pressing them to increase student test scores. Information makes schools accountable for year by year improvement various student groups. DDDM helps to recognize the problem and who is affected by the problem.

ACT, Inc. is an American 501(c)(3) nonprofit organization, primarily known for the ACT, a standardized test designed to assess high school students' academic achievement and college readiness. For the U.S. high school graduating class of 2019, 52 percent of graduates had taken the ACT test; the more than 1.78 million students included virtually all high school graduates in 17 states.

Data-driven instruction is an educational approach that relies on information to inform teaching and learning. The idea refers to a method teachers use to improve instruction by looking at the information they have about their students. It takes place within the classroom, compared to data-driven decision making. Data-driven instruction works on two levels. One, it provides teachers the ability to be more responsive to students’ needs, and two, it allows students to be in charge of their own learning. Data-driven instruction can be understood through examination of its history, how it is used in the classroom, its attributes, and examples from teachers using this process.

Washback effect refers to the impact of testing on curriculum design, teaching practices, and learning behaviors. The influences of testing can be found in the choices of learners and teachers: teachers may teach directly for specific test preparation, or learners might focus on specific aspects of language learning found in assessments. Washback effect in testing is typically seen as either negative, or positive. Washback may be considered harmful to more fluid approaches in language education where definitions of language ability may be limited; however, it may be considered beneficial when good teaching practices result. Washback can also be positive or negative in that it either maintains or hinders the accomplishment of educational goals. In positive washback, teaching the curriculum becomes the same as teaching to a specific test. Negative washback occurs in situations where there may be a mismatch between the stated goals of instruction and the focus of assessment; it may lead to the abandonment of instructional goals in favor of test preparation.

Monitoring Education for Sustainable Development (ESD) refers to measuring progress in ESD learning compared to policy commitments, provision, institutional support, resources and others. Monitoring and evaluation (M&E) of Education for Sustainable Development is widely discussed in literature on ESD, including debates regarding methodology and strategies for interpreting the data.

<span class="mw-page-title-main">Sustainable Development Goal 4</span> 4th of 17 Sustainable Development Goals to achieve quality education for all

Sustainable Development Goal 4 is about quality education and is among the 17 Sustainable Development Goals established by the United Nations in September 2015. The full title of SDG 4 is "Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all".

Randy Elliot Bennett is an American educational researcher who specializes in educational assessment. He is currently the Norman O. Frederiksen Chair in Assessment Innovation at Educational Testing Service in Princeton, NJ. His research and writing focus on bringing together advances in cognitive science, technology, and measurement to improve teaching and learning. He received the ETS Senior Scientist Award in 1996, the ETS Career Achievement Award in 2005, the Teachers College, Columbia University Distinguished Alumni Award in 2016, Fellow status in the American Educational Research Association (AERA) in 2017, the National Council on Measurement in Education's (NCME) Bradley Hanson Award for Contributions to Educational Measurement in 2019, the E. F. Lindquist Award from AERA and ACT in 2020, and elected membership in the National Academy of Education in 2022. Randy Bennett was elected President of both the International Association for Educational Assessment (IAEA), a worldwide organization primarily constituted of governmental and NGO measurement organizations, and the National Council on Measurement in Education (NCME), whose members are employed in universities, testing organizations, state and federal education departments, and school districts.

The learning crisis or global learning crisis is a term describing the fact that, despite a large increase in access to schooling, learning outcomes remain poor, especially in developing countries. Worldwide, millions of children who attend school do not acquire basic skills such as literacy and numeracy, and many more are far behind age-appropriate expectations in their national curricula. Proponents argue that this crisis needs to be addressed due to the importance of education in fostering children's development, social mobility, and subsequent opportunities.

References

  1. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 UNESCO (2019). The promise of large-scale learning assessments: acknowledging limits to unlock opportunities. UNESCO. ISBN   978-92-3-100333-2.
  2. Chudowsky, Naomi; Pellegrino, James W. (2003-02-01). "Large-Scale Assessments That Support Learning: What Will It Take?". Theory into Practice. 42 (1): 75–83. doi:10.1207/s15430421tip4201_10. ISSN   0040-5841. S2CID   143791579.
  3. 1 2 3 Maddox, Bryan (2023). "The uses of process data in large-scale educational assessments" (PDF). OECD Education Working Papers. 286. doi:10.1787/5d9009ff-en . Retrieved 5 June 2023.
  4. "Learning assessments | Unesco IIEP Learning Portal". learningportal.iiep.unesco.org. Retrieved 2020-03-10.
  5. Verger, Antoni; Parcerisa, Lluís; Fontdevila, Clara (2019-01-02). "The growth and spread of large-scale assessments and test-based accountabilities: a political sociology of global education reforms". Educational Review. 71 (1): 5–30. doi:10.1080/00131911.2019.1522045. ISSN   0013-1911. S2CID   150242878.
  6. Chudowsky, Naomi; Pellegrino, James W. (February 2003). "Large-Scale Assessments That Support Learning: What Will It Take?". Theory into Practice. 42 (1): 75–83. doi:10.1207/s15430421tip4201_10. ISSN   0040-5841. S2CID   143791579.
  7. 1 2 3 Clarke, Marguerite; Luna-Bazaldua, Diego (26 April 2021). Primer on Large-Scale Assessments of Educational Achievement. Washington, DC: World Bank. ISBN   978-1-4648-1659-8 . Retrieved 5 June 2023.
  8. Bennett, Randy Elliot. (1998). Reinventing Assessment. Speculations on the Future of Large-Scale Educational Testing. a Policy Information Perspective. Policy Information Center. OCLC   967116582.
  9. Montoya, S. 2016. The Cost of Ignorance Revisited: A Reply. "Measuring Learning: the Cost of Ignorance By Silvia Montoya" . Retrieved 2020-12-21. (Accessed December 2020).
  10. Lietz, Petra; Tobin, Mollie (2016-10-19). "The impact of Large-Scale Assessments in Education on education policy: evidence from around the world". Research Papers in Education. 31 (5): 499–501. doi: 10.1080/02671522.2016.1225918 . ISSN   0267-1522.
  11. "Learning about learning assessments". blogs.worldbank.org. Retrieved 2020-03-10.
  12. Stecher, Brain (November 1998). "The Local Benefits and Burdens of Large‐scale Portfolio Assessment". Assessment in Education: Principles, Policy & Practice. 5 (3): 335–351. doi:10.1080/0969595980050303. ISSN   0969-594X.
  13. Butler, Frances A. (1997). Accommodation strategies for English language learners on large-scale assessments : student characteristics and other considerations. Center for Research on Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, Los Angeles. OCLC   41041578.