Computational psychometrics

Last updated

Computational psychometrics is an interdisciplinary field fusing theory-based psychometrics, learning and cognitive sciences, and data-driven AI-based computational models as applied to large-scale/high-dimensional learning, assessment, [1] biometric, or psychological data. Computational psychometrics is frequently concerned with providing actionable and meaningful feedback to individuals based on measurement and analysis of individual differences as they pertain to specific areas of enquiry.

The relatively recent availability of large-scale psychometric data in accessible formats, alongside the rapid increase in CPU processing power, widespread accessibility and application of cluster and cloud computing, and the development of increasingly sensitive instruments for collecting biometric information has allowed big-data analytical and computational methods to expand the scale and scope of traditional psychometric areas of enquiry and modeling.[ citation needed ]

Pursuing a computational approach to psychometrics often involves scientists working in multidisciplinary teams with expertise in artificial intelligence, machine learning, deep learning and neural network modeling, natural language processing, mathematics and statistics, developmental and cognitive psychology, computer science, data science, learning sciences, virtual and augmented reality, and traditional psychometrics. [ citation needed ]

Another important subfield of computational science and, specifically, AI is what has been called psychometric artificial intelligence (PAI). PAI involves the use of psychometrically developed evaluations, such as intelligence tests and thinking style tests, to be solved algorithmically by an artificial agent. The goal of PAI is to put to the test the design and processing mechanisms proposed by AI researchers in order to get knowledge from both artificial and natural cognitive systems. [2] [3]

Application

Computational psychometrics incorporates both theoretical and applied components ranging from item response theory, classical test theory, and Bayesian approaches to modeling [4] knowledge acquisition and discovery of network psychometric models. [5] Computational psychometrics studies the  computational basis of  learning  and  measurement of traits, such as skills, knowledge, abilities, attitudes, and personality  traits via mathematical modeling, intelligent learning and assessment virtual systems, [6] and  computer simulation of large-scale, complex data which traditional psychometric approaches are ill-equipped to handle. Recent investigations into these hard to measure constructs include work on collaborative problem solving, [7] [8] [9] [10] teamwork, and decision making, among others.

Computational psychometrics is also related to the study of  social complexity.  Concepts such as  complex systems and  emergence have been considered in the study of team assembly and performance. In psychological and medical research it is focused on computational models based on technology enhanced-experimental results. Active areas of enquiry include cognitive, emotional, behavioral, diagnostic, and mental health issues. A computational psychometrics approach in this capacity frequently makes use of emerging capabilities such as biometric and multimodal sensors, virtual and augmented reality, as well as affective and wearable computing technologies. [11]

Related Research Articles

<span class="mw-page-title-main">Cognitive science</span> Interdisciplinary scientific study of cognitive processes

Cognitive science is the interdisciplinary, scientific study of the mind and its processes with input from linguistics, psychology, neuroscience, philosophy, computer science/artificial intelligence, and anthropology. It examines the nature, the tasks, and the functions of cognition. Cognitive scientists study intelligence and behavior, with a focus on how nervous systems represent, process, and transform information. Mental faculties of concern to cognitive scientists include language, perception, memory, attention, reasoning, and emotion; to understand these faculties, cognitive scientists borrow from fields such as linguistics, psychology, artificial intelligence, philosophy, neuroscience, and anthropology. The typical analysis of cognitive science spans many levels of organization, from learning and decision to logic and planning; from neural circuitry to modular brain organization. One of the fundamental concepts of cognitive science is that "thinking can best be understood in terms of representational structures in the mind and computational procedures that operate on those structures."

Psychometrics is a field of study within psychology concerned with the theory and technique of measurement. Psychometrics generally covers specialized fields within psychology and education devoted to testing, measurement, assessment, and related activities. Psychometrics is concerned with the objective measurement of latent constructs that cannot be directly observed. Examples of latent constructs include intelligence, introversion, mental disorders, and educational achievement. The levels of individuals on nonobservable latent variables are inferred through mathematical modeling based on what is observed from individuals' responses to items on tests and scales.

<span class="mw-page-title-main">Symbolic artificial intelligence</span> Methods in artificial intelligence research

In artificial intelligence, symbolic artificial intelligence is the term for the collection of all methods in artificial intelligence research that are based on high-level symbolic (human-readable) representations of problems, logic and search. Symbolic AI used tools such as logic programming, production rules, semantic nets and frames, and it developed applications such as knowledge-based systems, symbolic mathematics, automated theorem provers, ontologies, the semantic web, and automated planning and scheduling systems. The Symbolic AI paradigm led to seminal ideas in search, symbolic programming languages, agents, multi-agent systems, the semantic web, and the strengths and limitations of formal knowledge and reasoning systems.

The expression computational intelligence (CI) usually refers to the ability of a computer to learn a specific task from data or experimental observation. Even though it is commonly considered a synonym of soft computing, there is still no commonly accepted definition of computational intelligence.

Computational cognition is the study of the computational basis of learning and inference by mathematical modeling, computer simulation, and behavioral experiments. In psychology, it is an approach which develops computational models based on experimental results. It seeks to understand the basis behind the human method of processing of information. Early on computational cognitive scientists sought to bring back and create a scientific form of Brentano's psychology.

Mathematical psychology is an approach to psychological research that is based on mathematical modeling of perceptual, thought, cognitive and motor processes, and on the establishment of law-like rules that relate quantifiable stimulus characteristics with quantifiable behavior. The mathematical approach is used with the goal of deriving hypotheses that are more exact and thus yield stricter empirical validations. There are five major research areas in mathematical psychology: learning and memory, perception and psychophysics, choice and decision-making, language and thinking, and measurement and scaling.

Quantitative psychology is a field of scientific study that focuses on the mathematical modeling, research design and methodology, and statistical analysis of psychological processes. It includes tests and other devices for measuring cognitive abilities. Quantitative psychologists develop and analyze a wide variety of research methods, including those of psychometrics, a field concerned with the theory and technique of psychological measurement.

STUDENT is an early artificial intelligence program that solves algebra word problems. It is written in Lisp by Daniel G. Bobrow as his PhD thesis in 1964. It was designed to read and solve the kind of word problems found in high school algebra books. The program is often cited as an early accomplishment of AI in natural language processing.

<span class="mw-page-title-main">Cattell–Horn–Carroll theory</span> Psychological theory

The Cattell–Horn–Carroll theory, is a psychological theory on the structure of human cognitive abilities. Based on the work of three psychologists, Raymond B. Cattell, John L. Horn and John B. Carroll, the Cattell–Horn–Carroll theory is regarded as an important theory in the study of human intelligence. Based on a large body of research, spanning over 70 years, Carroll's Three Stratum theory was developed using the psychometric approach, the objective measurement of individual differences in abilities, and the application of factor analysis, a statistical technique which uncovers relationships between variables and the underlying structure of concepts such as 'intelligence'. The psychometric approach has consistently facilitated the development of reliable and valid measurement tools and continues to dominate the field of intelligence research.

James Frederick Allen is a computational linguist recognized for his contributions to temporal logic, in particular Allen's interval algebra. He is interested in knowledge representation, commonsense reasoning, and natural language understanding, believing that "deep language understanding can only currently be achieved by significant hand-engineering of semantically-rich formalisms coupled with statistical preferences". He is the John H. Dessaurer Professor of Computer Science at the University of Rochester

Cognitive computing refers to technology platforms that, broadly speaking, are based on the scientific disciplines of artificial intelligence and signal processing. These platforms encompass machine learning, reasoning, natural language processing, speech recognition and vision, human–computer interaction, dialog and narrative generation, among other technologies.

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence, its sub-disciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

Industrial artificial intelligence, or industrial AI, usually refers to the application of artificial intelligence to industry and business. Unlike general artificial intelligence which is a frontier research discipline to build computerized systems that perform tasks requiring human intelligence, industrial AI is more concerned with the application of such technologies to address industrial pain-points for customer value creation, productivity improvement, cost reduction, site optimization, predictive analysis and insight discovery.

Automatic Item Generation (AIG), or Automated Item Generation, is a process linking psychometrics with computer programming. It uses a computer algorithm to automatically create test items that are the basic building blocks of a psychological test. The method was first described by John R. Bormuth in the 1960s but was not developed until recently. AIG uses a two-step process: first, a test specialist creates a template called an item model; then, a computer algorithm is developed to generate test items. So, instead of a test specialist writing each individual item, computer algorithms generate families of items from a smaller set of parent item models.

Alina Anca von Davier is a psychometrician and researcher in computational psychometrics, machine learning, and education. Von Davier is a researcher, innovator, and an executive leader with over 20 years of experience in EdTech and in the assessment industry. She is the Chief of Assessment at Duolingo, where she leads the Duolingo English Test research and development area. She is also the Founder and CEO of EdAstra Tech, a service-oriented EdTech company. In 2022, she joined the University of Oxford as an Honorary Research Fellow, and Carnegie Mellon University as a Senior Research Fellow.

<span class="mw-page-title-main">Bruce M. McLaren</span> American researcher, academic and author (born 1959)

Bruce Martin McLaren is an American researcher, scientist and author. He is an Associate Research Professor at Carnegie Mellon University and a former President of the International Artificial Intelligence in Education Society (2017-2019).

Randy Elliot Bennett is an American educational researcher who specializes in educational assessment. He is currently the Norman O. Frederiksen Chair in Assessment Innovation at Educational Testing Service in Princeton, NJ. His research and writing focus on bringing together advances in cognitive science, technology, and measurement to improve teaching and learning. He received the ETS Senior Scientist Award in 1996, the ETS Career Achievement Award in 2005, the Teachers College, Columbia University Distinguished Alumni Award in 2016, Fellow status in the American Educational Research Association (AERA) in 2017, the National Council on Measurement in Education's (NCME) Bradley Hanson Award for Contributions to Educational Measurement in 2019, the E. F. Lindquist Award from AERA and ACT in 2020, and elected membership in the National Academy of Education in 2022. Randy Bennett was elected President of both the International Association for Educational Assessment (IAEA), a worldwide organization primarily constituted of governmental and NGO measurement organizations, and the National Council on Measurement in Education (NCME), whose members are employed in universities, testing organizations, state and federal education departments, and school districts.

Neuro-symbolic AI is a type of artificial intelligence that integrates neural and symbolic AI architectures to address the weaknesses of each, providing a robust AI capable of reasoning, learning, and cognitive modeling. As argued by Leslie Valiant and others, the effective construction of rich computational cognitive models demands the combination of symbolic reasoning and efficient machine learning. Gary Marcus, argued, "We cannot construct rich cognitive models in an adequate, automated way without the triumvirate of hybrid architecture, rich prior knowledge, and sophisticated techniques for reasoning." Further, "To build a robust, knowledge-driven approach to AI we must have the machinery of symbol manipulation in our toolkit. Too much useful knowledge is abstract to proceed without tools that represent and manipulate abstraction, and to date, the only known machinery that can manipulate such abstract knowledge reliably is the apparatus of symbol manipulation."

The Duolingo English Test(DET) is a standardized test of English language (similar to TOEFL, IELTS, and TOEIC), designed to be internet- rather than paper-based. DET was developed by Duolingo and grew in popularity and acceptance at universities during the COVID-19 pandemic. The test is used by over 5,000 university admissions offices including Harvard, Stanford, MIT, and Yale. Ireland accepts the test as part of its student visa program. DET is an online, adaptive test, and uses an algorithm to adapt the difficulty of the test as the test taker progresses. Universities in the UK such as the London School of Economics, Imperial College London, Kingston University, University of Southampton, Middlesex University also accept the Duolingo test.

Jacqueline P. Leighton is a Canadian-Chilean educational psychologist, academic and author. She is a full professor in the Faculty of Education as well as vice-dean of Faculty Development and Faculty Affairs at the University of Alberta.

References

  1. ↑ von Davier, Alina A. (2017). "Computational Psychometrics in Support of Collaborative Educational Assessments". Journal of Educational Measurement. 54 (1): 3–11. doi:10.1111/jedm.12129.
  2. ↑ Bringsjord, Selmer (September 2011). "Psychometric artificial intelligence". Journal of Experimental & Theoretical Artificial Intelligence. 23 (3): 271–277. doi: 10.1080/0952813x.2010.502314 . S2CID   19947368.
  3. ↑ Braynen, Alec (2022). Towards More Task-Generalized and Explainable AI through Psychometrics (Thesis).
  4. ↑ Polyak, Stephen T.; von Davier, Alina A.; Peterschmidt, Kurt (2017). "Computational Psychometrics for the Measurement of Collaborative Problem Solving Skills". Frontiers in Psychology. 8: 20–29. doi: 10.3389/fpsyg.2017.02029 . PMC   5712874 . PMID   29238314.
  5. ↑ Marsman, M.; Borsboom, D.; Kruis, J.; Epskamp, S.; van Bork, R.; Waldorp, L.J.; van der Maas, H.L.J.; Maris, G. (2018). "An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models". Multivariate Behavioral Research. 53 (1): 15–35. doi: 10.1080/00273171.2017.1379379 . PMID   29111774.
  6. ↑ Greiff, Samuel; Gasevic, Dragan; von Davier, Alina A. (2017). Using process data for assessment in Intelligent Tutoring Systems. A psychometrician's, cognitive psychologist's, and computer scientist's perspective. Army Research Laboratory. pp. 171–179. hdl:10993/32037.
  7. ↑ Innovative Assessment of Collaboration. Methodology of Educational Measurement and Assessment. 2017. doi:10.1007/978-3-319-33261-1. ISBN   978-3-319-33259-8.[ page needed ]
  8. ↑ von Davier, Alina A.; Hao, Jiangang; Kyllonen, Patrick (2017). "Interdisciplinary research agenda in support of assessment of collaborative problem solving: lessons learned from developing a Collaborative Science Assessment Prototype". Computers in Human Behavior. 76 (November): 631–640. doi:10.1016/j.chb.2017.04.059.
  9. ↑ Flor, Michael; Yoon, Su-Youn; Hao, Jiangang; Liu, Lei; von Davier, Alina (2016). "Automated classification of collaborative problem solving interactions in simulated science tasks". Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications: 31–41. doi: 10.18653/v1/W16-0504 . S2CID   390510.
  10. ↑ Flor, Michael; Yoon, Su-Youn; Hao, Jiangang; Liu, Lei; von Davier, Alina A. (June 2016). "Automated classification of collaborative problem solving interactions in simulated science tasks". Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications. San Diego, CA: Association for Computational Linguistics. W16-0504: 31–41. doi: 10.18653/v1/W16-0504 . S2CID   390510.
  11. ↑ Cipresso, Pietro; Matic, Aleksandar; Giakoumis, Dimitris; Ostrovsky, Yuri (2015). "Advances in Computational Psychometrics". Computational and Mathematical Methods in Medicine. 2015: 418683. doi: 10.1155/2015/418683 . PMC   4539436 . PMID   26346251.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.