Linear-on-the-fly testing

Last updated

Linear-on-the-fly testing, often referred to as LOFT, is a method of delivering educational or professional examinations. Competing methods include traditional linear fixed-form delivery and computerized adaptive testing. LOFT is a compromise between the two, in an effort to maintain the equivalence of the set of items that each examinee sees, which is found in fixed-form delivery, while attempting to reduce item exposure and enhance test security.

Fixed-form delivery, which most people are familiar with, entails the testing organization determining one or several fixed sets of items to be delivered together. For example, suppose the test contains 100 items, and the organization wished for two forms. Two forms are published with a fixed set of 100 items each, some of which should overlap to enable equating. All examinees that take the test are given one of the two forms.

If this exam is high volume, meaning that there is a large number of examinees, the security of the examination could be in jeopardy. Many of the test items would become well known in the population of examinees. To offset this, more forms would be needed; if there were eight forms, not as many examinees would see each item.

LOFT takes this to an extreme, and attempts to construct a unique exam for each candidate, within the given constraints of the testing program. Rather than publishing a fixed set of items, a large pool of items is delivered to the computer on which the examinee is taking the exam. Also delivered is a computer program to pseudo-randomly select items so that every examinee will receive a test that is equivalent with respect to content and statistical characteristics, [1] although composed of a different set of items. This is usually done with item response theory.

Related Research Articles

<span class="mw-page-title-main">Graduate Record Examinations</span> Standardized tests

The Graduate Record Examinations (GRE) is a standardized test that is part of the admissions process for many graduate schools in the United States and Canada and a few other countries. The GRE is owned and administered by Educational Testing Service (ETS). The test was established in 1936 by the Carnegie Foundation for the Advancement of Teaching.

<span class="mw-page-title-main">Educational Testing Service</span> Educational testing and assessment organization

Educational Testing Service (ETS), founded in 1947, is the world's largest private educational testing and assessment organization. It is headquartered in Lawrence Township, New Jersey, but has a Princeton address.

The Fundamentals of Engineering (FE) exam, also referred to as the Engineer in Training (EIT) exam, and formerly in some states as the Engineering Intern (EI) exam, is the first of two examinations that engineers must pass in order to be licensed as a Professional Engineer (PE) in the United States. The second exam is the Principles and Practice of Engineering exam. The FE exam is open to anyone with a degree in engineering or a related field, or currently enrolled in the last year of an Accreditation Board for Engineering and Technology (ABET) accredited engineering degree program. Some state licensure boards permit students to take it prior to their final year, and numerous states allow those who have never attended an approved program to take the exam if they have a state-determined number of years of work experience in engineering. Some states allow those with ABET-accredited "Engineering Technology" or "ETAC" degrees to take the examination. The exam is administered by the National Council of Examiners for Engineering and Surveying (NCEES).

<span class="mw-page-title-main">Law School Admission Test</span> US and Canadian standardized test

The Law School Admission Test is a standardized test administered by the Law School Admission Council (LSAC) for prospective law school candidates. It is designed to assess reading comprehension, analytical reasoning, and logical reasoning. The test is an integral part of the law school admission process in the United States, Canada, the University of Melbourne, Australia, and a growing number of other countries.

<span class="mw-page-title-main">Multiple choice</span> Assessment that are responded by choosing correct answers from a list of choices

Multiple choice (MC), objective response or MCQ is a form of an objective assessment in which respondents are asked to select only correct answers from the choices offered as a list. The multiple choice format is most frequently used in educational testing, in market research, and in elections, when a person chooses between multiple candidates, parties, or policies.

Step 2 Clinical Skills of the United States Medical Licensing Examination (USMLE) was an exam administered to medical students/graduates who wish to become licensed physicians in the U.S. It is similar to the COMLEX-USA Level 2-PE exam, taken by osteopathic medical students/graduates who seek licensure as physicians in the U.S. For US medical students, the exam fee is $1,300. For medical students at non-US medical schools, the tests cost is higher—currently $1,535. These fees do not include costs associated with travel and lodging to take the test. Historically, US students have taken Step 2 CS late in their senior year, prior to graduation. However, now that more residency programs require students to record a passing score, many US medical schools recommend students take Step 2 CS in the fall of their senior year.

Computerized adaptive testing (CAT) is a form of computer-based test that adapts to the examinee's ability level. For this reason, it has also been called tailored testing. In other words, it is a form of computer-administered test in which the next item or set of items selected to be administered depends on the correctness of the test taker's responses to the most recent items administered.

Project Management Professional (PMP) is an internationally recognized professional designation offered by the Project Management Institute (PMI). As of 31 July 2020, there are 1,036,368 active PMP-certified individuals and 314 chartered chapters across 214 countries and territories worldwide.

The Peabody Picture Vocabulary Test, the 2007 edition of which is known as the PPVT-IV, is an untimed test of receptive vocabulary for Standard American English and is intended to provide a quick estimate of the examinee's receptive vocabulary ability. It can be used with the Expressive Vocabulary Test-Second Edition (EVT-2) to make a direct comparison between the examinee's receptive and expressive vocabulary skills. The PPVT was developed in 1959 by special education specialists Lloyd M. Dunn and Leota M. Dunn. The current version lists L.M. Dunn and his son D.M. Dunn as authors.

<span class="mw-page-title-main">Certified flight paramedic</span>

A certified flight paramedic (FP-C) is a person who has met the advanced certification requirements for flight paramedics established for this designation by the International Board of Specialty Certification (IBSC), a not-for-profit organization responsible for the administration and development of specialty certification exams for critical care professionals. The FP-C exam was the first specialty paramedic certification offered by the Board for Critical Care Transport Paramedic Certification in 2000. This certification is designed for experienced paramedics who have demonstrated advanced knowledge of critical care medicine.

A computerized classification test (CCT) refers to, as its name would suggest, a Performance Appraisal System that is administered by computer for the purpose of classifying examinees. The most common CCT is a mastery test where the test classifies examinees as "Pass" or "Fail," but the term also includes tests that classify examinees into more than two categories. While the term may generally be considered to refer to all computer-administered tests for classification, it is usually used to refer to tests that are interactively administered or of variable-length, similar to computerized adaptive testing (CAT). Like CAT, variable-length CCTs can accomplish the goal of the test with a fraction of the number of items used in a conventional fixed-form test.

<span class="mw-page-title-main">Information Technology Professional Examination Council</span>

The Information Technology Professional Examination Council (ITPEC), is an organization that conducts information technology examinations throughout Asian countries. It was formed to promote information exchange among examination bodies in Asia, and to facilitate such efforts as the development of common examination questions and the administration of the exam on the same date and time. It was established to support international strategies in the spheres of software development and information technology human resources.

The Psychometric Entrance Test (PET) – commonly known in Hebrew as "ha-Psikhometri" – is a standardized test that serves as an entrance exam for institutions of higher education in Israel. The PET covers three areas: quantitative reasoning, verbal reasoning and English language. It is administered by the National Institute for Testing and Evaluation (NITE) and plays a considerable role in the admissions process. A score combining students' performance on the PET with the average score of their high school matriculation tests has been found to be a highly predictive indicator of students' academic performance in their first year of higher education.

The Principles and Practice of Engineering exam is the examination required for one to become a Professional Engineer (PE) in the United States. It is the second exam required, coming after the Fundamentals of Engineering exam.

The attribute hierarchy method (AHM), is a cognitively based psychometric procedure developed by Jacqueline Leighton, Mark Gierl, and Steve Hunka at the Centre for Research in Applied Measurement and Evaluation (CRAME) at the University of Alberta. The AHM is one form of cognitive diagnostic assessment that aims to integrate cognitive psychology with educational measurement for the purposes of enhancing instruction and student learning. A cognitive diagnostic assessment (CDA), is designed to measure specific knowledge states and cognitive processing skills in a given domain. The results of a CDA yield a profile of scores with detailed information about a student’s cognitive strengths and weaknesses. This cognitive diagnostic feedback has the potential to guide instructors, parents and students in their teaching and learning processes.

An item bank Or Question Bank is a term for a repository of test items that belong to a testing program, as well as all information pertaining to those items. In most applications of testing and assessment, the items are of multiple choice format, but any format can be used. Items are pulled from the bank and assigned to test forms for publication either as a paper-and-pencil test or some form of e-assessment.

Psychometric software is software that is used for psychometric analysis of data from tests, questionnaires, or inventories reflecting latent psychoeducational variables. While some psychometric analyses can be performed with standard statistical software like SPSS, most analyses require specialized tools.

<span class="mw-page-title-main">Exam</span> Educational assessment

An examination or test is an educational assessment intended to measure a test-taker's knowledge, skill, aptitude, physical fitness, or classification in many other topics. A test may be administered verbally, on paper, on a computer, or in a predetermined area that requires a test taker to demonstrate or perform a set of skills.

<span class="mw-page-title-main">Medical College Admission Test</span> Standardized examination for prospective medical students in the United States and Canada

The Medical College Admission Test is a computer-based standardized examination for prospective medical students in the United States, Australia, Canada, and the Caribbean Islands. It is designed to assess problem solving, critical thinking, written analysis and knowledge of scientific concepts and principles. Before 2007, the exam was a paper-and-pencil test; since 2007, all administrations of the exam have been computer-based.

<span class="mw-page-title-main">Matriculation exam (Finland)</span> Finnish university entrance exam

The Finnish Matriculation Examination is the matriculation examination taken at the end of secondary education to qualify for entry into university. In practice, the test also constitutes the high school's final exam(s), although there is a separate diploma on graduating from high school, based not on the exam, but on the grades of individual courses. Since 1919, the test has been arranged by a national body, the Matriculation Examination Board. Before that, the administration of the test was the responsibility of the University of Helsinki.

References

  1. Luecht, R.M. (2005). Some Useful Cost-Benefit Criteria for Evaluating Computer-based Test Delivery Models and Systems. Journal of Applied Testing Technology, 7(2). "Archived copy" (PDF). Archived from the original (PDF) on 2006-09-27. Retrieved 2006-12-01.{{cite web}}: CS1 maint: archived copy as title (link)