Progress testing

Last updated

Progress tests are longitudinal, feedback oriented educational assessment tools for the evaluation of development and sustainability of cognitive knowledge during a learning process. A progress test is a written knowledge exam (usually involving multiple choice questions) that is usually administered to all students in the "A" program at the same time and at regular intervals (usually twice to four times yearly) throughout the entire academic program. The test samples the complete knowledge domain expected of new graduates upon completion of their courses, regardless of the year level of the student). The differences between students’ knowledge levels show in the test scores; the further a student has progressed in the curriculum the higher the scores. As a result, these resultant scores provide a longitudinal, repeated measures, curriculum-independent assessment of the objectives (in knowledge) of the entire programme. [1]

Contents

History

Since its inception in the late 1970s at both Maastricht University [1] and the University of Missouri–Kansas City [2] independently, the progress test of applied knowledge has been increasingly used in medical and health sciences programs across the globe. They are well established and increasingly used in medical education in both undergraduate and postgraduate medical education. They are used formatively and summatively.

Use in academic programs

The progress test is currently used by national progress test consortia in the United Kingdom, [3] Italy, The Netherlands, [4] in Germany (including Austria), [5] and in individual schools in Africa, [6] Saudi Arabia, [7] South East Asia, [8] the Caribbean, Australia, New Zealand, Sweden, Finland, UK, and the USA. [9] The National Board of Medical Examiners in the USA also provides progress testing in various countries [10] [11] The feasibility of an international approach to progress testing has been recently acknowledged [12] and was first demonstrated by Albano et al. [13] in 1996, who compared test scores across German, Dutch and Italian medical schools. An international consortium has been established in Canada [12] [14] involving faculties in Ireland, Australia, Canada, Portugal and the West Indies.

The progress test serves several important functions in academic programs. Considerable empirical evidence from medical schools in the Netherlands, Canada, United Kingdom and Ireland, as well postgraduate medical studies and schools in dentistry and psychology have shown that the longitudinal feature of the progress test provides a unique and demonstrable measurement of the growth and effectiveness of students’ knowledge acquisition throughout their course of study [1] [12] [15] [16] [17] [18] [19] [20] [21] . [22] [23]

As a result, this information can be consistently used for diagnostic, remedial and prognostic teaching and learning interventions. In the Netherlands, these interventions have been aided by the provision of a web-based results feedback system known as ProF [24] in which students can compare their results with their peers across different total and subtotal score perspectives, both across and within universities.

Additionally, the longitudinal data can serve as a transparent quality assurance measure for program reviews by providing an evaluation of the extent to which a school is meeting its curriculum objectives. [1] [10] [25] The test also provides more reliable data for high-stakes assessment decisions by using measures of continuous learning rather than a one-shot method (Schuwirth, 2007). Inter-university progress testing collaborations provide a means of improving the cost-effectiveness of assessments by sharing a larger pool of items, item writers, reviewers, and administrators. The collaborative approach adopted by the Dutch and other consortia has enabled the progress test to become a benchmarking instrument by which to measure the quality of educational outcomes in knowledge. The success of the progress test in these ways has led to consideration of developing an international progress test. [25] [26]

The benefits for all main stakeholders in a medical or health sciences programme make the progress test an appealing tool to invest resources and time for inclusion in an assessment regime. This attractiveness is demonstrated by its increasingly widespread use in individual medical education institutions and inter-faculty consortia around the world, and by its use for national and international benchmarking practices.

Advantages

Progress tests provide a rich source of information: the comprehensive nature in combination with the cross-sectional and longitudinal design offers a wealth of information both for individual learners as well as for curriculum evaluations. [1]

Progress Testing fosters knowledge retention: the repeated testing of the same comprehensive domain of knowledge means that there is no point testing facts that could be remembered if studied the night before. Long term knowledge and knowledge retention is fostered because item content remains relevant long after the knowledge has been learned. Progress Testing removes the need for resit examinations: every new test occasion is a renewed opportunity to demonstrate growth of knowledge.

Progress Testing allows early detection of high achievers: some learners perform (far) beyond the expected level of their phase in training (e.g. they might have had relevant previous other training) and, depending on their performance, individual and more speeded pathways through the curriculum could be offered.

Progress Testing brings stability in assessment procedures: curriculum changes, changes in content, have no consequence for the progress test provided the end outcomes are unchanged.

Progress Testing provides excellent benchmarking opportunities: progress tests are not limited to a single school nor to PBL curricula and evaluations can easily be done to compare graduates and the effectiveness of different curriculum approaches.

Disadvantages

Naturally, there are disadvantages. The required resources for test development and scoring and the need for a central organization are two very important ones.

Scoring, [27] psychometric procedures [28] for reducing test difficulty variation and standard setting procedures [29] are more complex in progress testing.

Finally progress tests do not work in heterogeneous programs with early specialization (like in many health sciences programs). In more homogenous programs, such as most medical programs, they work really well and pay off in relation to driving learning and use of resources.

International programs using progress testing

Information from 2010+ (this list may not be complete or up to date, please add any other known progress test administrations)

University of Minho, School of Medicine, Portugal

Netherlands Group - Five medical faculties in the Netherlands (Groningen, Leiden, Maastricht, Nijmegen and VU Amsterdam) and additionally, the Ghent University in Belgium use the test

International Partnership for Progress Testing (IPPT) - undergraduate medical programs at McMaster University Medical School, Canada; University of Limerick, Ireland; University of Algarve, Portugal; University of Western Sydney, Australia

Progress Test Medizin (PTM), Charité-Universitätsmedizin Berlin, Germany (Germany – Berlin, Aachen, Cologne, Münster, Witten, Duisburg-Essen, Oldenburg, Hamburg, Neuruppin, Düsseldorf, Augsburg, Bielefeld; Austria – Graz, Innsbruck, Vienna, Linz)

NBME 1 (Barts, St. George’s London, Leeds and Queens University, Belfast UK)

NBME 2 (University of South Florida and Case Western Reserve University)

Southern Illinois University, Vanderbilt, University of New Mexico, Penn State, Texas Tech, Medical College of Georgia, University of Minnesota

University of Manchester School of Medicine, UK

Peninsula College of Medicine and Dentistry, UK

Cardiff University School of Medicine, Wales, UK

Swansea University Medical School, Wales, UK

University of Tampere, Finland

University of Helsinki, Finland

Karaganda State Medical University, Kazakhstan

Otago Medical School, New Zealand

Auckland Medical School, New Zealand

São Paulo City Medical School (UNICID), Brazil

University of Indonesia, Medical School

Catholic University of Mozambique

Pretoria, South Africa

CMIRA Program, Syrian-Lebanese Hospital Institute for Education and Research, Brazil

Alfaisal University - College of Medicine, Saudi Arabia

The College of Medicine at King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Saudi Arabia

Batterjee Medical College for Senescences & Technology Jeddah- Saudi Arabia

Sulaiman Alrajhi Colleges - School of Medicine, Albukairiyah city, Qassim region - Saudi Arabia Source: [9]

King Fisal University KFU - College of Medicine, Al-Ahsaa, Saudi Arabia

The College of Dentistry at King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Saudi Arabia

Flinders University - School of Medicine (post-graduate), Adelaide, Australia

Royal College of Surgeons in Ireland - Dublin, Ireland

Royal College of Surgeons in Ireland - Medical University of Bahrain

See also

Alfaisal University

Related Research Articles

<span class="mw-page-title-main">Problem-based learning</span> Learner centric pedagogy

Problem-based learning (PBL) is a student-centered pedagogy in which students learn about a subject through the experience of solving an open-ended problem found in trigger material. The PBL process does not focus on problem solving with a defined solution, but it allows for the development of other desirable skills and attributes. This includes knowledge acquisition, enhanced group collaboration and communication.

<span class="mw-page-title-main">King Saud University</span> Public university in Riyadh, Saudi Arabia

King Saud University is a public university in Riyadh, Saudi Arabia. Established in 1957 by King Saud bin Abdulalziz to address the country's skilled worker shortage, it is the first university in Saudi Arabia. It was known as Riyadh University from 1964 until it was reverted to its inceptive name in 1982. It was converted into an independent non-profit academic institution in 2023.

The United States Medical Licensing Examination (USMLE) is a three-step examination program for medical licensure in the United States sponsored by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME). Physicians with a Doctor of Medicine (MD) degree are required to pass the USMLE for medical licensure. However, those with a Doctor of Osteopathic Medicine degree (DO) are required to take the COMLEX-USA (COMLEX) exams.

This is an index of education articles.

An Individualized Education Program (IEP) is a legal document under United States law that is developed for each public school child in the U.S. who needs special education. It is created through a team of the child's parent(s) and district personnel who are knowledgeable about the child's needs. IEPs must be reviewed every year to keep track of the child's educational progress.

<span class="mw-page-title-main">Umm al-Qura University</span> University in Mecca, Saudi Arabia

Umm Al-Qura University is a public university in Mecca, Saudi Arabia. The university was established as the College of Sharia in 1949 during the reign of King Abdulaziz ibn Saud before being joined by new colleges and renamed as Umm Al-Qura through a royal decree in 1981.

<span class="mw-page-title-main">Medical education</span> Education related to the practice of being a medical practitioner

Medical education is education related to the practice of being a medical practitioner, including the initial training to become a physician and additional training thereafter.

The United States Medical Licensing Examination (USMLE) Step 1 is a standardized test that assesses a medical student's knowledge of basic science concepts and their application to clinical medicine. The exam is one of three components required for medical licensure in the United States and is typically taken by students after their second year of medical school.

In education, Response to Intervention is an approach to academic intervention used to provide early, systematic, and appropriately intensive assistance to children who are at risk for or already underperforming as compared to appropriate grade- or age-level standards. RTI seeks to promote academic success through universal screening, early intervention, frequent progress monitoring, and increasingly intensive research-based instruction or interventions for children who continue to have difficulty. RTI is a multileveled approach for aiding students that is adjusted and modified as needed if they are failing.

Integrative learning is a learning theory describing a movement toward integrated lessons helping students make connections across curricula. This higher education concept is distinct from the elementary and high school "integrated curriculum" movement.

<span class="mw-page-title-main">Education in Saudi Arabia</span>

Public education—from primary education through college—is open to every Saudi citizen. Education is the second-largest sector of government spending in Saudi Arabia. Saudi Arabia spends 8.8% of its gross domestic product on education, which is nearly double the global average of 4.6%. Islamic studies are part of the education system alongside scientific and social studies that vary from educational institution to another.

<span class="mw-page-title-main">George Hillocks Jr.</span>

George Hillocks Jr. was an emeritus professor in the Department of Education, with a joint appointment in the Department of English Language and Literature at the University of Chicago. He received in 2011 the James R. Squire Award of the National Council of Teachers of English for having "a transforming influence and [making] a lasting intellectual contribution to the profession." He also received many other major awards. His teaching career included the preparation of English teachers in the Master of Arts in Teaching program, and the mentoring of Ph.D. students in the doctoral program, at the University of Chicago. After retiring from the University he continued to present seminars and workshops for writing teachers across the US. His primary research interests centered on the teaching of writing, literature, and language in middle and high school English classes, and on large-scale writing assessment. When not teaching and writing, he was an accomplished bagpipe player, performing frequently for Chicago audiences and in international competitions.

Curriculum-based measurement, or CBM, is also referred to as a general outcomes measures (GOMs) of a student's performance in either basic skills or content knowledge.

<span class="mw-page-title-main">King Saud bin Abdulaziz University for Health Sciences</span> Public health science university headquartered in Riyadh, Saudi Arabia

King Saud bin Abdulaziz University for Health Sciences(KSAU-HS) is a public university with its main campus in Riyadh, Saudi Arabia, and branch campuses in Jeddah and al-Mubarraz. It was established in 2005 by Crown Prince Abdullah bin Abdulaziz as an academic institute specializing health sciences and is named after King Saud bin Abdulaziz, the ruler of Saudi Arabia between 1953 and 1964.

Qassim University is a major public university in Saudi Arabia. The main campus of Qassim University covers about eight square kilometers in the heart of the region. Qassim University has over 38 Colleges, offering over 30 PhD, 70 master's, 120 Bachelor's and diploma degrees. Qassim University has over 50,000 students. 6000 Faculty and staff members. During 2016/2017 academic year, 7019 male and 9692 female students were newly enrolled at Qassim University. Qassim University is among the top 7 Saudi universities. In 2015, QS ranked Qassim University as 46 among Arab Region Rankings.

<span class="mw-page-title-main">National Board of Medical Examiners</span> Medical accreditation board

The National Board of Medical Examiners (NBME), founded in 1915, is a United States non-profit which develops and manages assessments student physicians. Known for its role in developing the United States Medical Licensing Examination (USMLE) in partnership with the Federation of State Medical Boards (FSMB), USMLE examinations for medical students and residents are used by medical licensing authorities in the U.S. to help determine qualifications to grant and recognize medical licenses. NBME also creates assessments and materials that are used by medical students, medical educators, practicing physicians, and for state testing of physicians already holding licenses.

<span class="mw-page-title-main">Dar Al Uloom University</span> University in Saudi Arabia

Dar Al Uloom University is a university in Riyadh, Saudi Arabia.

<span class="mw-page-title-main">Howard Barrows</span> American physician and medical educator

Howard S. Barrows was an American physician and medical educator who was Professor Emeritus at the Southern Illinois University School of Medicine where he had previously served as Associate Dean for Educational Affairs and Chair of Medical Education. Trained as a neurologist, Barrows is best known today for his many innovations in medical education, particularly teaching using Problem-Based Learning (PBL), developed while he was a professor at McMaster University Medical School, assessing clinical skills using simulated patients, and studying clinical reasoning using stimulated recall techniques.

<span class="mw-page-title-main">Michael G. DeGroote School of Medicine</span> Medical school of McMaster University in Ontario, Canada

The Michael G. DeGroote School of Medicine, known as the McMaster University School of Medicine prior to 2004, is the medical school of McMaster University in Hamilton, Ontario, Canada. It is operated by the McMaster Faculty of Health Sciences. It is one of two medical programs in Canada, along with the University of Calgary, that operates on an accelerated 3-year MD program, instead of the traditional 4-year MD program.

<span class="mw-page-title-main">Higher education in Saudi Arabia</span>

Higher education in Saudi Arabia is the educational stage that follows the three years of secondary education. Higher education institutions are either governmental institutions or private institutions, and are mainly universities, colleges, and academies. There are three higher educational levels in Saudi Arabia: bachelor's degree, master's degree, and doctorate. Governmental universities in Saudi Arabia offer a free bachelor's degree education for Saudis and a monthly payment for each student during their studying period.

References

  1. 1 2 3 4 5 van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. 1996. Fifteen years of experience with progress testing in a problem-based learning curriculum. Medical Teacher 18(2):103–110.
  2. ↑ Arnold L, Willoughby TL. 1990. The quarterly profile examination. Academic Medicine 65(8):515–516.
  3. ↑ Swanson, D. B., Holtzman, K. Z., Butler, A., Langer, M. M., Nelson, M. V., Chow, J. W. M., et al. (2010). Collaboration across the pond: The multi-school progress testing project. Medical Teacher, 32, 480-485.
  4. ↑ Schuwirth, L., Bosman, G., Henning, R. H., Rinkel, R., & Wenink, A. C. G. (2010). Collaboration on progress testing in medical schools in the Netherlands. Medical Teacher, 32, 476-479.
  5. ↑ Nouns, Z. M., & Georg, W. (2010). Progress testing in German speaking countries. Medical Teacher, 32, 467-470.
  6. ↑ Aarts, R., Steidel, K., Manuel, B. A. F., & Driessen, E. W. (2010). Progress testing in resource-poor countries: A case from Mozambique. Medical Teacher, 32, 461-463.
  7. ↑ Al Alwan, I., Al-Moamary, M., Al-Attas, N., Al Kushi, A., ALBanyan, E., Zamakhshary, M., et al. (2011). The progress test as a diagnostic tool for a new PBL curriculum. Education for Health (December, Article No. 493).
  8. ↑ Mardiastuti, H. W., & Werdhani, R. A. (2011). Grade point average, progress test, and try outs test as tools for curriculum evaluation and graduates' performance prediction at the national board examination. Journal of Medicine and Medical Sciences, 2(12), 1302-1305.
  9. 1 2 Freeman, A., van der Vleuten, C., Nouns, Z., & Ricketts, C. (2010). Progress testing internationally. Medical Teacher, 32, 451-455.
  10. 1 2 De Champlain, A., Cuddy, M. M., Scoles, P. V., Brown, M., Swanson, D. B., Holtzman, K., et al. (2010). Progress testing in clinical science education: Results of a pilot project between the National Board of Medical Examiners and a US medical School. Medical Teacher, 32, 503-508.
  11. ↑ International Foundations of Medicine (2011). Retrieved 20 July 2011, from http://www.nbme.org/Schools/iFoM/index.html
  12. 1 2 3 Finucane, P., Flannery, D., Keane, D., & Norman, G. (2010). Cross-institutional progress testing: Feasibility and value to a new medical school. Medical Education, 44, 184-186.
  13. ↑ Albano, M. G., Cavallo, F., Hoogenboom, R., Magni, F., Majoor, G., Manenti, F., et al. (1996). An international comparison of knowledge levels of medical students: The Maastricht progress test. Medical Education, 30, 239-245.
  14. ↑ International Partnership for Progress Testing (2011). Retrieved 18 July 2011, from http://ipptx.org/
  15. ↑ Bennett, J., Freeman, A., Coombes, L., Kay, L., & Ricketts, C. (2010). Adaptation of medical progress testing to a dental setting. Medical Teacher, 32, 500-502.
  16. ↑ Boshuizen, H. P. A., van der Vleuten, C. P. M., Schmidt, H., & Machiels-Bongaerts, M. (1997). Measuring knowledge and clinical reasoning skills in a problem-based curriculum. Medical Education, 31, 115-121.
  17. ↑ Coombes, L., Ricketts, C., Freeman, A., & Stratford, J. (2010). Beyond assessment: Feedback for individuals and institutions based on the progress test. Medical Teacher, 32, 486-490.
  18. ↑ Dijksterhuis, M. G. K., Scheele, F., Schuwirth, L. W. T., Essed, G. G. M., & Nijhuis, J. G. (2009). Progress testing in postgraduate medical education. Medical Teacher, 31, e464-e468.
  19. ↑ Freeman, A., & Ricketts, C. (2010). Choosing and designing knowledge assessments: Experience at a new medical school. Medical Teacher, 32, 578-581.
  20. ↑ Schaap, L., Schmidt, H., & Verkoeijen, P. J. L. (2011). Assessing knowledge growth in a psychology curriculum: which students improve most? Assessment & Evaluation in Higher Education, 1-13.
  21. ↑ van der Vleuten, C. P. M., Verwijnen, G. M., & Wijnen, W. H. F. W. (1996). Fifteen years of experience with progress testing in a problem-based learning curriculum. Medical Teacher, 18(2), 103-109.
  22. ↑ van Diest, R., van Dalen, J., Bak, M., Schruers, K., van der Vleuten, C., Muijtjens, A. M. M., et al. (2004). Growth of knowledge in psychiatry and behavioural sciences in a problem-based learning curriculum. Medical Education, 38, 1295-1301.
  23. ↑ Verhoeven, B. H., Verwijnen, G. M., Scherpbier, A. J. J. A., & van der Vleuten, C. P. M. (2002). Growth of medical knowledge. Medical Education, 36, 711-717.
  24. ↑ Muijtjens, A. M. M., Timmermans, I., Donkers, J., Peperkamp, R., Medema, H., Cohen-Schotanus, J., et al. (2010). Flexible electronic feedback using the virtues of progress testing. Medical Teacher, 32, 491-495.
  25. 1 2 Verhoeven, B. H., Snellen-Balendong, H. A. M., Hay, I. T., Boon, J. M., Van Der Linde, M. J., Blitz-Lindeque, J. J., et al. (2005). The versatility of progress testing assessed in an international context: a start for benchmarking global standardization? Medical Teacher, 27(6), 514-520.
  26. ↑ Schauber, S., & Nouns, Z. B. (2010). Using the cumulative deviation method for cross-institutional benchmarking in the Berlin progress test. Medical Teacher, 32, 471-475.
  27. ↑ Muijtjens AM, Mameren HV, Hoogenboom RJ, Evers JL, van der Vleuten CP. 1999. The effect of a ‘don’t know’ option on test scores: Number-right and formula scoring compared. Medical Education 33(4):267–275.
  28. ↑ Shen L. 2000. Progress testing for postgraduate medical education: A four year experiment of American College of Osteopathic Surgeons Resident Examinations. Advances in Health Sciences Education: Theory and Practice 5(2):117–129
  29. ↑ Verhoeven BH, Snellen-Balendong HA, Hay IT, Boon JM, van der Linde MJ, Blitz-Lindeque JJ, Hoogenboom RJI, Verwijnen GM, Wijnen WHFW, Scherpbier AJJA, et al. 2005. The versatility of progress testing assessed in an international context: A start for benchmarking global standardization? Medical Teacher 27(6):514–520
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.