Electronic assessment

Last updated

Electronic assessment, also known as digital assessment, e-assessment, online assessment or computer-based assessment, is the use of information technology in assessment such as educational assessment, health assessment, psychiatric assessment, and psychological assessment. This covers a wide range of activities ranging from the use of a word processor for assignments to on-screen testing. Specific types of e-assessment include multiple choice, online/electronic submission, computerized adaptive testing such as the Frankfurt Adaptive Concentration Test, and computerized classification testing.

Contents

Different types of online assessments contain elements of one or more of the following components, depending on the assessment's purpose: formative, summative and diagnostic. [1] :80–82 Instant and detailed feedback may (or may not) be enabled.

In formative assessment, often defined as 'assessment for learning', digital tools are increasingly being adopted by schools, higher education institutions and professional associations to measure where students are in their skills or knowledge. This can make it easier to provide tailored feedback, interventions or action plans to improve learning and attainment. Gamification is one type of digital assessment tool that can engage students in a different way whilst gathering data that teachers can use to gain insight.

In summative assessment, which could be described as 'assessment of learning', exam boards and awarding organisations delivering high-stakes exams often find the journey from paper-based exam assessment to fully digital assessment a long one. Practical considerations such as having the necessary IT hardware to enable large numbers of student to sit an electronic examination at the same time, as well as the need to ensure a stringent level of security (for example, see: Academic dishonesty) are among the concerns that need to resolved to accomplish this transition.

E-marking is one way that many exam assessment and awarding bodies, such as Cambridge International Examinations, are utilizing innovations in technology to expedite the marking of examinations. In some cases, e-marking can be combined with electronic examinations, whilst in other cases students will still hand-write their exam responses on paper scripts which are then scanned and uploaded to an e-marking system for examiners to mark on-screen.

Application

E-assessment is becoming more widely used by exam awarding bodies, particularly those with multiple or international study centres and those which offer remote study courses. Industry bodies such as The e-Assessment Association (eAA), founded in 2008, as well as events run by the Association of Test Publishers (ATP) that focus specifically on Innovations in Testing, represent the growth in adoption of technology-enhanced assessment.

In psychiatric and psychological testing, e-assessment can be used not only to assess cognitive and practical abilities but anxiety disorders, such as social anxiety disorder, i.e. SPAI-B. Widely in psychology. [2] :4–10 Cognitive abilities are assessed using e-testing software, while practical abilities are assessed using e-portfolios or simulation software.

Types

Online assessment is used primarily to measure cognitive abilities, demonstrating what has been learned after a particular educational event has occurred, such as the end of an instructional unit or chapter. When assessing practical abilities or demonstrating learning that has occurred over a longer period of time an online portfolio (or ePortfolio) is often used. The first element that must be prepared when teaching an online course is assessment. Assessment is used to determine if learning is happening, to what extent and if changes need to be made. [1] :79

Independent work

Most students will not complete assignments unless there is an assessment (i.e. motivation). It is the instructor's role to catalyze student motivation. Appropriate feedback is the key to assessment, whether or not the assessment is graded. [1] :83–86

Group work

Students are often asked to work in groups. This brings on new assessment strategies. Students can be evaluated using a collaborative learning model in which the learning is driven by the students and/or a cooperative learning model where tasks are assigned and the instructor is involved in decisions. [1] :86–89

Pre-testing – Prior to the teaching of a lesson or concept, a student can complete an online pretest to determine their level of knowledge. This form of assessment helps determine a baseline so that when a summative assessment or post-test is given, quantitative evidence is provided showing that learning has occurred.

Formative assessment – Formative assessment is used to provide feedback during the learning process. In online assessment situations, objective questions are posed, and feedback is provided to the student either during or immediately after the assessment.

Summative assessment – Summative assessments provide a quantitative grade and are often given at the end of a unit or lesson to determine that the learning objectives have been met.

Practice Testing – With the ever-increasing use of high-stakes testing in the educational arena, online practice tests are used to give students an edge. Students can take these types of assessments multiple times to familiarize themselves with the content and format of the assessment.

Surveys – Online surveys may be used by educators to collect data and feedback on student attitudes, perceptions or other types of information that might help improve instruction.

Evaluations – This type of survey allows facilitators to collect data and feedback on any type of situation where the course or experience needs justification or improvement.

Performance testing – The user shows what they know and what they can do. This type of testing is used to show technological proficiency, reading comprehension, math skills, etc. This assessment is also used to identify gaps in student learning.

New technologies, such as the Web, digital video, sound, animations, and interactivity, are providing tools that can make assessment design and implementation more efficient, timely, and sophisticated.

Electronic marking

Electronic marking, also known as e-marking and onscreen marking, is the use of digital educational technology specifically designed for marking. The term refers to the electronic marking or grading of an exam. E-marking is an examiner led activity closely related to other e-assessment activities such as e-testing, or e-learning which are student led. E-marking allows markers to mark a scanned script or online response on a computer screen rather than on paper.

There are no restrictions to the types of tests that can use e-marking, with e-marking applications designed to accommodate multiple choice, written, and even video submissions for performance examinations. E-marking software is used by individual educational institutions and can also be rolled out to the participating schools of awarding exam organizations. e-marking has been used to mark many well-known high stakes examinations, which in the United Kingdom include A levels and GCSE exams, and in the US includes the SAT test for college admissions. Ofqual reports that e-marking is the main type of marking used for general qualifications in the United Kingdom.

History

Early adopters include the University of Cambridge Local Examinations Syndicate, (which operates under the brand name Cambridge Assessment) which conducted its first major test of e-marking in November 2000. Cambridge Assessment has conducted extensive research into e-marking and e-assessment. The syndicate has published a series of papers, including research specific to e-marking such as: Examining the impact of moving to on-screen marking on concurrent validity. [3]

In 2007, the International Baccalaureate implemented e-marking. In 2012, 66% of nearly 16 million exam scripts were "e-marked" in the United Kingdom. [Education 1] Ofqual reports that in 2015, all key stage 2 tests in the United Kingdom will be marked onscreen.

In 2010, Mindlogicx [4] implemented onscreen marking system for the first time in India at Anna University [5] enabling easy operations and efficient conduction of high stakes examination.

In 2014, the Scottish Qualifications Authority (SQA) announced that most of the National 5 question papers would be e-marked. [6]

In June 2015, the Odisha state government in India announced that it planned to use e-marking for all Plus II papers from 2016. [7]

Process

E-marking can be used to mark examinations that are completed on paper and then scanned and uploaded as digital images, as well as online examinations. Multiple-choice exams can be either marked by examiners online or be automarked where appropriate. When marking written script exams, e-marking applications provide markers with online tools and resources to mark as they go and can add up marks as they progress without exceeding the prescribed total for each question.

All candidate details are hidden from the work being marked to ensure anonymity during the marking process. Once marking is complete, results can be uploaded immediately, reducing both the time spent by examiners posting results and the wait time for students.

The e-marking FAQ [8] is a comprehensive list of answers to frequently asked questions surrounding e-marking.

Advantages

It has also been noted that in regards to university level work, providing electronic feedback can be more time-consuming than traditional assessments, and therefore more expensive. [9]

In 1986, Lichtenwald investigated the test validity and test reliability of either personal computer administration or a paper and pencil administration of the Peabody Picture Vocabulary Test-Revised (PPVT-R). His project report included a review and analysis of the literature of pre-mid 1980s E-assessment systems. [2]

A review of the literature of E-assessment from the 1970s until 2000 examined the advantages and disadvantages of E-assessments. [10]

A detailed review of the literature regarding advantages and disadvantages of E-assessment for different types of tests for different types of students in different educational environment from childhood through young adulthood was completed in 2010. [11] In higher education settings, there is variation in the ways academics perceive the benefits of e-assessment. While some perceive e-assessment processes as integral to teaching, others think of e-assessment in isolation from teaching and their students' learning. [12]

Academic dishonesty

Academic dishonesty, commonly known as cheating, occurs at all levels of educational institutions. In traditional classrooms, students cheat in various forms such as hidden prepared notes not permitted to be used or looking at another student's paper during an exam, copying homework from one another, or copying from a book, article or media without properly citing the source. Individuals can be dishonest due to lack of time management skills, the pursuit of better grades, cultural behavior or a misunderstanding of plagiarism. [1] :89

Online classroom environments are no exception to the possibility of academic dishonesty. It can easily be seen from a student's perspective as an easy passing grade. Proper assignments types, meetings and projects can prevent academic dishonesty in the online classroom. [1] :89–90 However, online assessment may provide additional possibilities for cheating, such as hacking. [13]

Two common types of academic dishonesty are identity fraud and plagiarism.

Identity fraud can occur in the traditional or online classroom. There is a higher chance in online classes due to the lack of proctored exams or instructor-student interaction. In a traditional classroom, instructors have the opportunity to get to know the students, learn their writing styles or use proctored exams. To prevent identity fraud in an online class, instructors can use proctored exams through the institutions testing center or require students to come in at a certain time for the exam. Correspondence through phone or video conferencing techniques can allow an instructor to become familiar with a student through their voice and appearance. Another option would be personalize assignments to students backgrounds or current activities. This allows the student to apply it to their personal life and gives the instructor more assurance the actual student is completing the assignment. Lastly, an instructor may not make the assignments heavily weighted so the students do not feel as pressured. [1] :89–90

Plagiarism is the misrepresentation of another person's work. It is easy to copy and paste from the internet or retype directly from a source. It is not only the exact wordage, but the thought or idea. [1] :90 It is important to learn to properly cite a source when using someone else's work.

Interoperability

To assist sharing of assessment items across disparate systems, standards such as the IMS Global Question and Test Interoperability specification (QTI) have emerged.

See also

Related Research Articles

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, aptitude and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests. Assessment can focus on the individual learner, the learning community, a course, an academic program, the institution, or the educational system as a whole. The word 'assessment' came into use in an educational context after the Second World War.

<span class="mw-page-title-main">Scottish Qualifications Authority</span> Public body responsible for accrediting educational awards in Scotland

The Scottish Qualifications Authority is the executive non-departmental public body of the Scottish Government responsible for accrediting educational awards. It is partly funded by the Education and Lifelong Learning Directorate of the Scottish Government, and employs approximately 750 staff based in Glasgow and Dalkeith.

Computerized adaptive testing (CAT) is a form of computer-based test that adapts to the examinee's ability level. For this reason, it has also been called tailored testing. In other words, it is a form of computer-administered test in which the next item or set of items selected to be administered depends on the correctness of the test taker's responses to the most recent items administered.

Educational technology is the combined use of computer hardware, software, and educational theory and practice to facilitate learning. When referred to with its abbreviation, edtech, it often refers to the industry of companies that create educational technology.

In US education terminology, rubric is "a scoring guide used to evaluate the quality of students' constructed responses". Put simply, it is a set of criteria for grading assignments. Rubrics usually contain evaluative criteria, quality definitions for those criteria at particular levels of achievement, and a scoring strategy. They are often presented in table format and can be used by teachers when marking, and by students when planning their work. In UK education the rubric is the set instructions at the head of an examination paper.

An objective structured clinical examination (OSCE) is an approach to the assessment of clinical competence in which the components are assessed in a planned or structured way with attention being paid to the objectivity of the examination which is basically an organization framework consisting of multiple stations around which students rotate and at which students perform and are assessed on specific tasks. OSCE is a modern type of examination often used for assessment in health care disciplines.

<span class="mw-page-title-main">Indonesia Open University</span> Indonesian state university

Universitas Terbuka is Indonesia's state university and employs open and distance learning (ODL) system to widen access to higher education to all Indonesian citizens, including those who live in remote islands throughout the country, as well as in various parts of the world. It has a total student body of 1,045,665 as of 2019/2020 according to Indonesia's Ministry of Education Higher Education database. According to a distance education institution in the UK, which published "The Top Ten Mega Universities", UT-3 ranks closely with universities from China and Turkey.

A course evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. The term may also refer to the completed survey form or a summary of responses to questionnaires.

Summative assessment, summative evaluation, or assessment of learning is the assessment of participants in an educational program. Summative assessments are designed to both assess the effectiveness of the program and the learning of the participants. This contrasts with formative assessment, which summarizes the participants' development at a particular time. Summative assessment is widely taught in educational programs in the United States. Michael Scriven claims that while all assessment techniques can be summative, only some are formative.

Formative assessment, formative evaluation, formative feedback, or assessment for learning, including diagnostic testing, is a range of formal and informal assessment procedures conducted by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. The goal of a formative assessment is to monitor student learning to provide ongoing feedback that can help students identify their strengths and weaknesses and target areas that need work. It also helps faculty recognize where students are struggling and address problems immediately. It typically involves qualitative feedback for both student and teacher that focuses on the details of content and performance. It is commonly contrasted with summative assessment, which seeks to monitor educational outcomes, often for purposes of external accountability.

Computer-aidedassessment (CAA) includes all forms of assessments students' progress, whether summative or formative, delivered with the help of computers. This covers both assessments delivered on computer, either online or on a local network, and those that are marked with the aid of computers, such as those using Optical Mark Reading (OMR). There are number of open source online tools to handle exams conducted on OMR sheets.

Assessment in computer-supported collaborative learning (CSCL) environments is a subject of interest to educators and researchers. The assessment tools utilized in computer-supported collaborative learning settings are used to measure groups' knowledge learning processes, the quality of groups' products and individuals' collaborative learning skills.

Corrective feedback is a frequent practice in the field of learning and achievement. It typically involves a learner receiving either formal or informal feedback on their understanding or performance on various tasks by an agent such as teacher, employer or peer(s). To successfully deliver corrective feedback, it needs to be nonevaluative, supportive, timely, and specific.

All India Secondary School Examination, commonly known as board exams, is a centralized public examination that students in schools affiliated with the Central Board of Secondary Education in India take after class 10. The board conducts the examination during the period of February every year even after implementing the ambitious project of Continuous and Comprehensive Evaluation. Now that the board has ended the Continuous and Comprehensive Evaluation pattern as a result Joint Employment Test, NEET, etc., the exam is conducted by the National Testing Agency instead. In this exam, Mathematics, Science and Social Science are compulsory with any two languages. Students can also opt skill subject such as information technology, painting, yoga, music or artificial intelligence. Successful candidates are awarded the Secondary School Completion Certificate, a Statement of Marks, and a Migration Certificate stating that the candidate has completed Secondary Schooling and can pursue higher education. For the 2016-17 academic year, the Central Board of Secondary Education has revived the old system of syllabus and marking scheme (complete syllabus for All India Secondary School Examination and marks out of 500 India has state exams and central exams.

<span class="mw-page-title-main">Continuous and Comprehensive Evaluation</span> Education system in India

Continuous and Comprehensive Evaluation (CCE) was a process of assessment, mandated by the Right to Education Act, of India in 2009. This approach to assessment was introduced by state governments in India, as well as by the Central Board of Secondary Education in India, for students of sixth to tenth grades and twelfth in some schools. From this the smaller classes student would have a practice to face the exam of board in younger age. The Karnataka government introduced CCE for grades 1 through 9 later it was also introduced for 12th grades students. The main aim of CCE was to evaluate every aspect of the child during their presence at the school. This was believed to help reduce the pressure on the child during/before examinations as the student will have to sit for multiple tests throughout the year, of which no test or the syllabus covered will be repeated at the end of the year, whatsoever. The CCE method was claimed to bring enormous changes from the traditional chalk and talk method of teaching, provided it is implemented accurately.

Continuous assessment is a form of educational examination that evaluates a student's progress throughout a prescribed course. It is often used as an alternative to the final examination system. Proponents of continuous assessment argue that the approach allows tracking of progress and has a chance of offering students more support, guidance, and opportunities to improve during the course or programme.

Adaptive comparative judgement is a technique borrowed from psychophysics which is able to generate reliable results for educational assessment – as such it is an alternative to traditional exam script marking. In the approach, judges are presented with pairs of student work and are then asked to choose which is better, one or the other. By means of an iterative and adaptive algorithm, a scaled distribution of student work can then be obtained without reference to criteria.

<span class="mw-page-title-main">Exam</span> Educational assessment

An examination or test is an educational assessment intended to measure a test-taker's knowledge, skill, aptitude, physical fitness, or classification in many other topics. A test may be administered verbally, on paper, on a computer, or in a predetermined area that requires a test taker to demonstrate or perform a set of skills.

Electronic exams offer benefits such as ease of marking, reduced need to read illegible handwriting, saving of time

Haggis is a high-level reference programming language used primarily to examine computing science for Scottish pupils taking SQA courses on the subject. Haggis is used as a tool to bridge the gap between pseudocode and typical computer programming.

References

  1. "The Rise of Onscreen Marking Around The World". RM Results. RM Results. Archived from the original on 13 April 2017. Retrieved 15 October 2015.
  1. 1 2 3 4 5 6 7 8 Rovai A, Ponton M, Baker J (2008). Distance Learning in Higher Education: A programmatic Approach to Planning, Design, Instruction, Evaluation, and Accreditation. Teachers College Press. ISBN   978-0-8077-4878-7.
  2. 1 2 Lichtenwald, Terrance G. (1986). "An Investigation of the validity, reliability, and acceptance by children of a microcomputer administration of the Peabody Picture Vocabulary Test-Revised (PPVT-R) Archived 2018-08-30 at the Wayback Machine ," http://files.eric.ed.gov/fulltext/ED558200.pdf
  3. "Examining the impact of moving to on-screen marking on concurrent validity" (PDF). Cambridge Assessment. Cambridge Assessment. Retrieved 15 October 2015.
  4. Jagannathan, K. T. (2011-06-18). "Mindlogicx to focus on exam management service for varsities". The Hindu. ISSN   0971-751X . Retrieved 2019-03-21.
  5. "Anna varsity outsources all its exams". The Economic Times. Retrieved 2019-03-21.
  6. "An Introduction to E-marking" (PDF). SQA. SQA. Retrieved 15 October 2015.
  7. "State government of India announces that it would be using e-marking for all streams from 2016". Times of India. Times of India. Retrieved 15 October 2015.
  8. E-marking: Frequently Asked Questions
  9. Grieve, Rachel; Padgett, Christine R.; Moffitt, Robyn L. (2016-01-01). "Assignments 2.0: The role of social presence and computer attitudes in student preferences for online versus offline marking". The Internet and Higher Education. 28: 8–16. doi:10.1016/j.iheduc.2015.08.002.
  10. Millsap, Claudette M. (2000). " by CM Millsap – 2000 Comparison of Computer Testing versus Traditional Paper and Pencil"
  11. Blazer, Christie (2010). "U.S. Department of .files.eric.ed.gov/fulltext/ED544707.pdf Information Capsule Archived April 11, 2005, at the Wayback Machine ," Research Services, Volume 0918.
  12. Mimirinis, M. (2019). Qualitative differences in academics’ conceptions of e-assessment. Assessment & Evaluation in Higher Education, 44(2), 233-248.
  13. Dawson, Phillip (17 February 2015). "Five ways to hack and cheat with bring-your-own-device electronic examinations". British Journal of Educational Technology. 47 (4): 592–600. doi:10.1111/bjet.12246.

Further reading