Course evaluation

Last updated

A course evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. The term may also refer to the completed survey form or a summary of responses to questionnaires.

Contents

They are a means to produce feedback which the teacher and school can use to assess their quality of instruction. The process of (a) gathering information about the impact of learning and of teaching practice on student learning, (b) analyzing and interpreting this information, and (c) responding to and acting on the results, is valuable for several reasons. [1] They enable instructors to review how others interpret their teaching methods. The information can be also used by administrators, along with other input, to make summative decisions (e.g., decisions about promotion, tenure, salary increases, etc.) and make formative recommendations (e.g., identify areas where a faculty member needs to improve). [2] Typically, these evaluations are combined with peer evaluations, supervisor evaluations, and results of student’s test scores to create an overall picture of teaching performance. Course evaluations are implemented in one of two ways, either summative or formative.

Course evaluation instruments

Course evaluation instruments generally include variables such as communication skills, organizational skills, enthusiasm, flexibility, attitude toward the student, teacher – student interaction, encouragement of the student, knowledge of the subject, clarity of presentation, course difficulty, fairness of grading and exams, and global student rating. [3] [4]

Summative evaluation

Summative evaluation occurs at the end of a semester, usually a week or two before the last day of class. The evaluation is performed by the current students of the class. Students have the option to reflect on the teachers’ instruction without fear of punishment because course evaluations are completely confidential and anonymous. This can be done in one of two ways; either with a paper form or with online technology. Typically, in a paper based format, the paper form is distributed by a student while the teacher is out of the room. It is then sealed in an envelope and the teacher will not see it until after final grades are submitted. The online version can be identical to a paper version or more detailed, using branching question technology to glean more information from the student. Both ways allow the student to be able to provide feedback. This feedback is to be used by teachers to assess the quality of their instruction. The information can also be used to evaluate the overall effectiveness of a teacher, particularly for tenure and promotion decisions. [5]

Formative evaluation

Formative evaluation typically occurs when changes can take place during the current semester, although many institutions consider written comments on how to improve formative as well. Typically this form of evaluation is performed by peer consultation. Other experienced teachers will review one of their peer’s instructions. The purpose of this evaluation is for the teacher to receive constructive criticism on teaching. Generally, peer teachers will sit in on a few lessons given by the teacher and take notes on their methods. Later on the team of peer teachers will meet with the said teacher and provide useful, non-threatening feedback on their lessons. The peer team will offer suggestions on improvement, which the said teacher can choose to implement.

Peer feedback is given to the instructor typically in the form of an open session meeting. The peers first reflect on the qualities that were good in the instruction. Then they move on to areas that need improvement. Next the instructor will make suggestions for improvement and receive feedback on those ideas.

Student feedback can be an important part of formative evaluation. Student evaluations are formative when their purpose is to help faculty members improve and enhance their teaching skills. [5] The teachers may require their students to complete written evaluation, participate in ongoing dialogue or directed discussions during the course of the semester. The use of a 'Stop, Start Continue' format for student feedback has been shown to be highly effective at generating constructive feedback for course improvement. [6]

At the Faculty of Psychology of the University of Vienna, Twitter was used for formative course evaluation. [7]

Criticism of course evaluations as measures of teaching effectiveness

Summative student evaluations of teaching (SETs) have been widely criticized, especially by teachers, for not being accurate measures of teaching effectiveness. [2] [8] [9] [10] Surveys have shown that a majority of teachers believe that a teacher's raising the level of standards and/or content would result in worse SETs for the teacher, and that students in filling out SETs are biased in favor of certain teachers' personalities, looks, disabilities, gender and ethnicity. [11] The evidence that some of these critics cite indicates that factors other than effective teaching are more predictive of favorable ratings. In order to get favorable ratings, teachers are likely to present the content which can be understood by the slowest student and consequently the content has been affected. [12] Quantitative fields tend to receive lower student evaluations. [13] Many of those who are critical of SETs have suggested that they should not be used in decisions regarding faculty hires, retentions, promotions, and tenure. Some have suggested that using them for such purposes leads to the dumbing down of educational standards. Others have said that the typical way SETs are now used at most universities is demeaning to instructors [14] and has a corrupting effect on students' attitudes toward their teachers and higher education in general. [15]

The economics of education literature and the economic education literature is especially critical. For example, Weinberg et al. (2009) finds SET scores in first-year economics courses at Ohio State University are positively related to the grades instructors assign but are unrelated to learning outcomes once grades are controlled for. [16] Others have also found a positive relationship between grades and SET scores but unlike Weinberg et al. (2009) do not directly address the relationship between SET scores and learning outcomes. [17] [18] A paper by Krautmann and Sander (1999) find that the grades students expect to receive in a course are positively related to SET scores. [19] Isely and Singh (2005) find it is the difference between the grades students expect to receive and their cumulative GPA that is the relevant variable for obtaining favourable course evaluations. [20] Another paper by Carrell and West (2010) use a data set from the U.S. Air Force Academy where students are randomly assigned to course sections (reducing selection problems). [21] It found that calculus students got higher marks on common course examinations when they had instructors with high SET scores but did worse when they took later courses requiring calculus. [21] The authors discuss a number of possible explanations for this finding, including that instructors with higher SET scores may have concentrated their teaching on the common examinations in the course rather than giving students a deeper understanding for later courses. [21] [22] Hamermesh and West (2005) find that students at the University of Texas at Austin gave attractive instructors higher SET scores than less attractive instructors. [23] However, the authors conclude that it may not be possible to determine if attractiveness increases the effectiveness of an instructor, possibly resulting in better learning outcomes. It may be the case that students pay more attention to attractive instructors. Meanwhile, a 2017 lawsuit was filed on grounds of xenophobic discrimination in course evaluations at the University of Kansas, with Peter F. Lake, the director of Stetson University's Center for Excellence in Higher Education Law and Policy, suggesting this is no isolated incident. [24]

The empirical economics literature is in sharp contrast to the educational psychology literature which generally argues that teaching evaluations are a legitimate method of evaluating instructors and are unrelated to grade inflation. However, similar to the economic literature other researchers outside of educational psychology have offered negative findings on course evaluations. For example, some papers have examined online course evaluations and found them to be heavily influenced by the instructor’s attractiveness and willingness to give high grades in return for very little work. [25] [26]

Another criticism of these assessment instruments is that largely the data they produce are difficult to interpret for purposes of self- or course-improvement, given the number of variables that can affect evaluation scores. [27] Finally, paper based course evaluations can cost a university thousands of dollars over the years, while an electronic survey is offered at minimal cost to the university.

Another concern that has been raised by instructors is that response rates to online course evaluations are lower (and therefore the results may be less valid) than paper-based in class evaluations. The situation is more complex that response rates alone would indicate. [28] Student-faculty engagement is offered as an explanation, where course level, instructor rank, and other variables lacked explanatory power.

See also

Related Research Articles

A teaching method is a set of principles and methods used by teachers to enable student learning. These strategies are determined partly on subject matter to be taught, partly by the relative expertise of the learners, and partly by constraints caused by the learning environment. For a particular teaching method to be appropriate and efficient it has take into account the learner, the nature of the subject matter, and the type of learning it is supposed to bring about.

Student-centered learning, also known as learner-centered education, broadly encompasses methods of teaching that shift the focus of instruction from the teacher to the student. In original usage, student-centered learning aims to develop learner autonomy and independence by putting responsibility for the learning path in the hands of students by imparting to them skills, and the basis on how to learn a specific subject and schemata required to measure up to the specific performance requirement. Student-centered instruction focuses on skills and practices that enable lifelong learning and independent problem-solving. Student-centered learning theory and practice are based on the constructivist learning theory that emphasizes the learner's critical role in constructing meaning from new information and prior experience.

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, aptitude and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests. Assessment can focus on the individual learner, the learning community, a course, an academic program, the institution, or the educational system as a whole. The word "assessment" came into use in an educational context after the Second World War.

Electronic assessment, also known as digital assessment, e-assessment, online assessment or computer-based assessment, is the use of information technology in assessment such as educational assessment, health assessment, psychiatric assessment, and psychological assessment. This covers a wide range of activities ranging from the use of a word processor for assignments to on-screen testing. Specific types of e-assessment include multiple choice, online/electronic submission, computerized adaptive testing such as the Frankfurt Adaptive Concentration Test, and computerized classification testing.

In the realm of US education, a rubric is a "scoring guide used to evaluate the quality of students' constructed responses" according to James Popham. In simpler terms, it serves as a set of criteria for grading assignments. Typically presented in table format, rubrics contain evaluative criteria, quality definitions for various levels of achievement, and a scoring strategy. They play a dual role for teachers in marking assignments and for students in planning their work.

Mastery learning is an instructional strategy and educational philosophy, first formally proposed by Benjamin Bloom in 1968. Mastery learning maintains that students must achieve a level of mastery in prerequisite knowledge before moving forward to learn subsequent information. If a student does not achieve mastery on the test, they are given additional support in learning and reviewing the information and then tested again. This cycle continues until the learner accomplishes mastery, and they may then move on to the next stage. In a self-paced online learning environment, students study the material and take assessments. If they make mistakes, the system provides insightful explanations and directs them to revisit the relevant sections. They then answer different questions on the same material, and this cycle repeats until they reach the established mastery threshold. Only then can they move on to subsequent learning modules, assessments, or certifications.

In an educational setting, standards-based assessment is assessment that relies on the evaluation of student understanding with respect to agreed-upon standards, also known as "outcomes". The standards set the criteria for the successful demonstration of the understanding of a concept or skill.

<span class="mw-page-title-main">Summative assessment</span> Assessment used to determine student outcomes after an academic course

Summative assessment, summative evaluation, or assessment of learning is the assessment of participants in an educational program. Summative assessments are designed to both assess the effectiveness of the program and the learning of the participants. This contrasts with formative assessment, which summarizes the participants' development at a particular time in order to inform instructors of student learning progress.

Formative assessment, formative evaluation, formative feedback, or assessment for learning, including diagnostic testing, is a range of formal and informal assessment procedures conducted by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. The goal of a formative assessment is to monitor student learning to provide ongoing feedback that can help students identify their strengths and weaknesses and target areas that need work. It also helps faculty recognize where students are struggling and address problems immediately. It typically involves qualitative feedback for both student and teacher that focuses on the details of content and performance. It is commonly contrasted with summative assessment, which seeks to monitor educational outcomes, often for purposes of external accountability.

The Keller Plan, also called the Personalized System of Instruction (PSI), was developed by Fred S. Keller with J. Gilmour Sherman, Carolina Bori, and Rodolpho Azzi in the middle 1960s as an innovative method of instruction for the then-new University of Brasília. PSI was conceived of as an application of Skinner's theories of learning, grounded in operant conditioning strategies of behaviorism.

Assessment in computer-supported collaborative learning (CSCL) environments is a subject of interest to educators and researchers. The assessment tools utilized in computer-supported collaborative learning settings are used to measure groups' knowledge learning processes, the quality of groups' products and individuals' collaborative learning skills.

Corrective feedback is a frequent practice in the field of learning and achievement. It typically involves a learner receiving either formal or informal feedback on their understanding or performance on various tasks by an agent such as teacher, employer or peer(s). To successfully deliver corrective feedback, it needs to be nonevaluative, supportive, timely, and specific.

Peer assessment, or self-assessment, is a process whereby students or their peers grade assignments or tests based on a teacher's benchmarks. The practice is employed to save teachers time and improve students' understanding of course materials as well as improve their metacognitive skills. Rubrics are often used in conjunction with self- and peer-assessment.

Peer feedback is a practice where feedback is given by one student to another. Peer feedback provides students opportunities to learn from each other. After students finish a writing assignment but before the assignment is handed in to the instructor for a grade, the students have to work together to check each other's work and give comments to the peer partner. Comments from peers are called as peer feedback. Peer feedback can be in the form of corrections, opinions, suggestions, or ideas to each other. Ideally, peer feedback is a two-way process in which one cooperates with the other.

<span class="mw-page-title-main">Differentiated instruction</span> Framework or philosophy for effective teaching

Differentiated instruction and assessment, also known as differentiated learning or, in education, simply, differentiation, is a framework or philosophy for effective teaching that involves providing all students within their diverse classroom community of learners a range of different avenues for understanding new information in terms of: acquiring content; processing, constructing, or making sense of ideas; and developing teaching materials and assessment measures so that all students within a classroom can learn effectively, regardless of differences in their ability. Differentiated instruction means using different tools, content, and due process in order to successfully reach all individuals. Differentiated instruction, according to Carol Ann Tomlinson, is the process of "ensuring that what a student learns, how he or she learns it, and how the student demonstrates what he or she has learned is a match for that student's readiness level, interests, and preferred mode of learning." According to Boelens et al. (2018), differentiation can be on two different levels: the administration level and the classroom level. The administration level takes the socioeconomic status and gender of students into consideration. At the classroom level, differentiation revolves around content, processing, product, and effects. On the content level, teachers adapt what they are teaching to meet the needs of students. This can mean making content more challenging or simplified for students based on their levels. The process of learning can be differentiated as well. Teachers may choose to teach individually at a time, assign problems to small groups, partners or the whole group depending on the needs of the students. By differentiating product, teachers decide how students will present what they have learned. This may take the form of videos, graphic organizers, photo presentations, writing, and oral presentations. All these take place in a safe classroom environment where students feel respected and valued—effects.

Continuous assessment is a form of educational examination that evaluates a student's progress throughout a prescribed course. It is often used as an alternative to the final examination system. Proponents of continuous assessment argue that the approach allows tracking of progress and has a chance of offering students more support, guidance, and opportunities to improve during the course or programme.

Teaching and learning centers are independent academic units within colleges and universities that exist to provide support services for faculty, to help teaching faculty to improve their teaching and professional development. Teaching centers also routinely provide professional development for graduate students as they prepare for future careers as teaching faculty. Some centers also may provide learning support services for students, and other services, depending on the individual institution. Teaching and learning centers may have different kinds of names, such as faculty development centers, teaching and learning centers, centers for teaching and learning, centers for teaching excellence, academic support centers, and others; a common abbreviation is TLC.

<span class="mw-page-title-main">Flipped classroom</span> Instructional strategy and a type of blended learning

A flipped classroom is an instructional strategy and a type of blended learning. It aims to increase student engagement and learning by having pupils complete readings at home, and work on live problem-solving during class time. This pedagogical style moves activities, including those that may have traditionally been considered homework, into the classroom. With a flipped classroom, students watch online lectures, collaborate in online discussions, or carry out research at home, while actively engaging concepts in the classroom with a mentor's guidance.

Educator effectiveness is a United States K-12 school system education policy initiative that measures the quality of an educator performance in terms of improving student learning. It describes a variety of methods, such as observations, student assessments, student work samples and examples of teacher work, that education leaders use to determine the effectiveness of a K-12 educator.

Data-driven instruction is an educational approach that relies on information to inform teaching and learning. The idea refers to a method teachers use to improve instruction by looking at the information they have about their students. It takes place within the classroom, compared to data-driven decision making. Data-driven instruction works on two levels. One, it provides teachers the ability to be more responsive to students’ needs, and two, it allows students to be in charge of their own learning. Data-driven instruction can be understood through examination of its history, how it is used in the classroom, its attributes, and examples from teachers using this process.

References

  1. ↑ Rahman, K. (2006). Learning from your business lectures: using stepwise regression to understand course evaluation data. Journal of American Academy of Business, Cambridge, 19(2), 272–279.
  2. 1 2 Dunegan, K. J., & Hrivnak, M. W. (2003). Characteristics of mindless teaching evaluations and the moderating effects of image compatibility. Journal of Management Education, 27(3), 280–303.
  3. ↑ Kim, C., Damewood, E., & Hodge, N. (2000). Professor attitude: its effect on teaching evaluations. Journal of Management Education, 24(4), 458–473.
  4. ↑ Tang, T. L.-P. (1997). Teaching evaluation at a public institution of higher education: factors related to the overall teaching effectiveness. Public Personnel Management, 26(3), 379–391.
  5. 1 2 Mohanty, G., Gretes, J., Flowers, C., Algozzine, B., & Spooner, F. (2005). Multi-method evaluation of instruction in engineering classes. Journal of Personnel Evaluation in Education, 18(2), 139–151.
  6. ↑ Hoon, A.E., Oliver, E., Szpakowska, K., and Newton P.M. 2014. Use of the ‘Stop, Start, Continue’ method is associated with the production of constructive qualitative feedback by students in higher education. Assessment and Evaluation in Higher Education. DOI:10.1080/02602938.2014.956282
  7. ↑ Stieger, S., & Burger, C. (2010). Let's go formative: continuous student ratings with Web 2.0 application Twitter. Cyberpsychology, Behavior, and Social Networking, 13(2), 163–167.
  8. ↑ Emery, C. R., Kramer, T. R., & Tian, R.G. (2003). Return to academic standards: a critique of student evaluations of teaching effectiveness. Archived 2009-09-19 at the Wayback Machine Quality Assurance in Education, 11(1), 37–46. Retrieved 2011-06-16.
  9. ↑ Merritt, D. (2008). Bias, the brain, and student evaluations of teaching. Archived 2008-10-08 at the Wayback Machine St. John's Law Review, 82, 235–287. Retrieved 2011-06-16.
  10. ↑ J. Scott Armstrong (2012). "Natural Learning in Higher Education". Encyclopedia of the Sciences of Learning.
  11. ↑ Birnbaum, M. H. (1999). A survey of faculty opinions concerning student evaluations of teaching. Archived 2016-03-04 at the Wayback Machine The Senate Forum (California State University, Fullerton), 14(1), 19–22. Longer version with references. Retrieved 2011-06-16.
  12. ↑ J. Scott Armstrong (2012). "Natural Learning in Higher Education". Encyclopedia of the Sciences of Learning. Archived from the original on 2012-10-28.
  13. ↑ Uttl, Bob, and Dylan Smibert. "Student evaluations of teaching: teaching quantitative courses can be hazardous to one’s career." PeerJ 5 (2017): e3299.
  14. ↑ Gray, M., & Bergmann, B. R. (September–October 2003). "Student teaching evaluations: inaccurate, demeaning, misused", Academe Online, 89(5). Retrieved 2011-06-16.
  15. ↑ Platt, M. (1993). What student evaluations teach. Perspectives on Political Science, 22(1), 29–40. Retrieved 2011-06-16.
  16. ↑ Weinberg, B. A., Hashimoto, M., & Fleisher, B. M. (2009). Evaluating teaching in higher education. Journal of Economic Education, 40(3), 227–261.
  17. ↑ McPherson, M. A., Jewell, R. T., & Kim, M. (2009). What determines student evaluation scores? A random effects analysis of undergraduate economics classes. Eastern Economic Journal, 35(1), 37–51.
  18. ↑ Langbein, L. (2008). Management by results: student evaluation of faculty teaching and the mis-measurement of performance. Economics of Education Review, 27(4), 417–428.
  19. ↑ Krautmann, A. C., & Sander, W. (1999). Grades and student evaluations of teachers. Economics of Education Review, 18(1), 59–63.
  20. ↑ Isely, P., & Singh, H. (2005). Do higher grades lead to favorable student evaluations? Journal of Economic Education, 36(1), 29–42.
  21. 1 2 3 Carrell, S. E., & West, J. E. (2010). Does professor quality matter? Evidence from random assignment of students to professors. Journal of Political Economy, 118(3), 409–432. Retrieved 2011-06-16.
  22. ↑ J. Scott Armstrong (2012). "Natural Learning in Higher Education" (PDF). Encyclopedia of the Sciences of Learning. Archived from the original (PDF) on 2011-11-05. Retrieved 2012-03-26.
  23. ↑ Hamermesh, D. S., & Parker, A. (2005). Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity. Economics of Education Review, 24(4), 369–376.
  24. ↑ Schmidt, Peter (January 13, 2017). "When Students' Prejudices Taint Reviews of Instructors". The Chronicle of Higher Education. Retrieved January 13, 2017.
  25. ↑ Felton, J., Mitchell, J., & Stinson, M. (2004a). Web-based student evaluations of professors: the relations between perceived quality, easiness and sexiness. Assessment & Evaluation in Higher Education, 29(1), 91–108.
  26. ↑ Felton, J., Mitchell, J., & Stinson, M. (2004b). Cultural differences in student evaluations of professors. Archived 2010-07-24 at the Wayback Machine Journal of the Academy of Business Education, Proceedings. Retrieved 2011-06-16.
  27. ↑ Marks, P. (2012). Silent Partners: student course evaluations and the construction of pedagogical worlds Archived 2014-01-19 at archive.today . Canadian Journal for Studies in Discourse and Writing, 24(1).
  28. ↑ Anderson, J., Brown, G., & Spaeth, S. (Aug/Sept 2006). Online student evaluations and response rates reconsidered. Archived 2011-08-18 at the Wayback Machine , Innovate (Fischler School of Education and Human Services, Nova Southeastern University), 2(6). Retrieved 2011-06-16.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.