Collegiate Learning Assessment

Last updated

The Collegiate Learning Assessment (CLA) is a standardized testing initiative in United States higher educational evaluation and assessment. It uses a "value-added" outcome model to examine a college or university's contribution to student learning which relies on the institution, rather than the individual student, as the primary unit of analysis. The CLA measures are designed to test for critical thinking, analytic reasoning, problem solving, and written communication skills. The assessment consists of open-ended questions, is administered to students online, and controls for incoming academic ability. An institution's average score on the CLA measures correlates highly with the institution's average SAT score (r = 0.90). [1] Institutional results are not published.

Contents

History and Test Format

The CLA was first launched in 2000 by the Council for Aid to Education (CAE), a national nonprofit organization based in New York City. Rather than testing for specific content knowledge gained in particular courses or majors, the intent was to assess “the collective and cumulative result of what takes place or does not take place over the four to six years of undergraduate education in and out of the classroom. [2] Of the entire test, the most well-developed and sophisticated part is its performance task component, [3] in which students are given ninety minutes to respond to a writing prompt that is associated with a set of background documents. The testing materials, including the documents, are accessed through a computer. CAE has published several examples of representative performance tasks, one of which is described below (from Academically Adrift: Limited Learning on College Campuses):

The “DynaTech” performance task asks students to generate a memo advising an employer about the desirability of purchasing a type of airplane that has recently crashed. Students are informed: “You are the assistant to Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech’s sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235.” Students are provided with the following set of documents for this activity: newspaper articles about the accident, a federal accident report on in-flight breakups in single engine planes, Pat Williams’s e-mail to her assistant and Sally Evans’s e-mail to Pat Williams, charts on SwiftAir’s performance characteristics, an article from Amateur Pilot magazine comparing SwiftAir 235 to similar planes, and pictures and descriptions of SwiftAir models 180 and 235. Students are then instructed to “prepare a memo that addresses several questions, including what data support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups, what other factors might have contributed to the accident and should be taken into account, and your overall recommendation about whether or not DynaTech should purchase the plane." [4]

CAE also publishes its scoring rubric. [5] The design of both the prompts and the criteria for evaluation demonstrates the CLA's focus on complex, holistic, real-world problem-solving as a measurement of high level learning. As a result, the argument goes, institutions that attempt to “teach to the test” will be schools that teach students “to think critically, reason analytically, solve problems, and communicate clearly." [6]

Criticisms

Again according to Academically Adrift, there are four primary criticisms of the CLA. Two criticisms relate to the validity of the instrument:

  1. By focusing on general skills rather than domain knowledge and specialization, the instrument may lack the construct validity to measure the specialized knowledge that students go to college to obtain.
  2. The CLA lacks instrumental validity to measure individual performance. This concern, however, may have been partially addressed by a 2009 test validity study organized by the Fund for the Improvement of Postsecondary Education (FIPSE). The results showed that while these tests should not be used as a basis to make institutional decisions about students as individuals (e.g., promotion or course placement), when aggregated in larger samples they can provide reliable estimates of institutional or group-level differences in performance on these tasks. [7] [8]

Two other criticisms relate to the normative implications of the CLA:

  1. An aversion to standardized testing in general and a belief that there is accountability in higher education due to market forces. That is, if students were not gaining something through education then they would not be paying for it. On this view, the CLA is a waste of time and money.
  2. The use of the test implies that the value-added from higher education can be measured. Individuals who oppose the idea that learning consists of acquiring specific knowledge or mental constructs, or is intended to improve performance on some measure (such as income or lifespan), may therefore oppose the measurement of learning.

See also

Voluntary System of Accountability—an initiative developed by American Association of State Colleges and Universities (AASCU) and the National Association of State Universities and Land-Grant Colleges (NASULGC) for 4-year public colleges and universities. The VSA endorses the use of the CLA for reporting out student learning outcomes through the College Portrait.

Related Research Articles

The following outline is provided as an overview of and topical guide to education:

<span class="mw-page-title-main">SAT</span> Standardized test widely used for college admissions in the United States

The SAT is a standardized test widely used for college admissions in the United States. Since its debut in 1926, its name and scoring have changed several times; originally called the Scholastic Aptitude Test, it was later called the Scholastic Assessment Test, then the SAT I: Reasoning Test, then the SAT Reasoning Test, then simply the SAT.

Standardized test Test administered and scored in a predetermined, standard manner

A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions and interpretations are consistent and are administered and scored in a predetermined, standard manner.

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, aptitude and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests. Assessment can focus on the individual learner, the learning community, a course, an academic program, the institution, or the educational system as a whole. The word 'assessment' came into use in an educational context after the Second World War.

Charter Oak State College Public community college in New Britain, CT

Charter Oak State College is a public online college based in New Britain, Connecticut. The college was founded in 1973 by the Connecticut Legislature and offers associate, bachelor's, and master's degrees. The college is adjacent to Central Connecticut State University and is named for Connecticut's famous Charter Oak.

Electronic assessment, also known as digital assessment, e-assessment, online assessment or computer-based assessment, is the use of information technology in assessment such as educational assessment, health assessment, psychiatric assessment, and psychological assessment. This covers a wide range of activity ranging from the use of a word processor for assignments to on-screen testing. Specific types of e-assessment include multiple choice, online/electronic submission, computerized adaptive testing such as the Frankfurt Adaptive Concentration Test, and computerized classification testing.

The Standards of Learning (SOL) is a public school standardized testing program in the Commonwealth of Virginia. It sets forth learning and achievement expectations for core subjects for grades K-12 in Virginia's Public Schools. The standards represent what many teachers, school administrators, parents, and business and community leaders believe schools should teach and students should learn. The Virginia Department of Education, schools, and school systems routinely receive essential feedback on the effectiveness of implementation and address effective instructional strategies and best practices. The Standards of Learning is supportive of and a direct response to the No Child Left Behind Act, which was signed into law by then-President George W. Bush on January 8, 2002. They address student achievement in four critical areas: (1) English, (2) mathematics, (3) science, and (4) history/social studies. Students are assessed in English and mathematics in grades 3-8 and upon completion of certain high school level courses. Science and history SOL are administered in grades 4, 5, and 8 and at the end of completing high school courses in these respective subjects.

The California Assessment of Student Performance and Progress (CAASPP), known until February 2014 as the Measurement of Academic Performance and Progress (MAPP), measures the performance of students undergoing primary and secondary education in California. In October 2013, it replaced the Standardized Testing and Reporting (STAR) Program.

The Carnegie Unit and the Student Hour are strictly time-based references for measuring educational attainment used by American universities and colleges; the Carnegie Unit assesses secondary school attainment, and the Student Hour, derived from the Carnegie Unit, assesses collegiate attainment.

A norm-referenced test (NRT) is a type of test, assessment, or evaluation which yields an estimate of the position of the tested individual in a predefined population, with respect to the trait being measured. Assigning scores on such tests may be described as relative grading, marking on a curve (BrE) or grading on a curve. It is a method of assigning grades to the students in a class in such a way as to obtain or approach a pre-specified distribution of these grades having a specific mean and derivation properties, such as a normal distribution. The term "curve" refers to the bell curve, the graphical representation of the probability density of the normal distribution, but this method can be used to achieve any desired distribution of the grades – for example, a uniform distribution. The estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population. That is, this type of test identifies whether the test taker performed better or worse than other test takers, not whether the test taker knows either more or less material than is necessary for a given purpose. The term normative assessment is used when the reference population are the peers of the test taker.

In an educational setting, standards-based assessment is assessment that relies on the evaluation of student understanding with respect to agreed-upon standards, also known as "outcomes". The standards set the criteria for the successful demonstration of the understanding of a concept or skill.

Authentic assessment is the measurement of "intellectual accomplishments that are worthwhile, significant, and meaningful," as contrasted with multiple-choice tests. Authentic assessment can be devised by the teacher, or in collaboration with the student by engaging student voice. When applying authentic assessment to student learning and achievement, a teacher applies criteria related to “construction of knowledge, disciplined inquiry, and the value of achievement beyond the school.”

The Rhode Island Department of Education (RIDE) is a state agency in Rhode Island that oversees the elementary and secondary education system from pre-Kindergarten through high school. It is headquartered in Providence. RIDE works closely with the Rhode Island Office of the Postsecondary Commissioner (RIOPC), the agency charged with overseeing higher education. Together, RIDE and RIOPC aim to provide an aligned, cohesive, and comprehensive education for all students.

Richard Arum is an American sociologist of education and stratification, best known for his research on student learning, school discipline, race, and inequality in K-12 and higher education.

<i>Academically Adrift</i>

Academically Adrift: Limited Learning on College Campuses is a book written by Richard Arum and Josipa Roksa, published by the University of Chicago Press in January 2011.

College and university rankings in the United States order the best U.S. colleges and universities based on factors that vary depending on the ranking. Rankings are typically conducted by magazines, newspapers, websites, governments, or academics. In addition to ranking entire institutions, specific programs, departments, and schools can be ranked. Some rankings consider measures of wealth, excellence in research, selective admissions, and alumni success. There is also much debate about rankings' interpretation, accuracy, and usefulness.

Placement testing is a practice that many colleges and universities use to assess college readiness and determine which classes a student should initially take. Since most two-year colleges have open, non-competitive admissions policies, many students are admitted without college-level academic qualifications. Placement tests assess abilities in English, mathematics and reading; they may also be used in other disciplines such as foreign languages, computer and internet technologies, health and natural sciences. The goal is to offer low-scoring students remedial coursework to prepare them for regular coursework. Less-prepared students are placed into various remedial situations, from adult basic education through various levels of developmental college courses.

Writing assessment refers to an area of study that contains theories and practices that guide the evaluation of a writer's performance or potential through a writing task. Writing assessment can be considered a combination of scholarship from composition studies and measurement theory within educational assessment. Writing assessment can also refer to the technologies and practices used to evaluate student writing and learning.

The Teaching Excellence and Student Outcomes Framework (TEF) is a controversial government assessment of the quality of undergraduate teaching in universities and other higher education providers in England, which may be used from 2020 to determine whether state-funded providers are permitted to raise tuition fees. Higher education providers from elsewhere in the United Kingdom are allowed to opt-in, but the rating has no impact on their funding. The TEF rates universities as Gold, Silver or Bronze, in order of quality of teaching. The first results were published in June 2017. This was considered a "trial year" and is to be followed by a "lessons learned exercise" that will feed into the 2018 TEF and longer-term plans for subject-level ratings.

Disengagement compact is the name assigned by educator George Kuh in 1991 to the tacit agreement between college teachers and their students that if teachers will minimize academic demands and grade generously, students for their part will write favorable course reviews and will allow teachers undisturbed time to focus on the research and publishing that their institutions reward with promotions and tenure. Commentators in the United States and Canada attribute the disengagement compact to market forces acting since the 1960s. The disengagement compact has been most discussed — and lamented — by educators convinced that engagement with teachers builds student competence in critical thinking, analytical reasoning, problem solving, and writing. Kuh maintains that the disengagement compact diminishes not only the skills acquisitions closely associated with academic learning but also the students' personal growth that teachers historically had helped to advance by engaging with students outside the classroom as well as inside. Beginning in 2000, educator and author Murray Sperber brought the disengagement compact to the attention of the general reading public, emphasizing its upsurge in large research universities.

References

  1. Benjamin, Robert; Chun, Marc (2003). "A New Field of Dreams: The Collegiate Learning Assessment Project". Peer Review. 5 (4): 26–29. Retrieved 2011-11-19.
  2. Hersch, Richard (2007). "Going Naked". Peer Review. 9.
  3. Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. 536: University of Chicago Press.{{cite book}}: CS1 maint: location (link)
  4. Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. University of Chicago Press.
  5. "Collegiate Learning Assessment Common Scoring Rubric". Council for Aid to Education. New York: Council for Aid to Education, 2008. Retrieved 2013-05-16.
  6. Shavelson, Richard (2008). "The Collegiate Learning Assessment". Ford Policy Forum 2008: Forum for the Future of Higher Education.
  7. Arum, Richard; Roksa, Josipa (2011). Academically Adrift: Limited Learning on College Campuses. University of Chicago Press.
  8. Klein, Stephen; Ou Lydia Liu; James Sconing (September 29, 2009). "Test Validity Study (TVS) Report" (PDF).{{cite journal}}: Cite journal requires |journal= (help)