Les Perelman

Last updated
Les Perelman
Les Perelman1.jpg
Born
Leslie Cooper Perelman

Education University of California,
Berkeley
(BA)
University of Massachusetts Amherst (MA, PhD)
OccupationEducator
Known forCriticism of standardized testing

Leslie Cooper Perelman is an American scholar and authority on writing assessment. [1] [2] [3] He is a critic of automated essay scoring (AES), [4] [5] and influenced the College Board's decision to terminate the Writing Section of the SAT. [6]

Contents

Perelman is currently a research affiliate at the Massachusetts Institute of Technology (MIT). [7] Perelman taught writing and composition at MIT, where he served as Director of Writing Across the Curriculum and an Associate Dean of Undergraduate Education. [8] He was an executive committee member of the Conference on College Composition and Communication. [9] and co-chair of its Committee on Assessment. [10]

Teaching

Perelman taught in and directed writing programs at Tulane University and the University of Southern California. At MIT, he taught writing and composition and served as the director of Writing Across the Curriculum and an Associate Dean in the Office of Undergraduate Education. [8]

Criticism of essay scoring

SAT

Following his 2005 study of essay samples as well as graded essays provided by the College Board for reference on the writing portion of the SAT, Perelman reported a high correlation between an essay's length and score. He also noted that the essays were not penalized for any factual inaccuracies. [11]

In 2013, Perelman met with David Coleman, the incoming president of the College Board, and an outcome of that conversation was Coleman's decision to abolish the mandatory SAT Writing Section. [12]

Automatic scoring

In 2012, Perelman demonstrated that long, pretentious essays could achieve higher scores from the ETS scoring engine e-Rater as opposed to well written essays. [13] In 2014, Perelman collaborated with students at MIT and Harvard to develop BABEL, the "Basic Automatic B.S. Essay Language" Generator. The nonsense essays generated by BABEL are claimed to perform well when graded by AES systems. Automated graders, Perelman argues, "cannot read meaning, and they cannot check facts. More to the point, they cannot tell gibberish from lucid writing." [14] Perelman's work is cited by the NCTE in their Position Statement on Machine Scoring, which expresses similar concerns about the limitations of AES:

Computer scoring systems can be "gamed" because they are poor at working with human language, further weakening the validity of their assessments and separating students not on the basis of writing ability but on whether they know and can use machine-tricking strategies. [15]

Influence on Australian Educational Testing

During 2017-2018, Perelman was commissioned by the New South Wales Teachers Federation to write three reports [16] [17] [18] to assist in efforts to reform Australia's national primary and secondary school assessments, the National Assessment Program – Literacy and Numeracy (NAPLAN). His work was a major factor in the decision of the National Education Council to overrule the Federal Education Minister and prevent the use of Automated Essay Scoring for the writing portion of the NAPLAN tests. [19]

Related Research Articles

<span class="mw-page-title-main">ACT (test)</span> American standardized test used for college admissions

The ACT is a standardized test used for college admissions in the United States. It is currently administered by ACT, a nonprofit organization of the same name. The ACT test covers four academic skill areas: english, mathematics, reading, and scientific reasoning. It also offers an optional direct writing test. It is accepted by all four-year colleges and universities in the United States as well as more than 225 universities outside of the U.S.

<span class="mw-page-title-main">SAT</span> Standardized test widely used for college admissions in the United States

The SAT is a standardized test widely used for college admissions in the United States. Since its debut in 1926, its name and scoring have changed several times. For much of its history, it was called the Scholastic Aptitude Test and had two components, Verbal and Mathematical, each of which was scored on a range from 200 to 800. Later it was called the Scholastic Assessment Test, then the SAT I: Reasoning Test, then the SAT Reasoning Test, then simply the SAT.

<span class="mw-page-title-main">General Certificate of Secondary Education</span> British public examinations, generally taken aged 15-16

The General Certificate of Secondary Education (GCSE) is an academic qualification in a range of particular subjects, taken in England, Wales, and Northern Ireland. State schools in Scotland use the Scottish Qualifications Certificate instead. However, private schools in Scotland often choose to follow the GCSE system in England.

<span class="mw-page-title-main">Standardized test</span> Test administered and scored in a predetermined, standard manner

A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions and interpretations are consistent and are administered and scored in a predetermined, standard manner.

<span class="mw-page-title-main">Educational Testing Service</span> Educational testing and assessment organization

Educational Testing Service (ETS), founded in 1947, is the world's largest private nonprofit educational testing and assessment organization. It is headquartered in Lawrence Township, New Jersey, but has a Princeton address.

The Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) is a standardized test administered by the College Board and cosponsored by the National Merit Scholarship Corporation (NMSC) in the United States. In the 2018–2019 school year, 2.27 million high school sophomores and 1.74 million high school juniors took the PSAT. Scores from the PSAT/NMSQT are used to determine eligibility and qualification for the National Merit Scholarship Program.

<span class="mw-page-title-main">College Board</span> US educational nonprofit testing organization

The College Board is an American not-for-profit organization that was formed in December 1899 as the College Entrance Examination Board (CEEB) to expand access to higher education. While the College Board is not an association of colleges, it runs a membership association of institutions, including over 6,000 schools, colleges, universities, and other educational organizations.

<span class="mw-page-title-main">Florida Comprehensive Assessment Test</span> Standardized test formerly used in public schools in Florida

The Florida Comprehensive Assessment Test, or the FCAT/FCAT 2.0, was the standardized test used in the primary and secondary public schools of Florida. First administered statewide in 1998, it replaced the State Student Assessment Test (SSAT) and the High School Competency Test (HSCT). As of the 2014-2015 school year FCAT was replaced in the state of Florida. The Florida Department of Education later implemented the Florida Standards Assessments (FSA) for English Language Arts, Reading, Mathematics and a Writing or typing test. A Comprehensive science test is still used for grades 5 and 8.

The National Council of Teachers of English (NCTE) is a United States professional organization dedicated to "improving the teaching and learning of English and the language arts at all levels of education. Since 1911, NCTE has provided a forum for the profession, an array of opportunities for teachers to continue their professional growth throughout their careers, and a framework for cooperation to deal with issues that affect the teaching of English." In addition, the NCTE describes its mission as follows:

The Council promotes the development of literacy, the use of language to construct personal and public worlds and to achieve full participation in society, through the learning and teaching of English and the related arts and sciences of language.

<span class="mw-page-title-main">Texas Assessment of Knowledge and Skills</span> Former Texas state standardized test

The Texas Assessment of Knowledge and Skills (TAKS) was the fourth Texas state standardized test previously used in grade 3-8 and grade 9-11 to assess students' attainment of reading, writing, math, science, and social studies skills required under Texas education standards. It is developed and scored by Pearson Educational Measurement with close supervision by the Texas Education Agency. Though created before the No Child Left Behind Act was passed, it complied with the law. It replaced the previous test, called the Texas Assessment of Academic Skills (TAAS), in 2002.

The National Curriculum assessment usually refers to the statutory assessments carried out in primary schools in England, colloquially known as standard assessment tasks (SATs). The assessments are made up of a combination of testing and teacher assessment judgements and are used in all government-funded primary schools in England to assess the attainment of pupils against the programmes of study of the National Curriculum at the end of Key Stages 1 and 2 where all pupils are aged 6, 7, 10 and 11 respectively. Until 2008, assessments were also required at the end of Key Stage 3 (14-year-olds) in secondary schools after which they were scrapped.

Holistic grading or holistic scoring, in standards-based education, is an approach to scoring essays using a simple grading structure that bases a grade on a paper's overall quality. This type of grading, which is also described as nonreductionist grading, contrasts with analytic grading, which takes more factors into account when assigning a grade. Holistic grading can also be used to assess classroom-based work. Rather than counting errors, a paper is judged holistically and often compared to an anchor paper to evaluate if it meets a writing standard. It differs from other methods of scoring written discourse in two basic ways. It treats the composition as a whole, not assigning separate values to different parts of the writing. And it uses two or more raters, with the final score derived from their independent scores. Holistic scoring has gone by other names: "non-analytic," "overall quality," "general merit," "general impression," "rapid impression." Although the value and validation of the system are a matter of debate, holistic scoring of writing is still in wide application.

<span class="mw-page-title-main">George Hillocks Jr.</span>

George Hillocks Jr. was an emeritus professor in the Department of Education, with a joint appointment in the Department of English Language and Literature at the University of Chicago. He received in 2011 the James R. Squire Award of the National Council of Teachers of English for having "a transforming influence and [making] a lasting intellectual contribution to the profession." He also received many other major awards. His teaching career included the preparation of English teachers in the Master of Arts in Teaching program, and the mentoring of Ph.D. students in the doctoral program, at the University of Chicago. After retiring from the University he continued to present seminars and workshops for writing teachers across the US. His primary research interests centered on the teaching of writing, literature, and language in middle and high school English classes, and on large-scale writing assessment. When not teaching and writing, he was an accomplished bagpipe player, performing frequently for Chicago audiences and in international competitions.

Year 7 is an educational year group in schools in many countries including England, Wales, Australia and New Zealand. It is the seventh full year of compulsory education and is roughly equivalent to grade 6 in the United States and Canada. Children in this year are between 11 to 12. Children who are in Year 7 can be considered as grade 6 in America or Canada.

<span class="mw-page-title-main">NAPLAN</span> Australias national school assessment program

The National Assessment Program – Literacy and Numeracy (NAPLAN) is a series of tests focused on basic skills that are administered to Australian students in year 3, 5, 7 and 9. These standardised tests assess students' reading, writing, language and numeracy and are administered by the Australian Curriculum, Assessment and Reporting Authority (ACARA). The National Assessment Program is overseen by the Education Ministers Meeting.

Automated essay scoring (AES) is the use of specialized computer programs to assign grades to essays written in an educational setting. It is a form of educational assessment and an application of natural language processing. Its objective is to classify a large set of textual entities into a small number of discrete categories, corresponding to the possible grades, for example, the numbers 1 to 6. Therefore, it can be considered a problem of statistical classification.

Ellis Batten “Bo” Page Ed.D. is widely acknowledged as the father of automated essay scoring, a multi-disciplinary field exploring computer evaluation and scoring of student writing, particularly essays. Page's development of and pioneering work with Project Essay Grade (PEG) software in the mid-1960s set the stage for the practical application of computer essay scoring technology following the microcomputer revolution of the 1990s.

Writing assessment refers to an area of study that contains theories and practices that guide the evaluation of a writer's performance or potential through a writing task. Writing assessment can be considered a combination of scholarship from composition studies and measurement theory within educational assessment. Writing assessment can also refer to the technologies and practices used to evaluate student writing and learning. An important consequence of writing assessment is that the type and manner of assessment may impact writing instruction, with consequences for the character and quality of that instruction.

Margaret Wu is an Australian statistician and psychometrician who specialises in educational measurement. She is an honorary professor at the University of Melbourne.

Mount Ousley Public School is a Kindergarten to grade six primary school, located in Fairy Meadow, New South Wales. The school was established in 1959, commonly known as 'MOPS'. In 2022, there were eight classes with 175 students in total, and 30 staff. The school is a secular government school primarily funded by NSW Department of Education and Parents and Citizens' Association (P&C).

References

  1. "NAPLAN's writing test is 'bizarre' but here's how kids can get top marks". ABC News. 8 April 2018.
  2. "Bientôt, les devoirs seront notés par des machines".
  3. "Essay-grading by software flawed, 'essentially impossible,' expert says".
  4. "Construct Validity, Length, Score, and Time in Holistically Graded Writing Assessments: The Case against Automated Essay Scoring (AES)" (PDF). WAC Clearinghouse . Retrieved June 14, 2015.
  5. "The BABEL Generator and E-Rater: 21st Century Writing Constructs and Automated Essay Scoring (AES)".
  6. "The man who killed the SAT essay". Boston Globe . Retrieved June 14, 2015.
  7. "People Directory". Massachusetts Institute of Technology . Retrieved June 11, 2015.
  8. 1 2 "iMOAT". Massachusetts Institute of Technology . Retrieved June 11, 2015.
  9. "2015 CCCC Officers and Executive Committee". National Council of Teachers of English . Retrieved June 11, 2015.
  10. "Committee on Assessment (November 2016)". National Council of Teachers of English . Retrieved July 29, 2016.
  11. Winerip, Michael (May 4, 2005). "SAT Essay Test Rewards Length and Ignores Errors". The New York Times . Retrieved June 11, 2015.
  12. Balf, Todd (6 March 2014). "The Story Behind the SAT Overhaul". The New York Times. Retrieved April 5, 2015.
  13. Winerip, Michael (22 April 2012). "Facing a Robo-Grader? Just Keep Obfuscating Mellifluously". The New York Times. Retrieved 5 April 2013.
  14. Kolowich, Steve (April 28, 2014). "Writing Instructor, Skeptical of Automated Grading, Pits Machine vs. Machine". The Chronicle of Higher Education . Retrieved June 11, 2015.
  15. "NCTE Position Statement on Machine Scoring". National Council of Teachers of English . Retrieved June 11, 2015.
  16. Perelman, Les. "Automated Essay Scoring and NAPLAN: A Summary Report" (PDF). New South Wales Teachers Federation. New South Wales Teachers Federation. Retrieved 28 December 2018.
  17. Perelman, Les (2018). Towards a New NAPLAN: Testing to the Teaching (PDF). Sydney: New South Wales Teacher Federation. ISBN   978-0-6482555-1-2 . Retrieved 28 December 2018.
  18. Perelman, Les. "Problems in the Design and Administration of the 2018 NAPLAN" (PDF). New South Wales Teacher Federation. Retrieved 28 December 2018.
  19. Koiziol, Michael. "Computer says no: Governments scrap plan for 'robot marking' of NAPLAN essays". No. 29 January 2018. Sydney Morning Herald. Retrieved 28 December 2018.