Research Assessment Exercise

Last updated

The Research Assessment Exercise (RAE) was an exercise undertaken approximately every five years on behalf of the four UK higher education funding councils (HEFCE, SHEFC, HEFCW, DELNI) to evaluate the quality of research undertaken by British higher education institutions. RAE submissions from each subject area (or unit of assessment) are given a rank by a subject specialist peer review panel. The rankings are used to inform the allocation of quality weighted research funding (QR) each higher education institution receives from their national funding council. Previous RAEs took place in 1986, 1989, 1992, 1996 and 2001. The most recent results were published in December 2008. [1] It was replaced by the Research Excellence Framework (REF) in 2014.

Contents

Various media have produced league tables of institutions and disciplines based on the 2008 RAE results. Different methodologies lead to similar but non-identical rankings.

History

The first exercise of assessing of research in higher education in the UK took place in 1986 under the Margaret Thatcher Government. It was conducted by the University Grants Committee under the chairmanship of the Cambridge mathematician Peter Swinnerton-Dyer. The purpose of the exercise was to determine the allocation of funding to UK Universities at a time of tight budgetary restrictions. The committee received submissions of research statements from 37 subject areas ("cost centres") within universities, along with five selected research outputs. It issued quality rankings labelled "outstanding", "above average", "average" or "below average". The research funding allocated to universities (called "quality-related" funding) depended on the quality ratings of the subject areas. According to Swinnerton-Dyer, the objective was to establish a measure of transparency to the allocation of funding at a time of declining budgets. [2]

A subsequent research assessment was conducted in 1989 under the name "research selectivity exercise" by the Universities Funding Council. Responding to the complaint of the universities that they weren't allowed submit their "full strength," Swinnerton-Dyer allowed the submission of two research outputs per every member of staff. The evaluation was also expanded to 152 subject areas ("units of assessment"). According to Roger Brown and Helen Carasso, only about 40 per cent of the research-related funding was allocated based on the assessment of the submissions. The rest was allocated based on staff and student numbers and research grant income. [2]

In 1992, the distinction between universities and polytechnics was abolished. The Universities Funding Council was replaced by regionwise funding councils such as the HEFCE. Behram Bekhradnia, the directory of policy at HEFCE, came to the conclusion that the research assessment needed to become "much more robust and rigorous." This led to the institution of the Research Assessment Exercise in 1992. The results of the 1992 exercise were nevertheless challenged in Court by the Institute of Dental Surgery and the judge warned that the system had to become more transparent. The assessment panels in the subsequent exercises had to be much more explicit about the criteria for evaluation and the working methods. In 1996, all volume-based evaluation was removed to account for the criticism that volume rather than quality was rewarded. [2]

The 1992 exercise also stipulated that the staff submitted for assessment had to be in post by a specific date (the "census date") in order to counter the criticisms that the staff that had moved on were still counted in the assessment. This led to the phenomenon of "poaching" of highly qualified staff by other universities ahead of the census date. In the 2001 exercise, the credit for the staff that moved institutions in the middle of the cycle could be shared between the two institutions. In the 2008 exercise, this was abolished. [2]

The assessment of 2008 also brought in a major change. Instead of a single grade for an entire subject area ("unit of assessment"), a grade was assigned to each research output. This was done to counter the criticism that large departments were able to hide a "very long tail" of lesser work and still get high ratings and, conversely, excellent staff in low-graded departments were unable to receive adequate funding. Thus the single grades for units of assessment were replaced by "quality profiles," which indicated the proportion of each department's research against each quality category. [2]

Scale

2008

The 2008 RAE used a four-point quality scale, and returned a profile, rather than a single aggregate quality score, for each unit. The quality levels—based on assessment of research outputs, research environment and indicators of esteem—are defined as:

RatingDescription
4*Quality that is world-leading in terms of originality, significance and rigour
3*Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence
2*Quality that is recognised internationally in terms of originality, significance and rigour
1*Quality that is recognised nationally in terms of originality, significance and rigour
UnclassifiedQuality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

Each unit of assessment was given a quality profile – a five-column histogram – indicating the proportion of the research that meets each of four quality levels or is unclassified.

Earlier Assessments

In 1992, 1996 and 2001, the following descriptions were used for each of the ratings.

2001 & 1996 Rating1992 RatingDescription
5*5*Research quality that equates to attainable levels of international excellence in more than half of the research activity submitted and attainable levels of national excellence in the remainders.
55Research quality that equates to attainable levels of international excellence in up to half of the research activity submitted and to attainable levels of national excellence in virtually all of the remainder. (Same definition)
44Research quality that equates to attainable levels of national excellence in virtually all of the research activity submitted, showing some evidence of international excellence. (Same definition)
3a3Research quality that equates to attainable levels of national excellence in over two-thirds of the research activity submitted, possibly showing evidence of international excellence. (Research quality that equates to attainable levels of national excellence in a majority of the sub-areas of activity, or to international level in some)
3b3Research quality that equates to attainable levels of national excellence in more than half of the research activity submitted. (Research quality that equates to attainable levels of national excellence in a majority of the sub-areas of activity, or to international level in some)
22Research quality that equates to attainable levels of national excellence in up to half of the research activity submitted. (Same definition)
11Research quality that equates to attainable levels of national excellence in none, or virtually none, of the research activity submitted. (Same definition)

These ratings have been applied to "units of assessment", such as French or Chemistry, which often broadly equate to university departments. Various unofficial league tables have been created of university research capability by aggregating the results from units of assessment.[ citation needed ] Compiling league tables of universities based on the RAE is problematic, as volume and quality are both significant factors.

Assessment process

The assessment process for the RAE focuses on quality of research outputs (which usually means papers published in academic journals and conference proceedings), research environment, and indicators of esteem. Each subject panel determines precise rules within general guidance. For RAE 2008, institutions are invited to submit four research outputs, published between January 2001 and December 2007, for each full-time member of staff selected for inclusion. [3]

In response to criticism of earlier assessments, and developments in employment law, the 2008 RAE does more to take into account part-time workers or those new to a sufficient level of seniority to be included in the process.

Criticism

The RAE has not been without its critics. In its different iterations, it has divided opinion among researchers, managers and policy makers. [4] Amongst the criticisms is the fact that it explicitly ignores the publications of most full-time researchers in the UK, on the grounds that they are employed on fixed term contracts. According to the RAE 2008 guidelines, most research assistants are "not eligible to be listed as research active staff". [5] Publications by researchers on fixed term contracts are excluded from the Assessment Exercise unless those publications can be credited to a member of staff who is eligible for the RAE. This applies even if the member of staff being assessed only made a minor contribution to the article. [6] The opposite phenomenon is also true, where non-research active staff on permanent contracts, such as lecturers who have been responsible primarily for teaching activities have also found themselves placed under deeper contractual pressure by their employing universities to produce research output. Another issue is that it is doubtful whether panels of experts have the necessary expertise to evaluate the quality of research outputs, as experts perform much less well as soon as they are outside their particular area of specialisation. [7]

Since 1996 the AUT, now incorporated within the UCU, has maintained a policy of opposition to the Research Assessment Exercise. [8] In its view,

The RAE has had a disastrous impact on the UK higher education system, leading to the closure of departments with strong research profiles and healthy student recruitment. It has been responsible for job losses, discriminatory practices, widespread demoralisation of staff, the narrowing of research opportunities through the over-concentration of funding and the undermining of the relationship between teaching and research.

The official Review of Research Assessment, the 2003 "Roberts Report" commissioned by the UK funding bodies, [9] recommended changes to research assessment, partly in response to such criticisms.

The House of Commons Science and Technology Select Committee considered the Roberts report, and took a more optimistic view, [10] asserting that, "the RAE had had positive effects: it had stimulated universities into managing their research and had ensured that funds were targeted at areas of research excellence", it concluded that "there had been a marked improvement in universities' research performance". Nevertheless, it argued that "the RAE in its present form had had its day", and proposed a reformed RAE, largely based on Roberts' recommendations.

Planned changes to RAE system

It was announced [11] in the 2006 Budget that after the 2008 exercise a system of metrics would be developed in order to inform future allocations of QR funding. Following initial consultation with the higher education sector, it is thought that the Higher Education Funding Councils will introduce a metrics based system of assessment for subjects in science, technology, engineering and medicine. A process of peer review is likely to remain for mathematics, statistics, arts, humanities and social studies subjects.

HEFCE has developed a new set of arrangements, known as the Research Excellence Framework (REF), [12] which has been introduced as a follow on to the 2008 RAE.

Related Research Articles

<span class="mw-page-title-main">University of Bath</span> University in Bath, United Kingdom

The University of Bath is a public research university in Bath, England. It received its royal charter in 1966, along with a number of other institutions following the Robbins Report. Like the University of Bristol and University of the West of England, Bath can trace its roots to the Merchant Venturers' Technical College, established in Bristol as a school in 1595 by the Society of Merchant Venturers. The university's main campus is located on Claverton Down, a site overlooking the UNESCO World Heritage city of Bath, and was purpose-built, constructed from 1964 in the modernist style of the time.

<span class="mw-page-title-main">London Metropolitan University</span> University in London, England

London Metropolitan University, commonly known as London Met, is a public research university in London, England. The University of North London and London Guildhall University merged in 2002 to create the university. The University's roots go back to 1848.

<span class="mw-page-title-main">Middlesex University</span> Public university in Middlesex, London, England

Middlesex University London is a public research university in Hendon, northwest London, England. The name of the university is taken from its location within the historic county boundaries of Middlesex.

College and university rankings order institutions in higher education based on factors that vary depending on the ranking. Some rankings evaluate institutions within a single country, while others assess institutions worldwide. Rankings are typically conducted by magazines, newspapers, websites, governments, or academics. In addition to ranking entire institutions, specific programs, departments, and schools can be ranked. Some rankings consider measures of wealth, excellence in research, selective admissions, and alumni success. Rankings may also consider various combinations of measures of specialization expertise, student options, award numbers, internationalization, graduate employment, industrial linkage, historical reputation and other criteria.

<span class="mw-page-title-main">Russell Group</span> British association of universities

The Russell Group is a self-selected association of twenty-four public research universities in the United Kingdom. The group is headquartered in Cambridge and was established in 1994 to represent its members' interests, principally to government and Parliament. It was incorporated in 2007. Its members are often perceived as being the UK's best universities, but this has been disputed.

Three national rankings of universities in the United Kingdom are published annually, by The Complete University Guide, The Guardian and jointly by The Times and The Sunday Times. Rankings have also been produced in the past by The Daily Telegraph and Financial Times. UK Universities also rank highly in global university rankings with 8 UK Universities ranking in the top 100 of the three major global rankings: QS World University Rankings, Times Higher Education World University Rankings and Academic Ranking of World Universities.

The Higher Education Funding Council for England (HEFCE) was a non-departmental public body in the United Kingdom, which was responsible for the distribution of funding for higher education to universities and further education colleges in England since 1992. It ceased to exist as of 1 April 2018, when its duties were divided between the newly created Office for Students and Research England.

<span class="mw-page-title-main">Quality Assurance Agency for Higher Education</span> British non-profit organization

The Quality Assurance Agency for Higher Education, usually referred to simply as the Quality Assurance Agency or QAA, is the UK higher education sector's independent expert quality body, with a remit to maintain and enhance the quality of teaching and learning in tertiary education in the UK and beyond. It conducts quality assessment reviews, develops reference points and guidance for providers, and conducts or commissions research on relevant issues.

<span class="mw-page-title-main">Birmingham Newman University</span> University in Birmingham, UK

Birmingham Newman University is a public university based in the suburb of Bartley Green in Birmingham, England. The university was founded in 1968 as Newman College of Higher Education. From 2008 to 2013, it was known as Newman University College, gaining full university status in 2013. From 2013 to 2023, it was known as Newman University and Birmingham Newman University in 2023.

The Performance Based Research Fund (PBRF) is a New Zealand tertiary education funding process, assessing the research performance of tertiary education organisations (TEOs) and then funding them on the basis of their performance. The PBRF will provide NZ$316 million to support the tertiary sector in 2018/19.

Advance HE is a British charity and professional membership scheme promoting excellence in higher education. It advocates evidence-based teaching methods and awards fellowships as professional recognition for university teachers. Founded in 2003, the Higher Education Academy was responsible for the UK Professional Standards Framework for higher education practitioners and merged to form Advance HE on 21 March 2018.

The National Student Survey is an annual survey, launched in 2005, of all final year undergraduate degree students at institutions in England, Scotland, Wales and Northern Ireland within the United Kingdom. The survey is designed to assess undergraduate students' opinions of the quality of their degree programmes, with seven different scores published including an "overall satisfaction" mark.

Sir Gareth Gwyn Roberts was a Welsh physicist specialising in semiconductors and molecular electronics, who was influential in British science policy through his chairmanship of several academic bodies and his two reports on the future supply of scientists and how university research should be assessed. He was knighted in 1997 for his services to higher education.

Peninsula College of Medicine and Dentistry (PCMD) was a Medical and Dental school in England, run in partnership with the University of Exeter, the University of Plymouth and the NHS in Devon and Cornwall. In January 2013 the school began disaggregation to form Plymouth University Peninsula Schools of Medicine and Dentistry and the University of Exeter Medical School.

LLAS was a staff development centre based at the University of Southampton which provided services to academic staff across the UK teaching Languages, Linguistics and Area Studies in higher education. Its first incarnation was as a subject centre of the Learning and Teaching Support Network (2000-2004), then a subject centre of the Higher Education Academy (2004-2011). Its second incarnation was as an enterprise unit of the University of Southampton.

The Information School or iSchool of the University of Sheffield, in Sheffield, South Yorkshire, England, was founded in 1963 as the University's Postgraduate School of Librarianship and became in 2010 the first UK iSchool. Other names were the Postgraduate School of Librarianship and Information Science and Department of Information Studies (1981-2011). As of 2021, it employs 33 academic staff, 16 administrative/support staff, 6 affiliated research staff, and has about 65 research students. The current head of school is Professor Val Gillet.

The Research Excellence Framework (REF) is a research impact evaluation of British Higher Education Institutions (HEIs). It is the successor to the Research Assessment Exercise and it was first used in 2014 to assess the period 2008–2013. REF is undertaken by the four UK higher education funding bodies: Research England, the Scottish Funding Council (SFC), the Higher Education Funding Council for Wales (HEFCW), and the Department for the Economy, Northern Ireland (DfE).

<span class="mw-page-title-main">University of the Arts London</span> Collegiate university in London, England

University of the Arts London is a collegiate university in London, England, specialising in arts, design, fashion and the performing arts. It is a federation of six arts colleges: Camberwell College of Arts, Central Saint Martins, Chelsea College of Arts, the London College of Communication, the London College of Fashion and the Wimbledon College of Arts. It was established as The London Institute in 1986, became a university in 2003, and took its present name in 2004. The university hosts one of the largest international student bodies out of all universities in the United Kingdom.

<span class="mw-page-title-main">Sandra Kemp</span>

Sandra Kemp is an academic and curator with a background in English literature. She is the director, The Ruskin - Library, Museum and Research Centre at University of Lancaster and visiting professor in the Department of Materials at Imperial College London. She was previously a research associate at IMAGES&CO, and has held leadership roles in the university and cultural sectors, most recently as senior research fellow at the Victoria and Albert Museum, head of college, London College of Communication and director of research at the Royal College of Art (RCA). She curated the Wellcome Trust-sponsored exhibition Future Face: Image, Identity, Innovation at the Science Museum, with a related programme at the National Portrait Gallery, a film festival and a debate on BBC Radio Five Live. She has also published and given public lectures in the fields of fiction, literary theory and cultural studies.

The Teaching Excellence and Student Outcomes Framework (TEF) is a government assessment of the quality of undergraduate teaching in universities and other higher education providers in England, which may be used from 2020 to determine whether state-funded providers are permitted to raise tuition fees. Higher education providers from elsewhere in the United Kingdom are allowed to opt-in, but the rating has no impact on their funding. The TEF rates universities as Gold, Silver or Bronze, in order of quality of teaching. The first results were published in June 2017. This was considered a "trial year" and is to be followed by a "lessons learned exercise" that will feed into the 2018 TEF and longer-term plans for subject-level ratings.

References

  1. ↑ "RAE 2008". Research Assessment Exercise. Retrieved 20 August 2013.
  2. 1 2 3 4 5 Paul Jump, Evolution of the REF, Times Higher Education, 17 October 2013.
  3. ↑ "Definitions". RAE 2008. Retrieved 20 August 2013.
  4. ↑ Alis, Oancea (2014). "Research assessment as governance technology in the United Kingdom: findings from a survey of RAE 2008 impacts". Zeitschrift für Erziehungswissenschaft. 17: 83–110.
  5. ↑ RAE 2008 Guidelines Para 79
  6. ↑ Madden, Andrew (19 December 2008). "The researchers the RAE forgot". Guardian. Retrieved 20 August 2013.
  7. ↑ Corbyn, Zoe. "RAE's non-specialist gambit could have led to blunders, says study". Times Higher Education. Retrieved 20 August 2013.
  8. ↑ "RAE 2008". University and College Union. Retrieved 20 August 2013.
  9. ↑ Review of research assessment Archived 20 July 2007 at the Wayback Machine – report by Sir Gareth Roberts to the UK funding bodies, May 2003
  10. ↑ "Science and Technology – Eleventh Report". UK Parliament. 23 September 2004. Retrieved 5 July 2014.
  11. ↑ Shepherd, Jessica (30 January 2007). "A difficult patch". Guardian. Retrieved 20 August 2013.
  12. ↑ "Research Excellence Framework". REF 2014. Retrieved 20 August 2013.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.