Performance Based Research Fund

Last updated

The Performance Based Research Fund (PBRF) is a New Zealand tertiary education funding process, assessing the research performance of tertiary education organisations (TEOs) and then funding them on the basis of their performance. [1] The PBRF will provide $316 million to support the tertiary sector in 2018/19. [2]

Contents

The PBRF model has three elements to:

All New Zealand Tertiary Education Organisations (TEOs) who are approved by the New Zealand Qualifications Authority (NZQA) to award degrees, and also receive Student Achievement Component (SAC) funding from the Tertiary Education Commission (TEC) are entitled to participate in the PBRF Quality Evaluation process. [3]

Quality evaluation

The major element, the Quality Evaluation, is held periodically. The first was held in 2003, the second, a partial round in which not all staff were required to submit portfolios, was held in 2006, the third in 2012 and the latest full round was held in 2018. Due to COVID-19, the next round was delayed by a year and is due to be held in 2025. [4]

Each Quality Evaluation assesses the quality of research conducted at TEOs, and funding is allocated accordingly. Quality is determined by an assessment of research degree completion numbers, the amount of external research funding an institution achieves, and an evaluation of the individual research performance of all academic staff teaching on degrees or employed to conduct research.

Each academic staff member is required (with some exceptions) to submit an Evidence Portfolio which records their research outputs, contribution to research environment, and peer esteem. They are then assessed as A, B, C or R category. The A indicates international standing, B national, C local and R research inactive or active at a lower level.

From 2006 two new categories, C(NE) and R(NE) were introduced, for new and emerging researchers who have not yet had the benefit of a full six year census period. Each staff member is assigned a numerical grade (in 2006 5 for an A, 3 for a B, 1 for a C or C (NE), and 0 for R and R (NE)). This is used to calculate an overall score. Since the numerical scores assigned for the 2003 assessment and that for the 2006 assessment differed, the results of the two assessments are not entirely comparable, despite the 2006 assessment being designed to be a partial round.

In assessing individuals rather than groups, PBRF differs from the otherwise similar Research Assessment Exercise (RAE) in the United Kingdom.

New Zealand universities up to 2012

In 2012, The Victoria University of Wellington was ranked first for research quality. The research rankings up to that year are below. [5]

However these figures are based on the AQS(N) reported average, which measured research quality against the number of full-time equivalent staff receiving an A, B, or C grade. This measure was discontinued for the 2018 round.

RankUniversity2012 Quality score2006 Quality score2003 Quality score
1 Victoria 5.513.833.39
2 Auckland 5.124.193.96
3 Otago 4.964.233.23
4 Canterbury 4.794.103.83
5 Waikato 4.533.732.98
6 Massey 4.313.052.11
7 Lincoln 4.022.972.56
8 AUT 3.591.860.77
National average4.662.962.59

Other tertiary institutions (2012)

Other tertiary institutions scoring above 2.0 in the 2012 PBRF round including Polytechnics, Institutes of Technology and Private Training Establishments are below.

Institution2012 Quality score
Laidlaw College 3.25
Te Whare Wānanga O Awanuiārangi3.09
Unitec New Zealand2.94
Eastern Institute of Technology2.83
Otago Polytechnic2.79
Manukau Institute of Technology 2.76
Carey Baptist College 2.73
Christchurch Polytechnic Institute of Technology2.57
Wellington Institute of Technology 2.51
Northland Polytechnic 2.44
Whitireia Community Polytechnic 2.37
Waikato Institute of Technology 2.36

2018 Results (Universities)

These results are based on the AQS(S) - average quality, based on the number of teaching and research staff in a given tertiary education organisation, and is now the primary measure of research quality. [6] These figures were reported by the TEC in 2019 [7]

RankUniversity2018 Quality score
1 Victoria 29.19
3 Otago 26.09
4 Canterbury 25.92
2 Auckland 24.94
5 Waikato 21.76
6 Massey 19.5
7 Lincoln 17.64
8 AUT 15.78

Other tertiary institutions (2018)

Other tertiary institutions scoring above 2.0 in the 2018 PBRF round including Polytechnics, Institutes of Technology and Private Training Establishments are below.

Institution2018 Quality score
Carey Baptist College 10.11
Whitecliffe College of Arts and Design7.4
Good Shepherd College – Te Hepara Pai5.45
Unitec New Zealand 5.24
New Zealand College of Chiropractic5.09
Otago Polytechnic 3.8
Laidlaw College 3.05
Media Design School 2.73
Te Whare Wānanga O Awanuiārangi2.66
Bethlehem Tertiary Institute2.47
Eastern Institute of Technology 2.35
Waikato Institute of Technology 2.25


Related Research Articles

Standardized test Test administered and scored in a predetermined, standard manner

A standardized test is a test that is administered and scored in a consistent, or "standard", manner. Standardized tests are designed in such a way that the questions and interpretations are consistent and are administered and scored in a predetermined, standard manner.

The Victorian Certificate of Education is one credential available to secondary school students who successfully complete year 11 and 12 in the Australian state of Victoria. The VCE is the predominant choice for students wishing to pursue tertiary education. An alternative to VCE is the Victorian Certificate of Applied Learnings (VCAL), a vocational based senior secondary school qualification.

College and university rankings are rankings of institutions in higher education based on combinations of various factors. None of the rankings give a comprehensive overview of the strengths of the institutions because all select a range of quantifiable characteristics to base their results on. Rankings have been conducted by magazines, newspapers, websites, governments, and academics. In addition to ranking entire institutions, organizations rank programs, departments, and schools. Rankings consider various combinations of measures of funding and endowment, research excellence and/or influence, specialization expertise, admissions, student options, award numbers, internationalization, graduate employment, industrial linkage, historical reputation and other criteria. Various rankings mostly evaluating on institutional output by research. Some rankings evaluate institutions within a single country, while others assess institutions worldwide. The subject has produced much debate about rankings' usefulness and accuracy. The expanding diversity in rating methodologies and accompanying criticisms of each indicate the lack of consensus in the field. Further, it seems possible to game the ranking systems through excessive self-citations or by researchers supporting each other in surveys. UNESCO has questioned whether rankings "do more harm than good", while acknowledging that "Rightly or wrongly, they are perceived as a measure of quality and so create intense competition between universities all over the world".

Educational assessment or educational evaluation is the systematic process of documenting and using empirical data on the knowledge, skill, attitudes, and beliefs to refine programs and improve student learning. Assessment data can be obtained from directly examining student work to assess the achievement of learning outcomes or can be based on data from which one can make inferences about learning. Assessment is often used interchangeably with test, but not limited to tests. Assessment can focus on the individual learner, the learning community, a course, an academic program, the institution, or the educational system as a whole. The word 'assessment' came into use in an educational context after the Second World War.

No Child Left Behind Act 2002 United States education reform law; repealed 2015

The No Child Left Behind Act of 2001 (NCLB) was a U.S. Act of Congress that reauthorized the Elementary and Secondary Education Act; it included Title I provisions applying to disadvantaged students. It supported standards-based education reform based on the premise that setting high standards and establishing measurable goals could improve individual outcomes in education. The Act required states to develop assessments in basic skills. To receive federal school funding, states had to give these assessments to all students at select grade levels.

The Research Assessment Exercise (RAE) was an exercise undertaken approximately every five years on behalf of the four UK higher education funding councils to evaluate the quality of research undertaken by British higher education institutions. RAE submissions from each subject area are given a rank by a subject specialist peer review panel. The rankings are used to inform the allocation of quality weighted research funding (QR) each higher education institution receives from their national funding council. Previous RAEs took place in 1986, 1989, 1992, 1996 and 2001. The most recent results were published in December 2008. It was replaced by the Research Excellence Framework (REF) in 2014.

Programme for International Student Assessment Scholastic performance study by the OECD

The Programme for International Student Assessment (PISA) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) in member and non-member nations intended to evaluate educational systems by measuring 15-year-old school pupils' scholastic performance on mathematics, science, and reading. It was first performed in 2000 and then repeated every three years. Its aim is to provide comparable data with a view to enabling countries to improve their education policies and outcomes. It measures problem solving and cognition.

In US education terminology, rubric is "a scoring guide used to evaluate the quality of students' constructed responses". Put simply, it is a set of criteria for grading assignments. Rubrics usually contain evaluative criteria, quality definitions for those criteria at particular levels of achievement, and a scoring strategy. They are often presented in table format and can be used by teachers when marking, and by students when planning their work.

Authentic assessment is the measurement of "intellectual accomplishments that are worthwhile, significant, and meaningful," as contrasted with multiple-choice tests. Authentic assessment can be devised by the teacher, or in collaboration with the student by engaging student voice. When applying authentic assessment to student learning and achievement, a teacher applies criteria related to “construction of knowledge, disciplined inquiry, and the value of achievement beyond the school.”

Research Quality Framework (RQF) was a component of Backing Australia's Ability, an initiative of the Australian Government to formulate a best practice framework for assessing research quality and the impact of research, and ensure that public funding was being invested in research which would deliver real benefits to the wider community. RQF was to bring public funding of research in line with government policy for funding to be determined by outcomes achieved.

Journal ranking is widely used in academic circles in the evaluation of an academic journal's impact and quality. Journal rankings are intended to reflect the place of a journal within its field, the relative difficulty of being published in that journal, and the prestige associated with it. They have been introduced as official research evaluation tools in several countries.

Tertiary education in Australia

Tertiary education in Australia is formal education beyond high school, consisting of both government and private institutions and divided into two sectors; vocational education and training and higher education. 69% of Australians aged 20–64 have a non-school qualification, and 24% have multiple qualifications.

University of Auckland Faculty of Engineering

The University of Auckland Faculty of Engineering is one of eight faculties that make up the University of Auckland. Located on Symonds Street, Auckland, it has been consistently rated as the best Engineering School in New Zealand for quality of research.

In the United States education system, School Psychological Examiners assess the needs of students in schools for special education services or other interventions. The post requires a relevant postgraduate qualification and specialist training. This role is distinct within school psychology from that of the psychiatrist, clinical psychologist and psychometrist.

Assessment in computer-supported collaborative learning (CSCL) environments is a subject of interest to educators and researchers. The assessment tools utilized in computer-supported collaborative learning settings are used to measure groups' knowledge learning processes, the quality of groups' products and individuals' collaborative learning skills.

The Research Excellence Framework (REF) is a research impact evaluation of British higher education institutions. It is the successor to the Research Assessment Exercise and it was first used in 2014 to assess the period 2008–2013. REF is undertaken by the four UK higher education funding bodies: Research England, the Scottish Funding Council (SFC), the Higher Education Funding Council for Wales (HEFCW), and the Department for the Economy, Northern Ireland (DfE).

A virtual learning environment (VLE) in educational technology is a web-based platform for the digital aspects of courses of study, usually within educational institutions. They present resources, activities, and interactions within a course structure and provide for the different stages of assessment. VLEs also usually report on participation; and have some level of integration with other institutional systems.

The Higher Education Quality Improvement Program (MECESUP) in Chile is part of the efforts of the Government of Chile to support the transition from its current economy to one based on knowledge, increasing the equity and the effectiveness of its tertiary education system.

Tertiary education in New Zealand

Tertiary education in New Zealand is provided by universities, institutes of technology and polytechnics, private training establishments, industry training organisations, and wānanga. It ranges from informal non-assessed community courses in schools through to undergraduate degrees and research-based postgraduate degrees. All post-compulsory education is regulated within the New Zealand Qualifications Framework, a unified system of national qualifications for schools, vocational education and training, and 'higher' education. The New Zealand Qualifications Authority (NZQA) is responsible for quality assuring all courses and tertiary education organisations other than universities. Under the Education Act 1989, The Committee on University Academic Programmes (CUAP) and the Academic Quality Agency (AQA) have delegated authority for quality assurance of university education. The Tertiary Education Commission (TEC) is responsible for administering the funding of tertiary education, primarily through negotiated investment plans with each funded organisation.

Educator effectiveness is a United States K-12 school system education policy initiative that measures the quality of an educator performance in terms of improving student learning. It describes a variety of methods, such as observations, student assessments, student work samples and examples of teacher work, that education leaders use to determine the effectiveness of a K-12 educator.

References

  1. "Performance-Based Research Fund". 19 September 2016.
  2. "Vote Tertiary Education - Education and Workforce Sector - Estimates 2018/2019" (PDF). New Zealand Treasury.
  3. Tertiary Education Commission (2019). "Performance-Based Research Fund (PBRF) 2018 Quality Evaluation results: Frequently Asked Questions" (PDF).{{cite web}}: CS1 maint: url-status (link)
  4. Tertiary Education Commission (2021). "PBRF 2025 Quality Evaluation".{{cite web}}: CS1 maint: url-status (link)
  5. Tertiary Education Commission. "Performance-Based Research Fund Evaluating Research Excellence – the 2012 Assessment Final Report" (PDF). Archived (PDF) from the original on 2013-05-12. Retrieved 2013-04-12.
  6. Ministry of Education. "Fact Sheet: Changes to the Performance-Based Research Fund" (PDF).{{cite web}}: CS1 maint: url-status (link)
  7. Tertiary Education Commission (2019). "Improving Research Quality: The results of the PBRF 2018 Quality Evaluation" (PDF).{{cite web}}: CS1 maint: url-status (link)