Standard: Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 Outcomes
Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 Outcomes
ERA and EI outcomes published
The Excellence in Research for Australia (ERA) and Engagement and Impact (EI) 2018 national reports, outcomes and other data are available on the Australian Research Council (ARC) website via the ARC Data Portal.
- ARC Data Portal
- ERA
- EI
ERA and EI Key Differences
ERA:
- Evaluates research quality
- Comprehensive data collection
- Fourth round in 2018
EI:
- Assesses the engagement and impact of research beyond academia
- Selective narrative and indicator based assessment
- Inaugural round in 2018
Useful background and terminology
FoR stands for Fields of Research from the Australian and New Zealand Standard Research Classification (ANZSRC). The FoR classification has three hierarchical levels of increasing specificity. The three levels are Divisions (broad disciplines), Groups (collections of related Fields) and Fields, respectively indicated by two, four or six digit classification numbers.
ERA 2018 Framework and Outcomes
How do we evaluate research quality in ERA?
- The unit of evaluation (UoE) is broadly defined as the Fields of Research (FoR) within an institution based on the Australia and New Zealand Standard Classification (ANZSRC).
- UoEs are 2-digit FoRs (22 broad disciplines) and 4-digit FoRs (157 specific disciplines).
- The indicators used in ERA include a range of metrics such as citation profiles which are common to disciplines in the natural sciences, and peer review of a sample of research outputs which is more broadly common in the humanities and social sciences.
- The outcomes (ratings) are determined and moderated by committees of distinguished researchers from Australia and overseas.
ERA Outcomes
Data (longitudinal across four ERA rounds)
- Discipline strengths
- Research output and indicator data
- University performance by discipline
- Workforce data–including gender
National UoE ratings ERA 2018 (aggregated four-digit results, grouped by two-digit FoR code)
Distribution of ERA 2018 ratings for all UoEs across all Australian universities
Distribution of rating results across ERA rounds
Change in UoE ratings between ERA 2015 and ERA 2018
HERDC research income by category and two-digit FoR code ERA 2018
All universities
HERDC research income by category across four ERA Rounds
All universities
Staffing profile by FTE, employment level and two-digit code ERA 2018
All universities
Staffing profile by headcount, employment level and two-digit code ERA 2018
All universities
Staffing profile by headcount and employment level across four ERA rounds
All universities
FTE staff by gender and employment level ERA 2018
All universities
EI 2018 Framework and Outcomes
EI 2018 Framework
Definitions
- Research is the creation of new knowledge and/or the use of existing knowledge in a new and creative way to generate new concepts, methodologies, inventions, and understandings. This could include the synthesis and analysis of previous research to the extent that it is new and creative.
- Aboriginal and Torres Strait Islander Research means that the research (as defined above) significantly:
- relates to Aboriginal and Torres Strait Islander peoples, nations, communities, language, place, culture or knowledges, and/or
- is undertaken with Aboriginal and Torres Strait Islander peoples, nations, or communities.
- Research impact is the contribution that research makes to the economy, society, environment or culture, beyond the contribution to academic research.
- Research engagement is the interaction between researchers and research end-users outside of academia, for the mutually beneficial transfer of knowledge, technologies, methods or resources.
- A research end-user is an individual, community or organisation external to academia that will directly use or directly benefit from the output, outcome or result of the research.
EI findings
EI 2018 provides a range of data. This data:
- is a rich new national resource
- includes examples of enabling institutional processes and infrastructure
- provides a benchmark of performance and
- includes best practice examples.
Engagement, Approach to Impact and Impact Ratings EI 2018
All universities
Percentages may not total 100 due to rounding
Charts showing national results for Engagement, Approach to Impact and Impact ratings for EI 2018
Additional data collected EI 2018
Additional data collected for EI 2018 includes:
- Additional Fields of Research - impact
- Additional Fields of Research - associated research
- Socio-Economic Objective (SEO) codes
- Australian and New Zealand Standard Industrial Classification (ANZSIC) codes
- Science and Research Priorities
- Keywords
- Beneficiaries
- Countries where impact is occurring
Keyword analyses
All universities
Word cloud showing some of the more common keywords that appeared in EI submissions nationally.
Countries of impact
All universities
Word cloud showing countries where university impacts are occurring, based on EI impact studies.
FoR mapping—Impact
All universities
FoRs are listed in numerical order. Some FoRs may not be labelled due to space limitations.
Sanke chart showing associated research FoRs mapped to impact FoRs.
Insights from EI 2018
Insights from EI 2018:
- Many pathways to impact
- Impact occurring in many research areas and in different ways
- Many types of collaborations
High rated engagement narratives
Common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20)
High rated engagement narratives – common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20):
- Engagement embedded in strategic plan
- Target the ‘right’ partner
- Engagement and impact linked to staff performance and promotion
- Dedicated business units – the ‘front door’
- Importance of ARC Linkage, ITRP and CRCs
- Embedded staff (between university and industry)
- Staff on advisory boards, committees etc. (between university and industry)
- Consultancies
- Specialist, end-user focused conferences and workshops
- Multi-media platforms
- Co-supervision and mentoring of HDRs
- Engagement with schools and educators
Engagement—discipline deep dives
STEM—06 Biological Sciences, 09 Engineering
Engagement discipline deep dives for STEM disciplines of Biological Sciences and Engineering. Common themes included:
- Co-location of partner industries to campus sites
- Provision of specialist services through unique infrastructure and equipment
- Reviews of national and international standards
- Citizen science
Engagement—discipline deep dives
HASS—18 Law, 20 Language, Communication and Culture
Engagement discipline deep dives for HASS disciplines of Law and Language, Communication and Culture. Common themes included:
- Working with policy makers and public interest groups – influence policy
- Involvement in developing/reviewing policy and professional standards
- Invitations from industry and governments to provide expert review/advice
- Invitations to provide submissions to inquiries
- Involvement in international agencies – WHO, UNICEF, UN
- Provision of access to specialist databases/repositories
- Provision of training for professionals
High rated approach to impact
Common themes from discipline deep dives (STEM 06 & 09, HASS 18 & 20)
Common themes from high rated ‘approach to impact’ submissions:
- Multi-disciplinary research centres
- important for creating larger research/end-user networks
- Formal partnerships
- Strong institutional, administrative and legal support to staff
- flexibility within staff contracts to support engagement – reduce teaching hours, providing time for collaboration and support for travel
- Co-location of research facilities
- Knowledge transfer experts
- Embedded/co-located staff and HDR students
- Multi-media platforms
- Competitive grants (supporting indirect costs)
Approach to impact—discipline deep dives
STEM—06 Biological Sciences, 09 Engineering
Approach to impact discipline deep dives for STEM disciplines of Biological Sciences and Engineering. A common theme was:
- Support for multi-disciplinary research centres, which was important for creating larger research/end-user networks.
Approach to impact—discipline deep dives
HASS—18 Law, 20 Language, Communication and Culture
Approach to impact discipline deep dives for HASS disciplines of Law and Language, Communication and Culture. Common themes included:
- Support for workshops, education and training services for training professionals and local communities
- Public seminars
- Websites and other social media
- Hiring across a broad range of expertise – including industry professionals
- Focus on providing early career researchers with engagement experience
Low rated engagement narratives—common themes
Common themes from low rated ‘engagement’ narratives:
- Engagement activities and research not clearly linked or well integrated
- Limited evidence of overarching engagement strategy and/or supporting processes
- Engagement indicators were comparatively low for the discipline with little explanation
- Little to no engagement activity in the reference period
- Limited evidence of mutually beneficial engagement
Low rated impact studies—common themes
Common themes from low rated ‘impact’ studies:
- Insufficient evidence to support the claims regarding research impact
- Identified potential impacts not actual impacts
- End-user impacts beyond academia not clearly articulated or of limited significance
- Impact stemmed from a non-research activity (e.g. training) or activity outside the reference period
- Link between research activity and claimed impact unclear.
Low rated approach to impact—common themes
Common themes from low rated ‘approach to impact’ submissions:
- Impact driven by individual(s) not the university
- Little evidence presented on university’s role
- University support considered business as usual
- University support not demonstrated to be strategic, well integrated or with the intention of facilitating impact
Aboriginal and Torres Strait Islander Research Results and Insights
Ratings for Aboriginal and Torres Strait Islander research
Charts showing the distribution of 'impact' and 'approach to impact' ratings for Aboriginal and Torres Strait Islander research.
High rated impact studies—who benefited?
High rated Aboriginal and Torres Strait Islanders studies—who benefited?
- Aboriginal and Torres Strait Islander people and communities across Australia
- professionals and industry groups
- general public
Examples:
- health care workers
- teachers and students
- artists and performers
- rangers
- journalists and publishers
- curators
- police officers and court officials
- business owners
- policy makers
Types of impacts
Types of impact (high rated Aboriginal and Torres Strait Islander impact studies):
- Improved physical and mental health
- Preserving and sharing of languages and culture
- Improved educational outcomes – school to tertiary
- Environmental preservation and management
- System wide legal changes
- Increased knowledge of First Nations history
- Economic savings in health and education
Additional impacts
Additional impacts:
- employment and upskilling of community members
- capability developed in industry professionals, and
- resources created and shared with sectors and public.
Mutual knowledge exchange was key.
Key themes of approaches to support impactful outcomes
Key themes of approaches to support impactful outcomes:
- Leadership and governance
- University investment, infrastructure and support
- Partnerships and communication
Examples of leadership and governance arrangements
Approach to impact – examples of leadership and governance arrangements:
- Frameworks
- Senior appointments
- Board appointments
- Advisory councils
- Strategic Plans
- Capability building next generation of leadership
[back to top]
Examples of university investment, infrastructure and support
Approach to impact – examples of university investment, infrastructure and support:
- Financial support
- Infrastructure
- Administrative support
- Build next generation of researchers
[back to top]
Examples of partnerships and communication approaches
Approach to impact – examples of partnerships and communication approaches:
- Formal agreements
- Long-term partnerships
- Appropriate communication and media
- Training and capability
[back to top]
ERA and EI next steps
ERA and EI next steps include the Review of Research Evaluation in Australia, which will consider:
- Key learnings ERA and EI
- Frequency of evaluations
- Reporting burden
- Sector feedback
- Various research reviews/reports-recommendations
[back to top]
Review of ANZSRC
Review of the Australian and New Zealand Standard Research Classification (ANZSRC):
- Joint review by the ARC, ABS, Statistics New Zealand, and the New Zealand Ministry of Business, Innovation and Employment
- Review will ensure research classifications reflect current practice and remain responsive to change in the sector
- Discussion Paper published earlier this year and public consultation closed on 7 June 2019
- Feedback is currently being examined and proposed revisions to ANZSRC will be published for comment later in 2019.
- Review anticipated to be completed mid-2020
[back to top]