Introductory statement
I, as the accountable authority of the Australian Research Council (ARC), present the 2016–17 annual performance statement of the ARC, as required under paragraph 39(1)(a) of the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and the Australian Research Council Act 2001. In my opinion, these annual performance statements are based on properly maintained records, accurately reflect the performance of the entity, and comply with subsection 39(2) of the PGPA Act.
Professor Sue Thomas Chief Executive Officer
Purpose
The ARC’s purpose is to grow knowledge and innovation for the benefit of the Australian community by funding the highest quality research, assessing the quality, engagement and impact of research and providing advice on research matters (Outcome 1).
A statement is provided for each of the three key activities that support the ARC’s purpose as follows:
- Key Activity 1: Funding the highest quality research
- Key Activity 2: Assessing the quality, engagement and impact of research
- Key Activity 3: Providing advice on research matters.
Key Activity 1: Funding the highest quality research
Description
The ARC administers the National Competitive Grants Program (NCGP). It comprises two programs, Discovery and Linkage, which consist of a range of research funding schemes.
In 2016–17, the NCGP comprised the following schemes (see Appendix 1 for description):
- Discovery Program: Australian Laureate Fellowships; Discovery Early Career Researcher Award; Discovery Indigenous; Discovery Projects; and Future Fellowships
- Linkage Program: ARC Centres of Excellence; Industrial Transformation Research Hubs; Industrial Transformation Training Centres; Linkage Infrastructure, Equipment and Facilities; Linkage Learned Academies Special Projects; Linkage Projects; and Special Research Initiatives.
By supporting excellent internationally competitive research through these schemes, the ARC aims to deliver outcomes of benefit to Australia and to build Australia’s research capacity.
Deliverables
The key deliverables for the NCGP, as identified in the Portfolio Budget Statements (PBS) 2016–17, are (PBS pages 169 and 172):
- administering the Discovery Program funding schemes which provide funding to eligible administering organisations for excellent, internationally competitive research projects, fellowships and awards
- administering the Linkage Program funding schemes which provide funding to eligible administering organisations for excellent, internationally competitive research projects, infrastructure, hubs and centres
- informing policy on research and research training, and cross sector research partnerships.
In 2016–17, the ARC:
- conducted selection processes which considered 6412 proposals for NCGP funding and awarded 1346 new grants, involving 3094 researchers, worth $906.2 million
- administered 4996 new and ongoing grants under the NCGP, providing over $730 million in grant payments to administering organisations (based on planned allocations recorded at time of grant award)
- undertook a range of NCGP-related policy development activities (see pages 42-47 for further information).
Explanatory notes
The measures identified for Key Activity 1 provide an indication of the NCGP’s contribution to delivering outcomes of benefit to Australia and building Australia’s research capacity.
The measures assess outputs and/or outcomes at different stages of the grant lifecycle including at the time proposals are submitted to the ARC, successful research projects are announced, successful research projects are completed, and in the years following completion of the project.
Several points should be considered in reading this section.
- Some measures relate to matters which are outside the ARC’s influence, potentially impacting on the ARC’s ability to achieve these targets. Factors include economic conditions affecting the ability of partner organisations to make financial contributions to research projects, and the longer-term realisation of research benefits.
- Measuring performance against a target of maintaining or increasing certain performance levels is done in the context of multiple year trends rather than single year results.
Readers should note that figures reported in this section are grants awarded and do not directly reflect financial statement information.
Data notes
Readers should also note the following when reading this section:
- Data for the Discovery and Linkage Programs includes the outcomes of scheme rounds with funding commencing in 2016–17 unless otherwise stated. Those scheme rounds are:
Discovery Program:
- Australian Laureate Fellowships (2016), Discovery Early Career Researcher Award (2017), Discovery Indigenous (2017), Discovery Projects (2017), and Future Fellowships (2016) schemes
Linkage Program:
- ARC Centres of Excellence (2017), Industrial Transformation Research Hubs (2017), Industrial Transformation Training Centres (2016), Linkage Infrastructure, Equipment and Facilities (2017), and Linkage Projects (2016) schemes.
- Data and information from final reports are drawn from final reports which have been approved by the ARC for research projects commencing in 2011 (noting that most ARC projects involve multi-year funding).
- Due to a change in data collection protocols (that is, from final reports being required six months after completion of the research project to 12 months after completion of the research project), this is the same baseline year that was used in the ARC Annual Report 2015–16.
ARC funded research is relevant to Australia’s needs and expectations
Why is this important?
Through identification of priority areas the Australian Government aims to focus research in areas that address challenges facing Australia. The ARC contributes to this aim by funding research in priority areas under the NCGP.
Performance
| KPI 1.1 | ARC-funded research is relevant to Australia’s needs and expectations | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 9; PBS 2016–17 page 170,174 | |||
| TARGET | Maintain or increase the proportion of ARC-funded research that addresses an area of research priority | |||
| COLLECTED | From proposal forms | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| NCGP | 67% Target n/a; first year of Science and Research Prioritie | 84% | 84% | 93% |
| DISCOVERY | 60% Target n/a; as above | 80% | 82% | 92% |
| LINKAGE | 84% Target n/a; as above | 96% | 91% | 98% |
Analysis of results
Approximately two-thirds of research projects awarded in 2016–17 involved research relevant to the Government’s nine Science and Research Priority areas: Food, Soil and Water, Transport, Cybersecurity, Energy, Resources, Advanced Manufacturing, Environmental Change and Health.
The lower result recorded for the Discovery Program (60 per cent) reflects the different objectives of the Discovery and Linkage Programs. The Discovery Program is aimed at supporting fundamental research, while the Linkage Program has a greater focus on supporting research partnerships between universities and organisations that will use the research conducted by universities.
Performance against the target cannot be measured in 2016–17 because different Government priorities were in place in different years. In 2016–17, the ARC measured against Science and Research Priority areas, in 2015–16 and 2014–15 against Strategic Research Priority areas and in 2013–14 against National Research Priority areas (see glossary for definitions).
The proportion of new research projects in the Science and Research Priority areas ranged from 56 per cent under the Future Fellowships scheme to 100 per cent under the ARC Centres of Excellence, Industrial Transformation Research Hubs and Industrial Transformation Training Centres schemes (Figure 4).
Of total funding awarded to new projects commencing in 2016–17, the priority areas allocated the largest proportions of funding were Advanced Manufacturing (33 per cent), Environmental Change (16 per cent), and Health (14 per cent) (Figure 5).
Additional context
All researchers applying for ARC funding are required to identify whether their research falls within a Science and Research Priority area. While funding is not specifically directed to these areas, most schemes ask assessors to consider whether the research will address or has the potential to address these areas.
In 2016–17, the ARC supported a number of priority areas in addition to the Science and Research Priority areas. It:
- funded research in identified Industrial Transformation Priority areas through the Industrial Transformation Research Program. In 2016–17, the Industrial Transformation Priority areas comprised Advanced Manufacturing; Food and Agribusiness; Oil, Gas and Energy Resources; Mining Equipment, Technology and Services; and Medical Technologies and Pharmaceuticals
- provided ongoing funding for a number of initiatives in priority areas which commenced in previous years including: the National Indigenous Research and Knowledges Network, the Special Research Initiative for the Antarctic Gateway, the Special Research Initiative for Type 1 Juvenile Diabetes, the Special Research Initiative for the Science of Learning, and the Special Research Initiative for Tropical Health and Medicine.
ARC-funded research builds Australia’s research capacity and ability to respond to emerging priorities
Why is this important?
Research performance is critically dependent on access to highly qualified and skilled personnel. The ARC’s aim is to foster research training and career development opportunities through its funding schemes, including opportunities under its Linkage Program for researchers to spend time in industry organisations.
Performance
| KPI 1.2 | ARC-funded research builds Australia’s research capacity and ability to respond to emerging priorities | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 9; PBS 2016–17 page 170,174 | |||
| TARGET | Maintain or increase the proportion of NCGP research that supports research training and early career researchers | |||
| COLLECTED | From proposal forms | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| NCGP | 74% Target n/a; first year of measure | n/a | n/a | n/a |
| DISCOVERY | 92% Target n/a; as above | n/a | n/a | n/a |
| LINKAGE | 56% Target n/a; as above | n/a | n/a | n/a |
Analysis of results
Approximately three-quarters of NCGP projects commencing in 2016–17 involved research training and/or early-career researchers (where the ARC defines an early career researcher as a researcher within five years of award of a PhD). The result for the Discovery Program is higher than the Linkage Program as it includes the Discovery Early Career Researcher Award scheme, which is specifically targeted at supporting early-career researchers.
Performance against the target in 2016–17 cannot be assessed. This is the first year this measure has been reported against and data for research training has been collected in an accessible format.
Additional context
In 2016–17, the ARC supported a range of research training and career development activities. It:
- provided direct support through:
- awards and fellowships to individuals at all career stages
- the Industrial Transformation Training Centres scheme, which is specifically targeted at funding Higher Degree by Research (HDR) students conducting research in Industrial Transformation Priority areas
- funding for postgraduate and postdoctoral researchers to work in teams with the world-class researchers awarded Australian Laureate Fellowships
- provided indirect support through other ARC-funded research projects, including:
- Discovery Projects and Linkage Projects
- ARC Centres of Excellence.
In addition, the ARC continued to:
- encourage the best researchers to undertake research at Australian universities by making all ARC fellowship and award schemes open to researchers whether they are already located in Australia or overseas (including returning Australians)
- include Research Opportunity and Performance Evidence (ROPE) as a selection criterion under all its funding schemes. ROPE requires assessors to look at opportunities available to a researcher when considering their track record
- administer the Discovery Indigenous scheme, which supports research programs led by Aboriginal and Torres Strait Islander researchers and aims to build the research capacity of HDR students and early career researchers
- encourage the participation of women through a number of targeted mechanisms.
Of the 3094 individual researchers supported on new NCGP grants commencing in 2016–17:
- 27 per cent were female researchers
- one per cent were Aboriginal and Torres Strait Islander researchers
- 12 per cent were early-career researchers
- ten per cent were recipients of fellowships and awards.
ARC-funded research encourages collaboration among researchers within and beyond the research sector
Why is this important?
Research is a collaborative activity with successful researchers and research organisations working together, partnering and networking to generate multiplier effects. The ARC’s aim in supporting engagement between universities and other sectors, is to maximise opportunities to build the scale and focus of research involving end-users, potentially resulting in direct benefits to the Australian community.
Performance
| KPI 1.3 | ARC-funded research encourages collaboration among researchers within and beyond the research sector | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 9; PBS 2016–17 pages 170,174 | |||
| TARGET | Maintain or improve proportion of partner organisations (stakeholders) that are satisfied with the research partnerships supported through ARC-funded research (Linkage Projects) | |||
| COLLECTED | From final report forms | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| LINKAGE | 98% Target met | 98% | 99% | 98% |
| TARGET | Maintain or increase the extent ARC-funded research involves collaboration with partner organisations, including industry or other end users—Average number of partner organisations involved on a proposal (Linkage Projects) | |||
| COLLECTED | From proposal forms | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| LINKAGE | 2.4 Target met | 2.3 | 1.9 | 2.3 |
| TARGET | Maintain or increase the extent ARC-funded research involves collaboration with partner organisations, including industry or other end users—Ratio of total funding (cash and in-kind) pledged by partner organisations to funding awarded by ARC (Linkage Projects) | |||
| COLLECTED | From proposal forms | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| LINKAGE | 1.9 Target met | 1.7 | 1.9 | 1.8 |
Analysis of results—Satisfaction of partner organisations
A partner organisation’s commitment of funding and other resources to a research project indicates an expectation of gain from that investment. Feedback from partner organisations analysed during 2016–17 indicated that a vast majority of partners (98 per cent) found that the collaborative research had been either ‘very beneficial’ or ‘beneficial’ to their organisation.
Analysis of results—Involvement of partner organisations
The average number of partner organisations involved in Linkage Program grants is an indicator of how Linkage schemes are encouraging collaboration between university and non-university researchers. In 2016–17, results under the Linkage Projects scheme were similar to results in previous years (Figure 6). Overall, the results continue to exceed the minimum requirement of one Australian partner organisation in all Linkage Program schemes.
Analysis of results—Contribution of partner organisations
Under both the Linkage Projects and Industrial Transformation Research Hubs schemes, the combined cash and in-kind contributions pledged by partner organisations are required to at least match the funding sought from the ARC. In 2016–17, this requirement was exceeded in both schemes. For the Linkage Projects scheme, every ARC dollar attracted $1.92 from partner organisations, while for research hubs every ARC dollar attracted $2.00 from partner organisations (Figure 7).
Additional context
Key funding schemes of the Linkage Program—Linkage Projects, Industrial Transformation Research Hubs and Industrial Transformation Training Centres—are specifically aimed at supporting collaborative research between universities and end-users, including industry. The Linkage Projects scheme involves a range of end-users including government, not-for-profit as well as industry, while the Industrial Transformation Research Program is more tightly focused on industry linkages (by virtue of the priorities supported by the scheme).
Following recent Government reviews the ARC has made a number of changes aimed at boosting collaboration between researchers and end-users. In 2016–17, the ARC:
- introduced a continuous application process for the Linkage Projects scheme on 1 July 2016. This measure was announced and implemented under the National Innovation and Science Agenda (NISA).
- made businesses with less than 20 employees exempt from the requirement for partner organisations to provide cash contributions under the Linkage Projects scheme. This recommendation was implemented with the release of new Linkage Projects funding rules which commenced on 1 January 2017.
Previously the ARC had also:
- introduced a new objective for the Industrial Transformation Research Program to ‘drive growth, productivity and competitiveness by linking key growth sectors’
- introduced a revised objective under the Linkage Projects scheme emphasising opportunities to obtain commercial benefits from research
- strengthened selection criteria for the schemes under the Linkage Program to focus on collaborative links with industry and commercial outcomes.
ARC-funded research enhances international engagement
Why is this important?
Research is a global activity with researchers seeking access to the best partners and facilities worldwide. The ARC’s aim in supporting international research collaboration is to maximise Australia’s contribution to, and benefits from, international research collaborations, partnerships, developments and policy. Benefits include access to ideas and resources.
Performance
| KPI 1.4 | ARC–funded research enhances international engagement | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 9; PBS 2016–17 pages 170,174 | |||
| TARGET | Maintain or increase the proportion of ARC-funded research that supports international collaboration (against a four year average) | |||
| COLLECTED | From proposal forms (instances of international collaboration) | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| NCGP | 71% Target met | 69% | 65% | 60% |
| DISCOVERY | 77% Target met | 74% | 69% | 65% |
| LINKAGE | 55% Target met | 56% | 50% | 46% |
Analysis of results
Over half of research projects funded under the Linkage Program and three-quarters under the Discovery Program involved collaboration with researchers or research groups overseas.
The proportion of ARC-funded research projects involving international collaboration has slowly increased over the last four years under the Discovery Program with figures for the Linkage Program dropping slightly in 2016–17 (Figure 8). The ARC encourages researchers to build international links by allowing project funding to be used for international travel and fieldwork. The mechanisms of support have not changed significantly in the past four years, indicating that the general upward trend of results for the Discovery Program represents a real change in levels of collaboration.
The proportion of research projects involving international collaboration range from 100 per cent under the ARC Centres of Excellence and Industrial Transformation Research Hubs schemes to 36 per cent under the Discovery Indigenous scheme (Figure 9). The most common mechanism of collaboration proposed is attending a workshop or conference (proposed by 70 per cent of projects), followed by short-term travel and short-term hosting of partner investigators.
Additional context
In pursuing this result in 2016–17, the ARC:
- identified international travel costs as eligible budget items under most NCGP funding schemes
- identified overseas organisations as eligible partner organisations under the Linkage Program schemes
- awarded Discovery International Awards under the Discovery Projects scheme to increase opportunities for collaboration between researchers, research teams and/or research centres in Australia and overseas
- continued to foster the cooperative use of international facilities under the Linkage Infrastructure, Equipment and Facilities scheme.
In addition to the results against KPI 1.4:
- approximately 29 per cent of research projects awarded funding under the Linkage Projects scheme involved international organisations. In total 121 international organisations were nominated as partner organisations on Linkage Projects grants commencing in 2016–17
- 135 Discovery International Awards were awarded under the Discovery Projects scheme for funding commencing in 2017
- a number of projects awarded funding under the Linkage Infrastructure, Equipment and Facilities scheme involved international collaboration:
- Australian membership of the International Ocean Discovery Program, the world’s largest collaborative research program in earth and ocean sciences
- equipment for international collaboration in the next-generation gravitational wave detector
- coherent laser links for space applications to enable continuing leadership and involvement in international space projects.
ARC funded research produces outcomes that improve our understanding of the world; and/or provide economic, environmental, social, health and/or cultural benefits to Australia
Why is this important?
Advances in knowledge and innovation make a significant contribution to Australia’s economy as well as to the health and prosperity of the Australian community.
Performance
| KPI 1.5 | ARC-funded research produces outcomes that improve our understanding of the world and/or provide economic, environmental, social, health and/or cultural benefits to Australia | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 9; PBS 2016–17 pages 170,174 | |||
| TARGET | Examples of benefits arising from ARC funded research | |||
| COLLECTED | From media reports | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| DISCOVERY | Target met | Met | Met | Met |
| LINKAGE | Target met | Met | Met | Met |
Analysis of results
The case studies in Part 4 demonstrate the breadth of outcomes resulting from ARC-funded research. The case studies outline research which is:
- creating high performance and low-cost energy storage solutions for a more sustainable future (case study 1)
- supporting economic development in regional and remote areas through a better understanding of the role of cultural and creative industries (case study 2)
- developing a new generation of sensors to detect and diagnose disease (case study 3)
- bringing about significant energy savings in buildings (case study 4)
- helping record and recognise the contributions of Aboriginal and Torres Strait Islander veterans and ex-service personnel (case study 5)
- providing infrastructure and technical expertise that will enable Australia to become a world leader in an exciting new field of modern physics (case study 6)
- expanding our understanding of significant historical events and helping to inform better protection and management of Australia’s maritime history and heritage (case study 7)
- producing technology that will reduce pollution from vehicle exhaust (case study 8)
- establishing Australia as a leader in innovation through the development of new technologies that recognise patterns in big data (case study 9)
- using DNA from Neanderthals to better understand today’s diseases (case study 10)
- collaborating internationally to create new materials that can be used in industrial processes (case study 11)
- producing knowledge to safeguard the future of an iconic Australian species—the platypus (case study 12)
- generating new insights into the connections between plate tectonics, past ocean chemistry and the evolution and extinction of life on Earth (case study 13)
- developing innovative products that can help perforated eardrums to heal (case study 14)
- helping farmers to better manage their crops and increase their farm productivity using technology (case study 15).
During the year the ARC continued to release information about ARC research on its website. It also published a booklet titled Making a difference: Outcomes of ARC-supported research.
Additional context
External acknowledgement is considered to be a proxy indicator of the excellence of the researchers supported by the ARC. ARC-funded researchers have a strong record of winning prizes and awards and this continued in 2016–17 (see Appendix 3 for details).
Performance summary
There were no changes in the ARC’s activities, organisational capability or environment that impacted significantly on its performance in delivering Key Activity 1: Managing research funding schemes.
As reported in last year’s annual report a key development during 2015–16 was the announcement that the ARC would be responsible for implementing a continuous application and assessment process under the Linkage Projects scheme from 1 July 2016. As a result of this initiative the ARC changed the way it conducts selection meetings for the Linkage Projects scheme, using videoconferencing arrangements rather than face-to-face meetings.
The ARC also continued its focus on efficient and effective delivery of the NCGP. In pursuing this objective in 2016–17, the ARC:
- continued to pursue best practice peer review processes (including addressing conflicts of interest, College of Experts, appeals process)
- reviewed the NCGP funding rules
- liaised regularly with institutions (including through regular information sessions and visits).
In March 2017, the ARC surveyed the higher education sector on the clarity of the Funding Rules and Funding Agreements, as well as general satisfaction with services provided by the ARC. The ARC received over 100 responses to the survey and the majority indicated satisfaction with the clarity of information within the Funding Rules and Agreements and the level of service provided by the ARC.
Key Activity 2: Assessing the quality, engagement and impact of research
Description
Excellence in Research for Australia (ERA) assesses research quality by discipline at Australian higher education institutions. Evaluated against international benchmarks, ERA identifies excellence across the full spectrum of research activities giving a rating between five, well above world standard, and one, well below world standard. ERA aims to improve Australia’s research capabilities and inform government policy.
In 2016–17, the ARC continued to develop a new Engagement and Impact (EI) assessment framework by testing possible methodologies through a pilot exercise. In 2018, the full assessment will run in conjunction with ERA to assess the engagement of university researchers with end-users. The assessment will also examine how universities are translating their research into economic, social, environmental, cultural and other impacts.
Together the frameworks:
- provide a unique, evidence-based resource to inform Australian Government research policy and the strategic direction of higher education institutions
- encourage researchers to produce high-quality research with real-world benefits.
Deliverables
The key deliverables for ERA, as identified in the PBS 2016–17 are:
- administering an evaluation framework to measure and report on the quality of research conducted at Australian higher education institutions
- developing a framework to measure engagement and impact of research conducted at Australian higher education institutions
- informing strategic policy advice on research quality, engagement and impact.
In 2016–17, the ARC:
- released State of Australian University Research 2015–16: Volume 2 Institutional Insights
- conducted the EI pilot submission and assessment phase
- continued to provide strategic policy advice on a wide range of research matters relating to the evaluation of research quality.
ERA provides assurance of the quality of research in Australia
Why is this important?
Providing assurance of the quality of research enables strategic planning by universities and the Government. This strategic planning further strengthens Australia’s research capabilities. It provides important information to prospective students and researchers about the performance of disciplines at individual universities and helps promote Australia’s research strengths on the world stage.
Performance
| KPI 2.1 | Evidence that ERA provides assurance of the quality of research in Australia | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 10; PBS 2016–17 page 176 | |||
| TARGET | ERA reports and activities inform Australian Government policy | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Target met | Met | Met | Met | |
Analysis of results
The three rounds of ERA conducted to date have established a data collection of the research activity and performance of Australia’s universities that spans over a decade. The dataset is a unique and valuable resource for a range of stakeholders. The ARC regularly responds to requests for information about the nature, extent and location of research strengths in particular disciplines. These requests come from individual researchers, postgraduates, government agencies and entrepreneurs.
ERA encourages excellent, internationally competitive research
Why is this important?
Improvements in research quality lead to a better return on investment in research, and improve the social rate of return of research1. The ARC’s aim is to ensure these outcomes can be realised for Australia through the conduct of a best practice evaluation of the quality of research at Australian higher education institutions.
Performance
| KPI 2.2 | ERA encourages excellent, internationally competitive research | |||
|---|---|---|---|---|
| SOURCE | CCP 2016–17 page 10; PBS 2016–17 page 176 | |||
| TARGET | ERA reports and activities inform strategic planning at eligible Australian higher education institutions | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Target met | Met | Met | Met | |
| TARGET | Research performance of Australian higher education institutions improves | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Not measured | Met | Not measured | Not measured | |
Analysis of results—Informs strategic planning activities
A survey of the websites of the 41 ERA participating universities revealed that the majority (approximately 54 per cent) of institutions referenced ERA in their strategic planning documents and/or in their annual report of performance. In addition, more than 90 per cent made references to ERA and their performance on their website.
One of the aims of ERA is to improve Australia’s research capacity and it is evident that Australian universities are using ERA data to support their strategic planning activities. Promotional material of universities frequently emphasise ERA outcomes as an indicator of current performance and/or to set targets in relation to future performance. These documents compare performance relative to both international and national benchmarks. They demonstrate that Australian universities are using ERA data to:
- build on existing and emerging research strengths
- address weaknesses and/or gaps in their research capacity
- identify opportunities for cross disciplinary research and collaboration
- benchmark performance against national strengths and weaknesses
- develop capacity in areas of strategic priority to their institutional missions, including aligning capacity with undergraduate teaching profile, research training load, and the concerns of local communities and other key stakeholders.
Analysis of results—Research performance improves
Performance against this target cannot be measured in 2016–17 because an ERA evaluation was not conducted.
ERA is conducted every three years with the most recent round being in 2015 and the next round to take place in 2018. The outcomes of ERA 2015 showed that the quality of research conducted by eligible Australian higher education institutions had improved since the previous evaluation.
The Engagement and Impact assessment encourages impactful engagement within and beyond the research sector
Why is this important?
Existing systems of research evaluation show that the transparent reporting of university performance drives institutions to modify and improve their behaviour. It is anticipated that the assessment and reporting of a university’s performance in both research engagement and impact will lead to greater collaboration between universities and research end-users. It is also expected to encourage improved performance in the translation and commercialisation of research. This in turn will deliver economic and social benefits and maximise the value of Australia’s public investment in research.
Performance
| KPI 2.3 | The Engagement and Impact assessment encourages impactful engagement within and beyond the research sector | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17 page 10; PBS 2016–17 page 176 | |||
| TARGET | Research engagement and impact (within and beyond the research sector) is benchmarked and improves | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Not measured; first round to be conducted in 2018 | n/a | n/a | n/a | |
Analysis of results
Performance against this target cannot be measured in 2016–17 because an EI assessment was not conducted.
In 2016–17, the ARC developed a methodology for the EI assessment and tested it in a pilot exercise. In November 2016, the ARC invited universities to participate in the pilot and nominate researchers and end-users for the pilot assessment panels. The pilot tested the methodology for engagement based on engagement indicators and narratives submitted by universities. The pilot also tested the methodology for impact based on impact studies submitted by universities. The pilot submission and assessment phases took place in May–June 2017, with almost all universities participating in the pilot.
Recognising the importance of Aboriginal and Torres Strait Islander research, the pilot also tested the inclusion of research impact studies in this area. The development of the appropriate methodology to recognise the engagement and impact of Aboriginal and Torres Strait Islander research is in its early stages. The ARC has commenced consultations with representatives of the Australian Indigenous research community and this will continue to be an important and integral part of development and implementation processes. Feedback received by the ARC so far indicates that a narrative approach is the most appropriate method to convey information and data on Aboriginal and Torres Strait Islander research particularly regarding engagement and impact. Information obtained from the pilot will be used to develop an approach for incorporating Aboriginal and Torres Strait Islander research in the EI assessment in 2018 and beyond. It will be informed by further consultations with relevant stakeholders.
Performance summary
There were no changes in the ARC’s activities, organisational capability or environment that impacted on its performance in delivering Key Activity 2: Measuring research excellence.
In 2016–17, the ARC worked on preparations for EI and ERA 2018. This included:
- recruiting and training panel members for the pilot EI assessment, and receiving submissions from universities participating in the pilot
- continuing its preparation for the upcoming 2018 ERA round, reviewing the guidance information for the 2018 submission process and seeking input from universities for updates to the ERA 2018 Journal List.
Stakeholder engagement continued to be an important input to the ARC’s processes, including for developing the EI methodology and preparing for the next ERA evaluation.
Key Activity 3: Providing advice on research matters
Description
The ARC’s commitment and contribution to policy development plays an essential role in facilitating excellent research outcomes for Australia. In doing so the ARC takes an active and collegiate approach to identifying and responding to emerging issues and challenges relevant to Australia’s research sector.
Through policy development and advice, the ARC aims to:
- reflect current Government priorities and initiatives in its operations
- contribute to the development of national research and innovation policy
- provide advice on research matters to the Government
- support the effective delivery of ARC programs.
Deliverables
As identified in the PBS 2016–17, policy advice underpins the ARC’s delivery of the NCGP, ERA and EI (PBS pages 158-166).
In 2016–17, the ARC:
- reviewed research-related policies and consulted with stakeholders regarding potential changes
- worked with the National Health and Medical Research Council (NHMRC) and Universities Australia to revise the Australian Code for the Responsible Conduct of Research
- participated in policy development across portfolios, including access to research outcomes
- continued to look for ways to increase the efficiency, comprehensiveness and timeliness of data gathering and reporting
- continued to monitor emerging issues relevant to the research sector
- monitored, incorporated and contributed to international developments in research.
ARC policy advice contributes to and responds to Australian Government policy development
Why is this important?
The ARC’s aim is to develop and implement policies and policy advice regarding research that supports achieving outcomes of benefit to Australia.
Performance
| KPI 3.1 | ARC policy advice contributes to and responds to Australian Government policy development | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17, page 11 | |||
| TARGET | ARC policy reflects government priorities and initiatives | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Target met | Met | Met | Met | |
Analysis of results
During 2016–17, the ARC:
- implemented initiatives from NISA and other reviews
- continued to implement the Government’s Science and Research Priority areas through all NCGP funding schemes, and Industrial Research Transformation Priority areas through the Industrial Transformation Research Program
- participated in a range of national innovation policy initiatives and discussions including the Performance Review of the Australian Innovation, Science and Research System and development of the 2030 Strategic Plan for Innovation, Science and Research
- worked cooperatively with other agencies, including the NHMRC and the Department of Education and Training on a range of research policy activities
National Innovation and Science Agenda
The ARC is leading implementation of two initiatives under NISA: (i) the introduction of continuous application and expedited decision-making processes under the Linkage Projects scheme, and (ii) a systematic national assessment of research engagement and impact at a university level by discipline, as a companion to the ERA exercise. In 2016–17, the ARC completed the first six months of the continuous submission and assessment process for Linkage Projects 2016, and initiated Linkage Projects 2017 from January 2017. In the first part of the year the ARC recruited and trained panel members for the pilot EI assessment, and in May, received submissions from universities participating in the pilot.
Review of Research Policy and Funding Arrangements (Watt Review)
The final report of the Watt Review, released in December 2015, included 28 recommendations. The Government formally responded to the review in May 2016, accepting all 28 recommendations. In addition to the NISA initiatives, the report included recommendations that relate to the ARC in a number of key areas including the use of experts with experience in industry in assessing proposals for funding, and intellectual property (IP) requirements relating to ARC-funded research.
Experts with experience in industry, collaboration and commercialising research have been identified and will be used to assess grant proposals for funding under schemes in the Linkage Program. The Industry Advisory Panel for the Industrial Transformation Research Program (ITRP) 2017, for example, has its membership sourced equally from industry and academia.
During 2016–17, the ARC started drafting a new IP management policy which will address the recommendations of the Watt Review about Source IP and use of the IP Toolkit model contract and term sheet. The ARC will consult with relevant government agencies and universities as part of the policy’s development.
Implementation of Science and Research Priority areas
Information about the ARC’s implementation of Science and Research Priority areas under the NCGP is provided on page 25.
National policy initiatives and discussions
The ARC made a submission to the Productivity Commission Inquiry into Data Availability and Use. The ARC submission recognised that open access to research data provides societal (informing better social, economic and other outcomes), research (contributing to the growth of science and research) and individual (flowing from the use and citation of researchers’ data outputs) benefits—and emphasised the ARC’s commitment to maximising these benefits.
The ARC participated in the Government’s Open Access Working Group (established under the Boosting Commercial Returns from Research Strategy) tasked with developing principles for open access to publicly funded research publications and data. In addition, the ARC was an active observer on a Universities Australia Deputy Vice Chancellors (Research) Committee working group tasked with developing a statement on access to research outputs.
Working with the National Health and Medical Research Council
As well as the ongoing work with the NHMRC and Universities Australia (UA) on revising the Australian Code for the Responsible Conduct of Research, the ARC also participated in the review of Aboriginal and Torres Strait Islander research ethics guidelines, and revisions to the National Statement on Ethical Conduct in Human Research (2007). In addition, the ARC also participated in the review of the Human Research Ethics Application (HREA). The ARC’s active participation in these reviews has ensured that the scope of these documents has become more relevant to research across all disciplines.
ARC policy advice supports best practice research
Why is this important?
Best practice research supports the conduct of high quality research and development of the robust evidence base required to drive discoveries of benefit to the community. It provides the basis for high quality research training and career development and delivers assurance to those who use outcomes of research for innovation. Good research practice also helps build assurance and public confidence in the research process and its outcomes.
Performance
| KPI 3.2 | ARC policy advice supports best practice research | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17, page 11 | |||
| TARGET | ARC policies support the NCGP particularly industry engagement and research workforce | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Target met | Met | Met | Met | |
Analysis of results
The ARC’s administration of the NCGP is supported by a range of policy documents. During 2016–17 the ARC:
- reviewed and re-released its Statement of Support for the Research Workforce, Statement of Support and Action Plan for Gender Equality; Statement of Support and Action Plan 2017 for Aboriginal and Torres Strait Islander Researchers, and Statement of Support for Interdisciplinary Research.
- drafted an ARC Intellectual Property Management Policy addressing the recommendations of the Watt Review (see page 44)
- released a revised ARC Research Integrity and Research Misconduct Policy further strengthening the ARC’s requirements in relation to best practice research. In conjunction with the Australian Code for the Responsible Conduct of Research, the revised ARC policy serves to support best practice research and safeguard confidence in the value of publicly funded research.
- released a revised ARC Open Access Policy.
Research workforce
The ARC Research Workforce Statement outlines the ways the ARC contributes to establishing a sustainable research workforce in Australia including by: funding the best research and researchers in all disciplines, supporting researchers at all career stages and from under-represented groups, providing funding support which gives researchers flexibility in shaping their careers, and providing support to attract and retain the highest quality researchers.
In 2016–17, the ARC reviewed the Research Opportunity and Performance Evidence selection criterion.
ARC policy advice engages researchers, including international organisations and researchers
Why is this important?
Involving stakeholders in the policy development and evaluation process helps to ensure policies and activities align with their expectations and needs. It increases the diversity of experience and knowledge available to the ARC in identifying relevant issues, analysing policy options and deciding on the appropriate policy instruments. Stakeholder feedback from implementation of policies gives the ARC exposure to valuable information about policy effects, resulting in more successful policy outcomes.
Performance
| KPI 3.3 | ARC policy advice engages researchers, including international organisations and researchers | |||
|---|---|---|---|---|
| SOURCE | CP 2016–17, page 11 | |||
| TARGET | ARC policies reflect national and international research policy developments | |||
| RESULTS | 2016–17 | 2015–16 | 2014–15 | 2013–14 |
| Target met | Met | Met | Met | |
Analysis of results
During 2016–17, the ARC:
- continued to engage with a broad range of stakeholders both nationally and internationally about research matters, including peer review and program evaluation
- continued to provide evidence-based policy advice, including investigating different ways of interrogating data to help reveal trends or issues of concern
- communicated any changes in its policies to stakeholders in Australia and internationally
- maintained an evaluation plan for key program initiatives
- through its consultation and development activities for ERA and EI, engaged a wide range of researchers, research leaders and managers and experts in research evaluation, both nationally and internationally.
National engagement
In 2016–17, the ARC engaged with academic peak bodies and university peak bodies on ERA and the EI assessment in a wide variety of forums. In 2016–17, the ARC obtained feedback from universities after ERA 2015, through both formal and informal mechanisms. This included direct requests for feedback, and face-to-face meetings with all participating universities, as well as meetings with a range of peak bodies. The ARC also sought feedback from the university sector on updating the ERA Journal List for 2018. This included a formal targeted consultation with the sector in early 2017. This work is informing the development of ERA 2018.
International engagement
The ARC hosted several international delegations in 2016–17, sharing information on the ERA methodology. The ARC also met with senior members of the United Kingdom (UK) research assessment and university sectors in London in early 2017 to gather and share information on measuring research engagement and impact.
The ARC included international researchers on the Steering Committee and working groups that are shaping the EI assessment methodology. Researchers are drawn from the UK, Germany, the United States (US) and Canada. A representative from the ARC visited Latin America to meet with government and industry representatives to promote Australia’s research strengths and showcase ERA as a tool for identifying Australian research strengths by university and by discipline.
The ARC also continued to be an active participant in the Global Research Council, attending both the Asia-Pacific regional meeting in Malaysia in November and the annual meeting in Canada in May.
The ARC hosted nine visits from international delegations promoting opportunities available to international researchers and research organisations through the NCGP. In addition, there were 18 instances where the ARC attended or contributed to meetings, interviews or engagements with international stakeholders. In 2017, the ARC consulted with 12 international research funding agencies on best practice peer review and assessment processes.
The ARC supported the Australian Government’s Science and Innovation Delegation to Europe in October and participated in the Australia-Germany Joint Science and Technology Meeting, the Australia- Switzerland Joint Science, Research and Innovation Committee Meeting, the Australia-France Science and Innovation Meeting and the Joint Science and Technology Cooperation Committee Meeting with the European Commission.
Performance summary
There were no changes in the ARC’s activities, organisational capability or environment that impacted its performance in delivering Key Activity 3: Providing policy advice.
A key development during the year was the conduct by the Government of a range of research and higher education review activities. This provided opportunities for the ARC to engage in discussions and respond to requests for information about a broad range of research and higher education issues.