ARC response to the Evaluation of the Industrial Transformation Research Program process and priorities
ARC response to the Evaluation of the Industrial Transformation Research Program process and priorities – PDF Format (210KB) – Word Format (42KB)
Purpose of this document
The purpose of this document is to provide the ARC’s response to the recommendations arising from the evaluation of the Industrial Transformation Research Program process and priorities.
The terms of reference of the evaluation are as outlined below.
Terms of reference
The evaluation’s terms of reference were to:
- Assess the effectiveness and efficiency of ITRP scheme design, processes and administration in meeting the objectives of supporting industry focused collaborative research (through ITRH) and higher degree by research and postdoctoral training (through ITTC), as well as addressing issues involved in the management of potentially complex collaborative projects.
- Assess the role and contribution of the ITRP in the broader context of Australian Government programs that support industry focused research, including its potential complementarity and overlaps with Industry Growth Centres, CRC, CRC-P and the Global Innovation Linkages Program.
ARC response to recommendations of the evaluation
Recommendation 1: Industrial Transformation Priorities
Recommendation
The ARC should consider revisions to improve the flexibility and clarity of the ITRP process with respect to the Industrial Transformation Priorities and the role of Industry Growth Centres.
Such revisions should seek to address stakeholder perceptions that the Industrial Transformation Priorities are narrow, that the Industry Growth Centres’ role in the ITRP process is unclear, and some reported difficulties in engagement with the Growth Centres. They may include the following elements:
- In the ITRP grant guidelines, retain the requirement for applications to address one or more of the Industrial Transformation Priorities.
- Continue to include all Industry Growth Centre priority areas in the Industrial Transformation Priorities, but also consider the inclusion of additional priorities for each round—in consultation with appropriate stakeholders, including relevant ministers and the Department of Industry, Innovation and Science (DIIS).
- Amend the ITRP guidelines to require applicants to engage with relevant industry experts (such as, but not exclusively, Growth Centres) to ensure the proposed research is targeted to support growth in the relevant Industrial Transformation Priority areas.
- Amend the ITRH and ITTC assessment criteria to require applicants to describe the ways in which engagement with industry experts (such as, but not exclusively, Growth Centres) has ensured that the proposed research is targeted to support growth in the relevant Industrial Transformation Priority areas.
- Ensure that industry expertise on ITRP Selection Advisory Committees covers any additional priority areas identified for each round.
The ARC should continue to work closely with DIIS and the Growth Centres in relation to their interaction with the ITRP and continue to emphasise the value of engagement with the Growth Centres in its ITRP advice and outreach activities.
ARC response
The ARC accepts this recommendation.
In the 2019 Linkage Program ITRP grant guidelines, the ARC:
- included reference to the Industry Growth Centres’ role in the assessment process in the process flowchart to clarify their role in the process
- retained the requirement for applications to address one or more of the Industrial Transformation Priorities in order to be eligible
- continued to include all Industry Growth Centre priority areas in the Industrial Transformation Priorities, and
- added engagement with industry experts (such as, but not exclusively, Growth Centres) to the assessment criteria.
The eligibility criteria for ITRP has not been changed to require applicants to engage with relevant industry experts in the 2019 Linkage Program ITRP grant guidelines.
The ARC undertook a range of outreach opportunities during the application period to promote the scheme and the benefits of engaging with Growth Centres and will continue to monitor applicant’s interaction with the Growth Centres and the ITRP.
Future scheme rounds can include new priorities if they are identified through consultation with key stakeholders and are agreed to by Government.
The ARC will maintain its usual process to establish a Selection Advisory Committee with expertise across all priority areas, as represented in applications.
Recommendation 2: Information on HDR students and PDRs
Recommendation
The ARC should collect additional data on HDR students and PDRs through Training Centre and Research Hub final reports.
To support improved monitoring and evaluation of ITRP support for industry focused research training, this should include data on the total number of HDR students and PDRs who participated over the life of the Training Centre or Research Hub, the nature of their involvement, whether they completed their training, and whether they were supported by ARC funding, partner organisation contributions, or other sources.
In addition, the ARC may wish to consider undertaking further survey research with HDR students and PDRs in future to better understand employment outcomes for ITRP research trainees.
ARC response
The ARC notes this recommendation.
The ARC currently requests the following information in Training Centre and Research Hub final reports:
- the number of ARC funded students on the project
- the gender breakdown of HDRs and PDRs
- the destination of PhD and Masters students, and
- whether the project was affected by recruitment issues.
The ARC will consider including additional questions regarding HDRs and PDRs in the final reports for future ITRP cohorts. The form for final reports for ITRP investments that commenced in 2014 was released in late 2019.
Recommendation 3: Data and monitoring
Recommendation
The ARC should consider using the metrics provided in this evaluation’s analysis as benchmarks to monitor performance of the ITRP and inform improvements.
Useful metrics from existing ARC data collections may include:
- number of partner organisations involved in funded projects
- types of partner organisations involved in funded projects
- international partner organisations involved in funded projects
- value of partner organisation cash and in kind contributions.
ARC response
The ARC accepts this recommendation.
The ARC will develop a set of metrics, based on those provided in the evaluation report, to monitor the performance of the ITRP and inform improvements.
Recommendation 4: Engagement, advice and outreach
Recommendation
The ARC should address issues highlighted in this evaluation in ongoing ITRP stakeholder engagement, advice and outreach.
In particular, the evaluation’s findings can help to inform the ARC’s advice to stakeholders in particular areas, such as:
- finding the right partners, including the importance of managing expectations and the dimensions of collaboration involved under the ITRP
- the views of HDR students and PDRs in relation to research training
- addressing key challenges in the ITRP application and establishment processes.
In addition, there may be opportunities for the ARC to use regular and ongoing engagement mechanisms to seek further information and feedback from stakeholders on issues raised in this evaluation, such as:
- the initiation of collaboration under the ITRP
- issues in the retention of partner organisations
- the usefulness of ARC advice and information (including, for example, in relation to the role of Growth Centres in the process).
ARC response
The ARC accepts this recommendation.
The ARC will undertake a targeted outreach program for ITRP before and during future application rounds. This outreach will involve the scheme Executive Director and scheme team personnel as required. The Industry Growth Centres will be approached to provide information and documentation or to participate in specific outreach opportunities.
The ARC is currently preparing for the next ITRP Forum, to be held prior to October 2020. The ARC will present on scheme application and assessment processes, including the role of the Industry Growth Centres. The Department of Industry, Innovation and Science, and representatives of the Growth Centres will be present to interact and discuss issues with potential applicants and research office staff.
The ARC hosts an ITRP Induction day for new ITRP Directors for each new scheme round. Topics include establishment processes such as establishment funds, recruitment, and launches. The ITRP induction day was held in November for the 2019 cohort of ITRP Directors. An induction day for the 2020 cohort of ITRP Directors is intended for the third quarter of 2020.
Furthermore, in 2019, the Major Investments Working Group developed an establishment toolkit for ARC Centres of Excellence, which provided the foundation for creating a similar toolkit for ITRP. The ITRP establishment toolkit has been developed for ITRP commencing in 2020 and is available on the ARC website.