The Peer Review Process
Conflict of Interest
The ARC Conflict of Interest and Confidentiality Policy is designed to ensure that all conflicts of interest are managed in a rigorous and transparent way. It aims to prevent individuals from influencing decisions unfairly and to maintain public confidence in the integrity, legitimacy, impartiality and fairness of the peer review process.
Any individual reviewing material for the ARC must agree to a confidentiality and conflict of interest statement, and must clearly disclose any conflicts of interest which may affect their ability to perform their role.
Examples of conflicts of interest for an assessor include holding funding with a named participant within the past two years, or having been a collaborator or co-author with a named participant on a research output within the last four years.
For more information on the timeframes that apply for common conflicts of interest, please refer to the Identifying and Handling a Conflict of Interest in NCGP processes page.
Unconscious Bias
It is widely accepted that everyone possesses unconscious biases which are automatic coping related responses based on our own background, culture and personal experiences. The unwitting effect of unconscious bias is discrimination and prejudice pervading the peer review process.
In order to mitigate the effects of unconscious bias, the ARC suggests all assessors undergo a process to identify and raise awareness of personal biases, question the basis for thoughts and feelings, build skills in cultural diversity and reflect on the strength of a culture that embraces differences.
Assessors are strongly encouraged to undertake Implicit Association Tests to assist in identifying unconscious biases prior to commencing assessments.
Assignment
For large schemes, the Selection Advisory Committee may be comprised of multiple discipline Panels. The Field of Research (FoR) codes nominated in the application will normally determine the Panel in which the application will be considered. ARC Executive Directors review the appropriateness of the applications in each Panel and may transfer applications to different Panels. They also identify interdisciplinary applications that need General Assessors from more than one Panel.
The responsible Executive Director assigns at least two General Assessors to each application (known as Carriages). One of these General Assessors will be designated as Carriage 1 (lead Carriage) and will be responsible for coordinating and leading discussion regarding the application at the selection meeting.
For some schemes, the General Assessors assign Detailed Assessors to each of the applications for which they are Carriage 1. For other schemes, Executive Directors assign applications to an appropriate number of Detailed Assessors.
FoR codes and expertise text within an assessor’s RMS profile plus the information provided in the application (FoR and Socio-economic Objective (SEO) codes and project summary) are used to match Detailed Assessors with applications within their area of expertise. It is therefore important that Detailed Assessors update their RMS profile regularly, including the Fields of Research codes and expertise text, to ensure that applications are assessed by the most relevant assessors. Detailed Assessors may decline an assignment where there is a significant mismatch of expertise. In that case the ARC will arrange a re-assignment where it is feasible to do so.
Assessment
Assessment criteria
The scheme selection criteria are provided in the Assessor Handbook and in the scheme Grant Guidelines. Assessment criteria and weightings vary from scheme to scheme depending on the nature and objectives of the scheme. Most scheme assessment criteria include ‘investigator’ and ‘proposed program of research’ components. Assessors must assess applications against the relevant scheme specific selection criteria.
Assessment against the ‘investigator’ criterion must include consideration of the opportunities the researcher has had to build their research profile. For example, researchers who are early in their research career or have had an interrupted research career, including employment outside academia, unemployment, caring and parental responsibilities and other personal circumstances, will have this taken into account. Similarly, researchers whose current or previous conditions of employment have allowed them a relatively large amount of research time will have this considered when assessors are examining opportunity to build a research profile. This provides assessors with information to determine whether a Chief Investigator (CI) has had extensive opportunity for research compared to a CI who may have had more limited time for research. In this way, the quality and benefit of achievements are given more weight than the quantity or rate of particular achievements enabling excellent researchers to be competitive regardless of their career path. Please see the ARC Research Opportunity and Performance Evidence (ROPE) Statement for more information.
Scoring Matrix
An A–E scoring scale is used by both Detailed and General Assessors to score applications against the assessment criteria. Please note that the wording and percentages used in the scoring scale vary between schemes. The Assessor Handbook for each scheme includes detailed descriptions of the criteria for each scoring band in the scoring matrix. Please consult the Assessor Handbook to ensure you use the correct rating scale for the scheme round under assessment.
Detailed Assessments
Detailed assessors submit online assessments to the ARC through the Research Management System (RMS). The merits of each application must be assessed against the selection criteria set out in the scheme Grant Guidelines.
The structure of the assessment form varies from scheme to scheme, but typically Detailed Assessors provide scores and written comments for each selection criterion against the scoring matrix provided in the Assessor Handbook. The comments given by the Detailed Assessors should correlate to the scores given and also provide constructive details that justify the ratings and enable an appropriate rejoinder.
For most schemes, the ARC aims to have a minimum of two Detailed Assessors assigned to each Application.
Please refer to the Assessors Resources page for further information regarding the ARC expectations for high quality assessments.
Rejoinder
In most ARC schemes, once Detailed Assessors submit their assessments to the ARC, applicants are invited to respond to the assessor comments via a rejoinder process.
Scores and assessor details are not released to applicants.
The rejoinder process allows applicants to clarify any misunderstanding or difference of opinion about perceived weaknesses in the application. This process also provides an important feedback mechanism that can help researchers improve future applications. The Detailed Assessments (scores and comments) and the rejoinders are then provided to the General Assessors (Carriages) assigned to that application.
General Assessments
General Assessors use the scoring matrix in the Assessor Handbook to assess an application's merits against selection criteria as set out in the scheme Grant Guidelines.
General Assessors are asked to undertake a preliminary assessment and scoring of their Carriage applications when they are first given access to their assignments. It is essential for Carriages to review their assignments immediately to assist in identifying conflicts early in the process. If a conflict is identified, the Carriage must reject the application assignment and the ARC can assign a new Carriage.
Once the rejoinder period has closed, General Assessors are given access to Detailed Assessor comments and scores, and the applicant’s rejoinder. General Assessors consider these when providing final application scores.
For most schemes, General Assessors are expected to discuss their assessment with the other General Assessor/s assigned to a given application before they finalise and submit scores to the ARC. This process of consultation is to assist where scores or opinions may not align and is essential for interdisciplinary/cross-panel assignments.
General Assessors submit online assessments to the ARC through the Research Management System (RMS).
Initial Ranking of Applications
The assessor scores each assessment criterion separately using the A–E scoring bands. A ‘Proposal Score’ which takes into account the criterion weightings for the specific scheme is automatically calculated when an assessor enters all the scores and saves the application assessment.
Proposal Scores from Detailed and General Assessors are then combined to provide a single value (the overall application score) for each application. This is performed by calculating the median of the average of the General ratings and the average of the Detailed ratings.
For some schemes, the General Assessor’s raw scores are normalised to redistribute all of the assessments provided by a General Assessor across the full spectrum of scores.
The creation of an overall application score allows applications to be sorted into an initial ranked list to be used as the basis of selection panel discussions.
Selection Meetings
Selection Advisory Committee (SAC) meetings for ARC schemes are held throughout the year.
SAC members (General Assessors) attend selection meetings to identify, discuss and recommend the highest quality applications and to recommend how much funding these applications should receive.
Selection of applications to recommend for funding
The SAC considers all application information, assessor reports and rejoinder statements in determining the final list for recommendation.
Budget recommendations
SAC members consider each budget request in detail and make a recommendation as to the level of funding to be provided. Considerations include value for money, the justification for the requested budget items, and whether the items are allowable under the scheme Grant Guidelines.
Approval
The SAC’s funding recommendations are provided to the relevant decision maker for consideration (the ARC Board for all NCGP schemes except designated schemes, which are reserved for decision by the Minister for Education). Under the ARC Act 2001, the Minister for Education may at any time decide that the ARC Board should not approve a grant on security, defence, or international relations grounds.
Once grants to be awarded have been approved, the ARC will publish a Network Message advising the announcement of grant outcomes on the ARC website and release of outcomes in RMS.