These notes are intended to provide reviewers with specific guidance for the completion of the reviewer form. They should be read in conjunction with the reviewer principles.
Specific guidance is available for each section of the report to be completed. A full justification for your assessment of the application should be provided. The prompts are given as a reminder of those issues that are likely to be most significant in determining the overall merit of an application. Please provide as full a response as you believe you are qualified to. You should note that your review will be sent back, unattributed, to the investigator, who will then be allowed the opportunity to comment on any factual errors and answer any specific queries you have raised.
EPSRC is committed to support the recommendations and principles set out by the San Francisco Declaration on Research Assessment. You should not use journal-based metrics, such as journal impact factors, as a surrogate measure of the quality of individual research articles, to assess an investigator’s contributions, or to make funding decisions.
For the purpose of research assessment, please consider the value and impact of all research outputs (including datasets, software, inventions, patents, preprints and other commercial activities) in addition to research publications. You should consider a broad range of impact measures, including qualitative indicators of research impact such as influence on policy and practice.
The content of a paper is more important than publication metrics or the identity of the journal in which it was published, especially for early-stage investigators. Therefore, you should not use journal impact factor (or any hierarchy of journals), conference rankings and metrics such as the H-index or i10-index when assessing UKRI grants.
GEN2: Assessment methodology
You are asked to assess the application or report against a number of criteria. These criteria vary according to the scheme or funding opportunity that the application has been submitted to. Prompts are provided as a reminder of those issues that are likely to be most significant in determining the overall merit of an application. A full justification for your assessment of the application should be included in each section. Please provide as full a response as you believe you are qualified to.
You are asked throughout to assess ‘the application’ but be clear that this means the ideas, concepts and approaches contained therein, and not the specific form of the document itself. The clarity of presentation may help or hinder your ability to review an application, so a comment to this effect would be appropriate, but this should not become in any form a competition in stylish writing. Elegance of presentation is not, of itself, an assessment criterion for an EPSRC grant.
There is no set way for answering questions on the form. However, prioritisation meetings generally find reviews most useful where they explicitly identify the main strengths and weaknesses in the application, while also giving a clear view on which should be accorded the greater significance and why. It is also a helpful technique to raise issues or concerns with the application in the form of explicit questions for the applicants. This makes it easier for the panel to assess how complete and convincing the applicant’s responses are.
It is important that EPSRC funds are used ethically and responsibly. This is mainly assured by requiring that universities have in place and operate appropriate ethical approval processes. Ethical considerations should not therefore normally be an assessment criterion and you should not take these into account when making your assessment.
If the application is in a subject or area that causes you serious personal concern, to the extent that you feel you cannot provide an objective review, you should decline to review the application giving the reason as ‘other’, and stating ‘ethical issues’ in the comment box. If you have a concern that the application raises ethical issues that have not been clearly identified or addressed, you should raise this directly with EPSRC, which will need to make a policy decision on how the application should be treated.
GEN4: Linked proposals
Where two or more applications have been formally linked to form a single research project, you are requested to submit a single review covering the project as a whole.
GEN5: Web links in the application
The application you are asked to review includes a case for support. In some instances, the case for support may include a link to a web site containing information on the research proposed. Reviewers are not required to consider this additional information when providing comments on an application. If you do choose to look at this information, it is possible that your anonymity to the applicant will be compromised.
GEN8: Scope of scheme
Programme grants are a flexible mechanism to provide funding to world-leading research groups to address significant major research challenges. They are intended to support a variety of activities focusing on one strategic research theme. Although it is expected that most applications will be interdisciplinary and collaborative, they can address key challenges in a single discipline.
GEN10: Flexible working
It is important that researchers and their research teams are able to work flexibly and in a way that meets their personal circumstances. EPSRC therefore allows applicants to tailor the support that they request in order to facilitate this. This might include, for example:
- part-time working for the principal investigator, co-investigators, post-doctoral research assistants, technicians or wider team costed on a grant
- support for costs over and above standard care arrangements to allow the principal investigator or their team to attend activities associated with the grant (for example conferences) where costs cannot be met by the employer
- support for other adjustments and adaptations that may be needed due to the personal or health circumstances of the principal investigator or their team.
You should also consider the unequal impact that flexible working, alternative career routes and career breaks might have had on the track record and career development of those individuals included in the proposal. Flexible working includes, but is not limited to, reduction in full-time hours, long-term partial return to work, job sharing, compressed working hours, term-time only working, annualised hours.
Where applicants wish to include details on flexible working or a career break for reviewers to take into account in their assessment, they are not required to explain the personal circumstances that resulted in the need for this. Instead, they should describe the impacts on an individual’s track record and career development. Where reference has been made to a period of flexible working or a career break, you should recognise that this is likely to affect productivity and career development, for example publication record, track record of securing funding, or the ability to build networks or to take up opportunities in a different geographical location. You should also consider that this unequal impact may continue beyond the return to work. As such, the focus should be on the stated impact rather than the duration.
GEN11: Impact of COVID-19 on track record and career development
EPSRC recognises that the COVID-19 pandemic has caused major interruptions and disruptions across our communities and is committed to ensuring that individual applicants and their wider team, including partners and networks, are not penalised for any disruption to their career such as breaks and delays, disruptive working patterns and conditions, the loss of ongoing work, and role changes that may have been caused by the pandemic.
When undertaking your assessment of the research project, you should consider the unequal impacts of the impact that COVID-19 related disruption might have had on the track record and career development of those individuals included in the proposal, and you should focus on the capability of the applicant and their wider team to deliver the research they are proposing.
EPSRC acknowledges that it is a challenge for applicants to determine the future impacts of COVID-19 while the pandemic continues to evolve. Applicants have been advised that their applications should be based on the information available at the point of submission and, if applicable, the known application specific impacts of COVID-19 should be accounted for.
Where known impacts have occurred, these should have been highlighted in the application, including the assumptions and information at the point of submission. Applicants were not required to include contingency plans for the potential impacts of COVID-19. Requests for travel both domestically and internationally could be included in accordance with the relevant scheme guidelines, noting the above advice.
When undertaking your assessment of the research project, you should assess the project as written, noting that any changes that the project might require in the future, which arise from the COVID-19 pandemic, will be resolved as a post-award issue by UKRI if the project is successful. Potential complications related to COVID-19 should not affect your assessment or the score you give the project.
GEN12: Matched funding
EPSRC does not assess the presence or value of any matched funding provided by the university before making a funding decision. Unless specified in the call or scheme guidance documentation, EPSRC does not require matched funding, either cash or in kind, to secure funding.
EPSRC assessment processes, including expert reviewing and panels, may acknowledge the impact of university contributions, but will not consider the level of matched university funding as a factor on which to base funding decisions.
Particularly with the increased pressures of COVID-19, EPSRC would like to stress to assessors that any cash or in kind support from the university for a grant is regarded as a benefit to building partnerships but is not expected to equate to cash or its equivalent (for example, provision of studentships, secondments, training and access to equipment).
EXC2: Excellence (including overall vision and ambition)
The applicants must articulate the overall research vision. To enhance the potential of achieving it, a world-class team of complementary expertise should have been brought together. The research programme should be ambitious, creative, innovative, and address key research challenges. It should also be sustainable beyond the lifetime of the grant.
A strong scientific case for support must be demonstrated, with the proposed research set into the context of the current state of knowledge and other work under way in the field. The importance of the research theme proposed to the strategy of the respective universities involved should also be articulated. The expected outputs should result in a significant step change, with major impact on the research area beyond the immediate team, and appreciably raise the UK’s international profile.
Please comment on the degree of research excellence of the application making reference to:
- the novelty, relationship to the context, and timelines
- the ambition, adventure, and transformative aspects identified
- the appropriateness of the proposed methodology.
(For multidisciplinary applications please state which aspects of the application you feel qualified to review.)
Secondary major criterion
Drawing upon what the applicant has said, reviewers should comment on:
- how the proposed research contributes to or helps maintain the health of other research disciplines, contributes to addressing key UK societal challenges, contributes to current or future UK economic success or enables future development of key emerging industries
- the extent to which the research proposed has the potential to meet national strategic needs by establishing or maintaining a unique world-leading research activity (including areas of niche capability)
- how the research fits with and complements other UK research already funded in the area or related areas, including the relationship to the EPSRC portfolio and our stated strategy set out in our portfolio.
The extent to which each bullet point is addressed will depend on the nature of the research proposed. Reviewers should comment on how the research relates to EPSRC’s research areas and strategies (many projects will be relevant to more than one EPSRC research area) and complements EPSRC’s current portfolio. Information on the portfolio is available through EPSRC’s Grants on the Web (GoW).
The reviewer form asks reviewers to comment on the national importance of the research. Include how the research:
- contributes to, or helps maintain, the health of other disciplines
- contributes to addressing key UK societal challenges or contributes to future UK economic success and development of emerging industry(s)
- complements other UK research already funded in the area, including any relationship to the EPSRC portfolio
- meets national needs by establishing or maintaining a unique world-leading activity
- complements other UK research already funded in the area, including any relationship to the EPSRC portfolio.
APP2: Applicant and partnerships
Programme grants are for world-leading teams. You should explicitly comment on the international standing of not only the principal investigator but also the rest of the team and the appropriateness of the academic partnership.
Please comment on the applicant’s ability to deliver the proposed project, making reference to the:
- appropriateness of the track record and international benchmarking of the applicant(s)
- balance of skills of the project team, including collaborators
- development and promotion of the careers of all its team members, including investigators, research assistants, technicians and aligned students
- ability of the principal investigator and team to lead and manage a large, complex investment with sufficient support, infrastructure and resources for the day-to-day running of the programme grant.
MAN3: Resources and management
The proposal must demonstrate a clear management plan that will ensure that resources, including manpower, are deployed in the most effective way to deliver high-quality research outputs that have the potential to induce a step change in the knowledge of the subject area. It is expected that the deliverables and milestones will be routinely reviewed to ensure that the most exciting and promising lines of research are pursued and that sufficient resources are assigned to ensure the project is professionally managed. For more advice, see guidance on managing large research activities.
Please comment on the effectiveness of the proposed planning and management, and on whether the requested resources are appropriate and have been fully justified, making reference to the:
- effectiveness of the proposed planning and management
- appropriateness of the requested resources
- suitability of proposed strategy for flexible allocation of resources and use of an independent advisory board.
ANI1: Animal research and human participation
Where the applicants have ticked any boxes confirming that the application involves either animal research or human participation, then you are asked to comment specifically on any ethical considerations and particularly on whether ethical approval procedures have been complied with. You should also comment on any potential adverse consequences for humans, animals or the environment and whether these risks have been addressed satisfactorily in the application. It is particularly important that resources relating to these aspects are explicitly justified in terms of need, scale and nature of resource. For example, for animal research you should comment specifically on the need to use animals, the choice of species and the number of animals it is intended to use.
ASS1: Overall assessment
You should provide your overall assessment of the application. Think of this as your report to the prioritisation panel, highlighting the strengths and weaknesses you identified in the individual questions and then making a clear and explicit recommendation about whether or not you believe the application warrants funding.
Not all questions carry equal weighting. Research quality (excellence) will always be pre-eminent, and no application can be funded without clearly demonstrating this aspect. National importance should also be a major consideration in making your assessment. The weighting between the remaining aspects will depend on the specific nature of the particular application. You should indicate those aspects that you accorded higher or lower priority and why.
The reviewer form asks reviewers to summarise their view of the application.
ASS2: Overall assessment – part assessment
It may be that you feel you can only comment with authority on a specific part or component of an application, for example with a multidisciplinary project, or perhaps where there is a strong user-led element. In that case you should identify those aspects that you are able to comment on, and then give your review on just those aspects. Different reviewers will have been asked to cover those aspects you cannot, and the panel will then have the job of integrating these different comments. It is particularly important therefore that the panel have clear advice on the merits of each component. Your comments, scores and confidence level should explicitly reflect your views on those aspects you can assess, and you are asked not to moderate these in any way to reflect those areas you feel you cannot comment on.
A risk with part assessment is that it will miss the added value of the overall project (the whole ideally being greater than the sum of the parts) so even where you can only comment with authority on one aspect, it will be helpful to the panel to have your views on how compelling the arguments for the overall application are. Other issues you might also comment on are the uniqueness (or otherwise) of the collaboration, the value of the contribution of the component you can judge, and the significance of this in terms of future potential development in your own field.
ASS3: Overall assessment – overall score
You should assign a score using the six-point scale provided. This should reflect your overall conclusion, and should be consistent with your comments on the individual sections of your review, taking account of all the assessment criteria and the various weightings you applied.
The six levels of the scoring scale are as follows.
- This application is scientifically or technically flawed
- This application does not meet one or more of the assessment criteria
- This application meets all assessment criteria but with clear weaknesses
- This is a good application that meets all assessment criteria but with minor weaknesses
- This is a strong application that broadly meets all assessment criteria
- This is a very strong application that fully meets all assessment criteria.
ASS4: Level of confidence
To assist the prioritisation panel in reaching their overall conclusion on the application, and to help EPSRC in monitoring the effectiveness of its reviewer selection procedures, you are asked to indicate your confidence in your review. This should report your own confidence, or otherwise, in being able to make your assessment rather than your confidence in the success of the application if it were funded. If, for any reason, you feel that you are not able to assess the application, please advise EPSRC accordingly.
The reviewer form asks reviewers to score their confidence as low, medium or high.
ADV1 (Advocacy for engineering and the physical sciences)
The proposal must demonstrate how the group will be advocates for the engineering and physical sciences. Applicants should specifically address how they will influence its policy makers on the importance of engineering and physical sciences. Advocacy through public engagement activities can also be considered, as long as these are directly related to the programme of research applied for.
Please comment on the degree of advocacy for the engineering and physical sciences proposed.
The proposal should demonstrate the added value and synergy of supporting the research through a programme grant. This should include how the length and flexibility of both staff and other resources will be used and the added benefits of this approach. It should also demonstrate how it is different from standard proposals in terms of the particular research activities that will be undertaken.
Please comment on the added value delivered through the proposal, making reference to the:
- added value and need for supporting this research as a coherent programme of interrelated research activities and not a number of smaller research grants
- need for the added flexibility of resources and the longer term nature of the grant to achieve the proposed research goals.
Last updated: 1 March 2022