Key Performance Indicators

Key Performance Indicators (KPIs) represent standard measuring values that help institutions assess their performance in a consistent and periodic way. KPIs are widely used by private and public organisations to monitor progress towards set goals and the efficient use of resources.

As government-funded organisations, the European Research Infrastructure Consortia (ERICs) have a moral, and in many cases also legal, obligation to report on their performance to relevant stakeholders, funding bodies, and the public.

In principle, KPIs are distinct from socio-economic impact indicators (SEIIs). The former focuses on the efficiency of processes and the recognition of accomplishments, while the latter tracks outcomes of these activities and their long-term effects. Since performance and impact can often be linked, it is not unusual to see similar or the same indicators used to monitor both performance and impact

While the need for performance evaluation is widely recognised, there is no firm agreement on the measures that should be used. The term KPI is often used to mean a management tool primarily aimed at internal monitoring of performance on a relatively short timescale. In the case of RIs it is more useful to think of performance metrics and qualitative case studies aimed more at external reporting to key stakeholders, and monitored over longer timescales, particularly where factors such as socio-economic impact are concerned.

1. KPIs should be tailored to the specific objectives and mission of each ERIC

• Every ERIC and RI is unique in terms of its raison d’etre, objectives, mission and vision. Even ERICs active in the same scientific domain differ.

2. KPIs should be only used to benchmark an RI against its own performance and not to compare RIs

• KPIs recommended by ESFRI should provide a common ground for a monitoring system that should later be tailored to the specific needs of each ERIC.
• Many metrics only make sense when tracked over a several-year period. This should be taken into consideration when evaluating data.
• Success does require an understanding of both a baseline and a trend, but growth is not necessarily always the ideal scenario as it differs from progress

3.KPIs should comply with well-proven criteria for setting up indicators and measures

• KPIs should be defined according to one of the internationally recognised standards for developing indicators and measures, such as e.g.
• • RACER (Relevant, Acceptable, Credible, Easy and Robust) criteria developed as part of the European Commission’s Impact Assessment Guidelines
  • SMART (Specific, Measurable, Achievable, Relevant, Time-bound) criteria
  • CREAM (Clear, Relevant, Economic, Adequate, and Monitorable)

Because of their strong connection to the European Commission and their relevance to the European Research Area, ERICs are in favour of RACER criteria.

4. KPIs and indicators to measure socio-economic impact are not the same, even though a limited number of selected KPIs could be used to measure impact

• The purpose of KPIs is to measure performance, evaluate success in delivering results, and monitor progress towards set goals.
• The purpose of indicators assessing socio-economic impact is to evaluate how RIs transform their environment and what influence they have beyond scientific results. Some RIs enable science by producing data which are used for scientific research or by providing access to state-of-the-art instrumentation, so the definition of impact can vary across RIs.
• KPIs are considered an internal management tool and thus it is more useful to think of performance metrics and qualitative case studies aimed more at external reporting to key stakeholders and monitored over longer timescales.

The full position paper can be accessed here. 

ERIC Forum has also conducted a survey which aimed to assess the current status of KPIs implementation across the community and collect feedback on the KPIs proposed by ESFRI.

When asked why ERICs had adopted KPIs, ERICs reported being motivated by both internal and external factors. The key reasons can be summarised as follows:

• To have an internal management tool that helps the ERICs to achieve tangible results, and enables the monitoring of progress against set objectives, mission, and vision;
• To document improvements and developments, and monitor performance in various areas of interest, including e.g. scientific outcomes and outputs, technical performance, operations, adequacy of services, access, trainings, user uptake etc.;
• To have an instrument for communicating the successes of the ERICs to various audiences and stakeholder groups by presenting measurable results;
• To improve the long-term impact and value of the ERICs, demonstrate their effectiveness, efficiency and accountability, and address the requests of governing bodies;
• To mitigate risks by detecting problems early on, and provide informed feedback on key strategic documents and policies of the ERICs;
• In case of distributed ERICs, to better evaluate the work of headquarters and measure the level of engagement with each national node.

The full survey can be accessed here (LINK)