LEARNING THROUGH FORMATIVE EVALUATION
The ERA-Net SES Coordination (here represented by the ERA-Net SES Support Team) wants to maximise the success of the awarded projects and build a family spirit. Projects receive communication support and feedback in the framework of a formative self-evaluative process.
Formative evaluation denotes a progressive self-evaluation that facilitates a process or increases the quality of its outcomes. The method serves to detect unanticipated issues and flaws in the projects through reviewing their processes and outputs and feeding results back to the project consortia.
In practice, for ERA-Net SES, the formative evaluation process can be seen as a profiling and follow-up process. The supporting team of the Knowledge Community Management can help shape the public picture of the project and – if wanted – help to overcome difficulties or dead ends in research questions. It can get the projects in contact with other projects dealing with similar issues and research endeavours.
Profiling with Key Performance Indicators (KPIs)
To successfully link the projects together and thus create synergies between them, the Knowledge Community Management together with the Call Management has created a profiling method that helps to identify synergies of projects as well as gaps in the research agenda that could be addressed in later calls.
This profiling process is based on a set of project-focused KPIs. However, the KPIs will not be used to "rank”, "rate” or "judge about” the projects in any way. Instead, they will serve to monitor the projects regarding their focus topics in terms of the three layer research model. It will thereby become visible which projects are working on similar topics and can benefit from a collaborative approach. On the other hand, it will serve to highlight the unique features and differentiators of individual projects that can complement each other and be highlighted in dissemination activities or asked to participate in working groups on specific topics. These project-profiling KPIs are based on the work of the Grid Plus project.
On a second level, the project KPIs serve to profile the entire ERA-Net SES programme against the goals that were set for the initiative, e.g. number of spin-off projects replicating project solutions from ERA-Net SES.
On a third level, contributions to national and/or European policy goals as set by different roadmaps can be traced.
The working process underlying the profiling actions is mainly based on annual surveys and virtual meetings to discuss the results of the surveys’ evaluation. In order to deepen the knowledge about individual projects and their outcomes beyond the scope of the profiling, the support team meets – virtually and physically – with individual project teams on an annual basis to give expert feedback and learn about current developments, (intermediary) results, issues and open questions.
Creating Knowledge from Surveys
Surveys in the framework of the formative evaluation can help to detect common or diverging opinions on certain topics among the ERA-Net SES funded projects and serve to prepare discussion within the working groups. In general, answers to the surveys also help formulating recommendations to program owners and decision makers. Here the evaluation exercises meet the knowledge generation process of the Knowledge Community.
