Framework for monitoring and evaluation of the JPI as well as IPR and regulatory issues
The objective of this WP is to provide the governing bodies of the JPI (Management Board, Scientific Advisory
Board and Stakeholders Advisory Board) with elements to make a judgment on the progress of the JPI
as compared to the vision and objectives of the JPI in terms of societal challenges to be addressed. This
information will include an analysis of processes that may be causing the observed results, to guide corrective
measures if necessary. This WP will build procedures to be implemented for the evaluation, based on the
objectives of the JPI, including a strategy of short-term as well as long-term evaluation.
In addition, this WP will contribute to yield the return of the JPI by establishing mechanisms to provide all actors involved in the problem of AMR with scientific results as relevant for them, from open distribution for regulatory bodies or clinicians to IPR protection for industry interesting results. This WP will set up rules for handling of IPR and dissemination of research results according again to the objectives of the JPI.
Description of work and role of partners
Task 6.1 Define the framework for evaluation and monitoring of the JPI (M1 to M15)
(Task leader: ISCIII; Contributors: ZonMw, SRC)
The actors, criteria, indicators, information systems for evaluating and monitoring the JPI will be based on three dimensions:
- Research policy governance, related to the governing structure and procedures
- Scientific performance, related to the scientific productivity, including effectiveness to respond to societal challenges
- Stakeholder involvement and satisfaction (to ensure that their needs are addressed)
We will define a list of criteria for each dimension, based on existing literature and experiences, such as those
of ESF, ERA-LEARN, or other JPI as organized by the project JPIs To Co Work (where the Swedish Research
Council is participating and ISCIII is coordinating). The list will then be validated in a process of consultation to
the management board, scientific advisory board and stakeholder advisory board.
For each criterion a number of indicators will be developed (For instance competitive “scientific productivity”
can be a criterion that can be measured with different indicators). We will select those which are valid, reliable, feasible and affordable. These features will be analyzed by an expert group including WP participants and those involved to provide the necessary information. Pilot studies will be conducted if needed in special cases. Qualitative methods such as document reviews or interviews will be applied as appropriate.
Monitoring will be distinguished from evaluation. Monitoring will be designed to assess to what extent these
activities follow the plans, and whether to continue or discontinue or redefine a series of activities of the JPI.
Evaluation will be designed to make an overall assessment of the achievement of most general objectives and
goals of the JPI, and to be able to analyses the reasons for it. The Strategic Research Agenda will therefore be a major guide to the evaluation. Task 6.1 will also be coordinated with task 4.3 on monitoring and evaluating JPIs joint activities, which will feed into the more general task 6.1.
An information system will be designed to feed the monitoring and evaluation tasks. Monitoring and evaluation
contents and methods will be developed in parallel to the rest of activities of this CSA, which will provide the
basis to select criteria, indicators and methods. The information system will include sources of information,
persons in charge of feeding them, as well as periodicity. Task 6.1 will begin in month 1 and should end by month 16, establishing a Standard Operating Procedure (SOP).
Task 6.2 Pilot, validate and expand the monitoring and evaluation system (M16 to M36) (Task leader: ISCIII; Contributors: ZonMw, SRC, ANR, JÜLICH, NASR)
After month 16 we will begin to operate the monitoring and evaluation system as appropriate given the
development of the JPI by this time. That may mean more qualitative methods at the beginning and eventually
include more quantitative analysis. A part of the evaluation system may need to be developed after task 6.1 is
over, because some of the operations of the JPI may not be running before. Therefore, task 6.2 will include the development of new contents of the monitoring or evaluation system.
Task 6.3 IPR and regulatory issues (M16 to M33)
(Task leader: ISCIII; Contributors: ZonMw, SRC, ANR, NASR)
We will review existing approaches to the protection of intellectual property rights in other JPIs, ERA-nets and
article 185, and contact the European Patent Office and the International Patent Office for that purpose. We also will build on the services of IPR in Cordis.
We will define a policy combining protection of IPR with open access or dissemination of research results
depending on the type of result and the interests of different stakeholders. This policy will be validated by the
management board, the scientific advisory board and the stakeholders’ advisory board. The policy will be guided by the objective of promoting innovation in industry, policy making or the general population. Scientific advice on regulations will be included.