|
|
 |
| Methodology of database development and good practice selection – Second Edition, September 2006 |
| This concise working paper is a description of the methodology applied for developing the ERIK Network Good Practice Database, which contains descriptions of successful projects developed in the framework of the Technological Innovation theme of the Regional Programmes of Innovative Actions (RPIA). The paper includes the description of the ERIK self-evaluation approach as well as the presentation of the main phases of data selection and database development. The paper was initially produced in the first ERIK phase (2002-2004) but has now been updated for the second phase 2006-2007. References therefore refer to current activities. |
| Contents
|
|
1. General approach: regional self-evaluation
One of the main goals of the ERIK Network is to select and disseminate information on good practices developed by regions in the course of implementation of Regional Programmes of Innovative Actions (RPIAs). The scope of the Network has been enlarged in the second phase in order to allow the analysis of good practices from regional innovation strategies in general. To this end the Network has built an on-line database containing good practices in the field of Technological Innovation, that is, practices whose focus is on the first RPIA theme - “Regional Economies Based on Knowledge and Technological Innovation ”.
The process of good practice selection requires an evaluation which, on one hand, has to encompass such elements as the assessment of the proposal preparation, the implementation process, and the final deliverables, and, on the other hand, the analysis of economic, political, and social contextual factors. Thereby, the ERIK database is designed to include the analyses of projects, actions and programmes.
The limited availability of budget and time made it impossible to undertake a fully-fledged theme related evaluation in all RPIA regions. The solution was to involve ERIK member regions both in the data selection and the evaluation of programmes/actions/projects. The regions self-evaluate their practices in a regional survey. The survey questionnaire is based on the methodological framework developed by ERIK. The Network’s attempt is to build the good practice database in a most open and participatory way, by taking into account the opinions of practitioners from the ERIK member regions, the European Commission as well as distinguished academics.
The implementation of the final version of the database was preceded by a pilot phase that allowed the repeated testing and refinement of what was destined to be the final on-line questionnaire.
|
2. Types of indicators, thematic scope and data sources
The questionnaire is constituted of several groups of questions (identified here as indicators) that can be divided into the following groups:
- - Contextual indicators
o general socio-economic profile
o governance and policy making style
o RTDI indicators
|
} |
context of RPIA (program, action and projects) |
- Specific indicators on selected thematic areas
- General RPIA indicators (program level)
- Good practice indicators (program, action or project level)
The following section presents the outline of each group of indicators covered by the database.
2.1. Contextual indicators
The socio-economic and RTDI indicators, as well as questions regarding the governance and policy making style, serve to develop a data pool on the regional context of the assessed program/action/project. The assumption is that it is impossible to determine whether an activity is a good practice without relevant knowledge about its context. The questionnaire also includes questions on the governance (e.g. RPIA partnership development process) and the regional policy making. The general context indicators have been retrieved from generally accessible sources such as, e.g., EUROSTAT.
The contextual indicators cover such features as, e.g.:
- - general socio-economic profile
- o basic structural indicators
o employment figures
- - governance and policy making style
- o partnership building process
o RPIA in the context of the regional innovation policy
- - RTDI indicators
- o R&D spending
o R&D personnel
o education
o patents
2.2. General RPIA related indicators
RPIA related information is largely based on the original texts of the programs as well as specific questions on the program development. The questions cover:
- partnership (authorship)
- thematic focus of the actions
- objectives and beneficiaries
- selection criteria
- program management and implementation process
- funding (e.g. budget of actions, public vs. private funding)
- mainstreaming RPIA
- results and impact assessment
2.3. Good practice indicators
The good practice section is based on a survey developed within the first ERIK phase in collaboration with and with the approval of the European Commission (DG Regional Policy). The survey attempts to cover a wide range of factors which could be used to analyse a “good practice” in the context of the knowledge society and of the identified Thematic Working Groups (TWGs). The following groups of information are requested:
- description (thematic focus, sector)
- main innovative features and success factors
- transferability
- partnership
- management
- durability
- impact assessment
The good practices inserted by the regions may concern an overall programme, an action within the programme or a project approved by the programme
2.4 Specific indicators on selected thematic areas - input from ERIK's TWGs
ERIK Thematic Working Groups (TWGs) have been actively involved in developing specific indicators related to their field of work, that is:
- Clusters and business networks led by the West Midlands (UK)
- Services and support to start-up and spin-off - led by Niederösterreich (A)
- Science and industry relationship led by Emilia-Romagna (I)
- Innovation Finance led by Småland med öarna (S) (Replacing the group “Regional innovation benchmarking and foresight” from the first ERIK phase)
The TWGs' leaders developed the background papers as well as the set of indicators relevant for their field of work (see TWG section). The thematic indicators were analysed and selected by ERIK partners and associate regions (though the final decision rested with the TWG coordinator). Once chose the indicators were inserted into the database and used to gain further information on good practices and to help with the evaluation process. (In depth descriptions of the indicators for the TWG Start-Ups)
|
3. Main phases of database development and good practice selection
This section outlines the main phases of database development including (1) introductory analysis, (2) questionnaire preparation and pilot phase, (3) project self-evaluation process, (4) database improvements according to feedback.
3.1. Introductory analysis
The first phase of good practice database development was devoted to the thorough analysis of RPIAs and other relevant documents with information on activities developed in the framework of ERDF’s Innovative Actions (e.g. material from regional innovation awards) as well as studies, reports and databases on regional aspects of innovation policy (e.g. EUROSTAT databases, RINNO, Innovation Scoreboard, etc.).
3.2. Research method selection and questionnaire development
The introductory analysis formed the basis for the choice of the methodological approach. It was decided that a regional survey had to be produced in order to select data on the good practices. The data collected through the survey were then integrated by generally accessible data from EUROSTAT and other relevant sources.
The questionnaire offers regions the opportunity to self-evaluate their PRIAs and good practices. The draft methodology and questionnaire were both open for discussion, comments and contributions from the ERIK Network members as well as selected academics. In the pilot phase the questionnaire was tested on Toscana and Emilia-Romagna regions.
3.3. Regional survey - project self-evaluation process
The second phase of the database development is gathering data from the regions. The on-line questionnaire is filled in by the regions themselves. The ERIK managing staff provides assistance to regions filling in the questionnaire.
The feeding of the database consists of two stages of self-evaluation. The first self-evaluation exercise focuses on the preparation and implementation process of the programmes and projects. After the RPIA is concluded, the second stage of self-assessment is to assess the actual deliverables and impact on the local/regional economy. Regions are also asked for a qualitative assessment of their RPIA. Wherever possible results of the ex-post evaluation are requested from the region. Indeed this is one of the criteria used for selecting good practices as case studies.
3.4. Database Improvements
This phase is on-going and will continue as long a constructive feedback is provided from database users. As more data is inserted issues come up concerning, for example, search options, clarity of questions, visualisation and export options. Constructive input is welcomed from all those who use the database either to insert data or to access it. |
4. Explanatory note
The ERIK Database questionnaire is divided into four sections:
- Contact Data
- Regional Data
- RPIAs
- Good Practices
Throughout the questionnaire a five (5) rank ordinal scale is used. The most relevant answer should be ranked as 5 and the least important as 1. This rule applies to all ranked questions.
Please report most recent data if possible. Wherever the “year” field is present, this refers to the year of the answer to the directly previous question. In case the “year” field is absent, the information should refer to the current (at the time of answering) situation.
Most relevant fields you should possibly fill in show with an * sign. Nevertheless, in order to give you the opportunity to complete the questionnaire at different time-sessions, partial saves can be made – i.e. regions will not loose data if they stop before completing the entire questionnaire. It is vital to click “submit” at the end of each section or the data will not be saved.
|
5. Beyond the database: interregional learning and policy benchmarking
The ERIK database is a tool that is to serve the overall aim of inter-regional learning process and regional innovation policy benchmarking. ERIK’s database information will allow a comparative analysis of different measures and approaches to policy making with an aim of improving the overall performance and narrow the gap to the best performing regions.
One way the inter-regional learning process may take place is through the exchange of good practices between regions. The good practice database supports practical measures of inter-regional good practice exchange. It is available on-line so that interested parties may have an instant access to relevant information. Interested parties need only register with the database and they are instantly provided with account details (user email and password). The web-based database will allow a first contact between regions whose projects in the database prove to attract most interest and other regions. Next steps could be defined by the regions and could lead to e.g. further exchange of materials, study visits or even short term placement schemes. Depending on the future role of ERIK, the Network could play a more or less active role in providing support to the good practice exchange process. |
|
|
