External evaluation

management of the

CompLeap project

• Application stage of the project funding was challenging for the key actors in charge of preparing the Grant Application for the European Commission. The actual time left for the preparation was relatively short. In addition, the funding decision was delayed which lead to  challenges in the beginning of the project. These two factors have had an impact on the project actors’ de facto possibilities to implement the project goals in an effective manner.
• The composition of the project partners is multifaceted expertise which have supported the achievement of the project goals. 
• Due to the several reasons, the project organization is partially perceived as a complex one by  some project actors. In addition, the actual ownership of the project has remained unclear in some cases.
•  The changes in the project key staff had challenged the daily management and effective implementation of the project in the starting phase. However, the situation has improved in the course of the project and the Project Management Committee has demonstrated their good ability to work together.

• Weekly meetings as a part of the project management.  
• Webinars for the project key actors: such as ones arranged for the users of the prototype within the project. 
• Majority of the project documentation as public wiki pages. 

2) Monitoring and

evaluation of the 

CompLeap project

• The project actors (Project Management committee and Steering Committee) recognize the key practices of the monitoring and evaluation in a similar way. This is an asset, that has advanced the achievement of the goals set for the project.
•  Mid-term review by European Commission is seen beneficial among the key project staff and Steering Committee members. Its recommendations have been seriously considered as well as implemented in the project management and other work package operations.
•  Mid-term review along with the implemented requests has evidently supported the achievement of the goals set for the project. 
• The project partners´ opinions on the sufficiency and regularity of the monitoring of the project vary to some extent.  However, the project actors have been capable to enhance the monitoring practices in the course of the project.

• Mid-term review as a European Commission practice has visibly been beneficial to the CompLeap project. It has helped to enhance the various management practices as well as brought some new elements to the actual project implementation.  
• Work package specific local evaluation meetings at Finnish National Agency for Educations have served the effective implementation of the project. 
• Systematic feedback collection from the webinar participants.

architecture and

services development

• The representatives of the project consortium partners share the understanding of the key goals set for project. This is an asset, that has supported the achievement of the project outcomes.
• The key goals of the project - the design of the framework architecture and the development of the service prototypes – were both completed in the course of the project. In terms of the maturity the framework architecture can be seen as a mature outcome and the prototypes as a raw outcome.
•  The actual impact of the CompLeap project outcomes among the original target groups, immigrants and NEETs, is still low. the original goal setting – to develop digital learner services prototypes – does not fully comply with needs of  the specified target groups, whose ability to use digital services independently is often limited, along with the limited language skills. However, there is evident potential regarding the needs of these customer groups too, if the project key outcomes are more clearly linked with the professional counselling and guidance expertise in Finland, Netherlands and across the EU countries.
• The project implementation implies a successful adaptation of the learner-centeredness.

services capacity to be

integrated to the digital

learner services

(nationally and

internationally, incl.

Europass).

• The completion of the international dimension of the CompLeap project is still relatively low, in terms of concrete outcomes linked to the framework architecture and a developed services prototype. Experiences on the national use and integration capacity of the architecture are limited to Finland. However, the Finnish case highlights well the opportunities built into the framework architecture and competence-based digital learner services.
•  The deployment workshops held in Netherlands, Germany, Estonia and Croatia are seeds, that carry potential to the further impacts and effects, if the key project outcomes are advanced adequately in these countries to make local solutions of them. 
• The project actors show a good capacity to collect and analyze data and information on the possibilities to integrate the CompLeap key outcomes to the existing services nationally and internationally.  This is evident in the Europass Case Study and the Sustainability Plan (Deliverable 23). 
•  The national data resources on education should be seen as a primary product, that make advanced digital learner services possible. Countries across EU should address the establishment of the national data resources as one of the future priorities in the field of education infrastructure development. They are a precondition to the possible shared framework architectures along with the competence-based digital learner services for the transnational audience
• National decision-making plays a key role in the digital learner services that advance the national data resources on education jointly. The EU should create incentives for the voluntary cross-country cooperation in this field.

stakeholder relations

• Stakeholders have been widely included in the design and implementation of the CompLeap project. In general, stakeholders are satisfied with their possibilities to join the project. The stakeholder issues have been approached in an ambitious manner in the CompLeap project.
• There are several concrete outcomes of the communication and stakeholder work in the project, such as stakeholder events, webinars, external web pages and marketing videos (animations).
• The management of the expectations among the key project actors as well as stakeholders is essential for the project such as CompLeap. This should be carefully addressed in the similar future projects to avoid the gaps between the goals, expectations as well as key outcomes of the projects.

• Demo sessions  
• stakeholder webinars  
• marketing videos  and printed marketing material  
• external CompLeap web pages