Responsible partners: Finnish National Agency for Education (EDUFI) and Jyväskylä Educational Consortium Gradia with the Associated partners in Finland and other EU-countries
Contact persons: Annica Moore/EDUFI, Topias Kähärä/EDUFI and Tarja Puura/Gradia
Countries involved: Finland, the Netherlands, Germany and some other EU-countries via networks
Piloting the deployment of the learner plan prototype will overlap with the piloting of the architecture and the analytics prototype, as all the three elements are integral parts of the user experience.
The personalized learner plan prototype will include two independent but interconnected modules, including:
a personal competence profile
personalised learning opportunities based on usage data and personal profile
Defining the user groups
The user groups of the learner plan prototype will follow the target user groups outlined in the Grant Agreement and the project altogether. This includes e.g. immigrants, individuals not in education, employment or training (NEETs) and individuals in the midst of a career shift (possibly due to abrupt or long-term unemployment or newly graduated secondary level students, heading for further education).
In addition to the aforementioned user groups, guidance counselors and admission services are key user groups for the piloting. The extent of the piloting covers their training. A total of four piloting instances/workshops are held, one in each associated partner organization. In its entirety, the piloting will involve ca. 100 people.
The test population consists of two groups:
Ca. 20 - 100 learners, who could test the usage of the service in a guided environment. The test population will be selected from the associated partner institutions, based on the availability of the user groups defined above.
The learner plan prototype is designed to be able to use independently. Therefore, in addition to the 20 - 100 learners from our associated partner institutions, an undefined number of users, who test the service anonymously and independently.
Methods
The piloting of the learner plan prototype is twofold. First, more structured piloting will include using the service by individuals from the target user groups (listed above). This includes monitoring the users while they are using the service (user shadowing) and taking notes of any visible emotional or cognitive experiences the usage of the service entices. After user shadowing, users may be interviewed and given surveys to fill out on the usage of the service.
When conducting user shadowing, the preliminary key points of evaluation are ease of use, clarity and perceived impact. These evaluation points are, however fluid and should change when more feedback is gained and more detailed points of evaluation are found.
Other methods of piloting the learner plan prototype include associate partner workshops, surveys and user manuals for the associate partners. After receiving a version of the service that can be used for piloting, the workshops focus on collecting testing and piloting observations and feedback. User surveys can be used in this collection process. In order for the associate partners to manage piloting independently, personalized (based on the elements of testing and the concerned user group) manuals for the use of service as well as the testing of the service will be provided.
The other, relatively open piloting concept includes anonymous individual use of the service, accompanied by a user survey. This approach is necessary due to the projected individual use of the service when the service is in production.
The learner plan prototype is implemented in Finnish and English. As end user piloting is mostly conducted in Finland, the use of Finnish is required. That is because testing situations with a service in a foreign language for the end user can provide biased results. If the prototype were to be tested only in English, the project wouldn’t receive the most authentic user experiences.
Main questions when piloting the learner plan
What kind of obstacles does the user experience when developing their learner path in the service?
Is the concept of the learner path clear to the user?
Does the usage of the competence profile and the learner plan feel easy/natural/useful/meaningful?
Does the learner perceive the use of the service to have an impact on their life/future?
Prerequisites and limitations for deployment
The first prerequisite for piloting the deployment of the learner plan prototype is a functioning mock-up/HTML-prototype of the services/functionalities to receive feedback for the user interface. In later stages of the project, (from May 2019) a preliminary beta-version (or PoC) of the service with some integrations to existing databases and systems using fabricated model users is needed. The service is developed iteratively, and therefore more functionalities will be added to the prototypes after each cycle of piloting as feedback on the functionalities is gained.
Another prerequisite for the piloting is reaching the necessary user groups. Even though the piloting will not use real user data due to national legislation, the users involved in piloting and the fabricated users should be as close to each other as possible, while being part of the designed central user groups. The timeline for piloting the deployment takes place from May until autumn. During this time, the learners matching most user groups are either starting their holidays or a new semester. Due to this, we are probably facing the fact that there will be limited time for counselling and limited amount of end users for the piloting. We must also remember that the consortium needs time for reporting before the project time ends – not only piloting.
Measuring success
The most important points of measuring success during the piloting of the learner plan prototype are commitment, collection of feedback during piloting, user satisfaction and dissemination. The measurements are elaborated below.
Criterion | Objective | Measurement |
Commitment | Associates are motivated to be involved in deployment. | Confirmation of the deployment plan, participation in workshop activities and receiving feedback in piloting. Contracts with the WP-leader for partner costs and travel costs. |
Collection of feedback during piloting | Feedback on the deployed services has been collected from the field as well as end users.
| Feedback channels are functioning and easy to obtain and use. |
User satisfaction | Users see the benefits and the usability of the services and are willing to use them in the future. A better than average grade in feedback.
| The amount of feedback and the grades received from the services. Grade system 1 - 5. |
Dissemination | Associates are planning the use of developed services among other services. They are interested in further development. | Expressions of interest, participation in the dissemination seminar. Activity in project´s social media channels. |