Integration from the organization's own metadata repository into Metax is always done within a project together with CSC's Fairdata team (fairdata (at) csc.fi).
- The project involves a technical contact person from CSC who collaborates with the organization's own technical contacts, as well as the product owner of Metax to ensure progress.
During the project, the integration is implemented in Metax's demo environment, and at an agreed-upon time, the tested implementation is put into production.
- The metadata is imported into Metax's metadata catalog, which CSC creates for the organization.
- A more detailed description of Metax's datamodel and mandatory fields can be found here: https://wiki.eduuni.fi/x/7w7PCg
- CSC creates a service account (username and password) for the organization to import the metadata into the aforementioned metadata catalog.
- The organization creates the necessary API calls (messages) to import the metadata.
Project Progress, Example
At the kickoff meeting, detailed project stages and schedules are agreed upon with the organization.
Below is an example of how the project typically progresses.
Example Project Tasks
| Tehtävä | Kuvaus | Ympäristö | Vastuu |
|---|---|---|---|
| Creation of the organization's own metadata catalog, demo | CSC creates a metadata catalog for the organization where the metadata will be imported. The name and other basic information of the catalog are agreed upon. | Demo | CSC |
| Creation of the organization's Service Account, demo | CSC creates a username and password for the organization, granting rights to import and update metadata in the organization's metadata catalog. | Demo | CSC |
| Payload formats, API Interface Documentation, Integration Constraints | CSC provides payload formats and API interface documentation to the organiztion and collaboratively agrees on integration constraints (e.g., PID practices). | - | CSC |
| Familiarization with payload formats and API Interfaces, Identifying Mapping Needs | The organization familiarizes itself with Metax's API interfaces and payload formats, collecting open questions to be addressed collectively. | - | Organization |
| Detailed Review of message contents and API Interfaces, Identification of Mapping Needs | Organization's questions regarding the usage of API interfaces message formation and payloads are addressed. Mapping needs (conversion of codes and identifiers used by the organization to those recognized by Metax, as well as any required changes between metadata models) are identified at this stage. | - | Organization (CSC strongly supports) |
| Mappings | Mapping tables are created for each conversion requiring code values. (Organization's value <-> Metax recognized value)
Mapping between metadata models can be done in collaboration between the organization and CSC or independently by the organization. | - | CSC / Organization |
| Forming the API calls (messages and payloads) | The organization implements the needed API calls. | - | Organization |
| Testing | The organization tests message formation and API usage in Metax's demo environment. CSC assists in reviewing imported metadata and payloads if needed. Note! Besides importing metadata, the update and deletion of information are also tested. | Demo | Organization |
| Responding to testing observations | The organization communicates with CSC if challenges or changes requiring CSC support are observed during testing, such as in payloads or API calls. | Demo | Organization (CSC strongly supports) |
| Agreement on Production Deployment | At the final stages of testing, the timing for production deployment is agreed upon. | Production | Organization |
| Creation of the organization's own metadata catalog, production | CSC creates a metadata catalog for the organization where the metadata will be imported. | Production | CSC |
| Creation of the organization's Service Account, production | CSC creates a username and password for the organization, granting rights to import and update metadata in the organization's metadata catalog. | Production | CSC |
| Import of metadata into Production, Phase 1 | The organization imports the initial metadata into the production environment (1 or a few datasets at most). | Production | Organization |
| Review of metadata in Production | Ensure that the metadata imported in Phase 1 has been correctly transferred to production and is of good quality. Note! At this stage, also check how the metadata appears in the Research.fi portal. | Production | Organization (CSC strongly supports) |
| Import of metadata into Production, Phase 2 | The organization fully initiates the integration (import, update, delete of metadata) in production. | Production | Organization |
Instructions for Message Formation:
- Usernames and passwords are transmitted in the payload as Base64 encoded authentication strings.
- Key = ‘Authorization’
- Value = ‘Basic ‘+ base64 enccoded login:password
- Many of Metax's fields work as "reference data," meaning they require an identifier referencing various code lists in the background.
- These fields include, for example, license, access rights, keywords, disciplines, language, organizational information of the actors, infrastructure, and funding organization.
When importing such information, it is important to use identifiers recognized by Metax.
- If necessary, CSC assists the organization in converting used identifiers to Metax's identifiers (mapping of "value sets").
- The page https://metax.demo.fairdata.fi/swagger/v2/#/dataset_api lists API operations that can be used. From the same page, you can also test operations:
- Click on the desired operation.
- Click the "Try it out" button.
- Fill in the desired parameters and set the value of the "dryrun" parameter to true (this way, operations are not actually executed in Metax).
- Click "Execute"
For some operations, a username is required, which cannot be directly provided on the page. However, even without a username, you can see the correct syntax for the call.
- Other useful operations:
- List all datasets in a specific catalog:
- Example of a dataset (with mandatory fields filled): https://metax.demo.fairdata.fi/rest/v2/datasets?data_catalog=<catalog identifier>&limit=1000
- Example content of a call to import metadata into Metax: Example_post.py
- TA more detailed description of Metax's database fields (with examples) and mandatory information can be found here: https://wiki.eduuni.fi/x/7w7PCg
Other Links
Reference data: https://metax.fairdata.fi/docs/v2/reference_data.html
General Metax documentation: https://metax.fairdata.fi/docs/