How to use the API

2 How to use the Api

    2.1 Login and Logout

        2.1.1 How to Login and obtain a session ID

        2.1.2 How to Log out

    2.2 Retrieving metadata

        2.2.1 Filtering the results

        2.2.2 Filtering by submissions

    2.3 Retrieving information about files

    2.4 Enumerations

        2.4.1 API details

        2.4.2 Metadata


2. How to use the API

In the following sections we detail how to use this API both for metadata retrieval and submission. The examples we provide here use the curl command to perform all the actions, but they can be easily adapted to be used on your preferred scripting language (e.g. Python) or in REST clients (e.g. Postman).


2.1 Login and Logout


2.1.1 How to obtain a session ID

An active session is required to work with the API. Each time you log in with your credentials a new session is started, which is identified by an X-Token. Below an example on how to obtain one using curl:

curl -X POST https://ega.crg.eu/submitterportal/v1/login -d username=ega-box-xxx --data-urlencode password="thepassword" -d loginType="submitter"

All responses of the API are in JSON format. If successful, the response should include a new Token to use during the session:

"session" : {
        "sessionToken" : "12c810d5-0974-4d11-937f-bd75a81de762",
        "expeditionTime" : 1429107047601,
        "expirationTime" : 1429110647601,
        "method" : "N/A",
        "ipAddress" : "192.168.110.14"
      }

NOTE: To save space, from this point onwards we will simplify the URL: We will write path instead of the common part of the URL ( https://ega.crg.eu/submitterportal/v1) that is repeated in every call. We will leave out the header: -H “X-Token: 12c810d5-0974-4d11-937f-bd75a81de762” (but remember that it always must be present!)


2.1.2 How to log out

Simply use the following endpoint:

curl -X DELETE -H “X-Token:  12c810d5-0974-4d11-937f-bd75a81de762” path/logout


2.2 Retrieving metadata

The calls to the metadata service are basically identical for all EGA objects as well as the filters and methods available. Importantly, EGA objects can fall under a submission project. This concept is not very important when retrieving data, as it only provides another way to filter the results, however, it has a key role when submitting new objects. We will deal with this later on.

The following objects can be queried by the catalog, which correspond to the name of the method:

  • analyses
  • dacs
  • datasets
  • experiments
  • policies
  • runs
  • samples
  • studies
  • submissions

Filtering by status: Status can be filtered by appending '?status=[name of the status]' at the end (see Enumerations for the complete list of values).

For submitted objects (registered in Webin or XML Programmatically), it is necessary to append ?status=SUBMITTED When no status is provided, an aggregation of DRAFT, VALIDATED, VALIDATED_WITH_ERRORS, PARTIALLY_SUBMITTED is returned (that is, all except SUBMITTED). Therefore, only metadata objects registered via the REST API and not submitted yet are retrieved.

Below some examples:

Display all studies on draft, validated and validated with errors:

curl -X GET path/studies

Display all studies that have been validated and had errors:

curl -X GET path/studies?status=VALIDATED_WITH_ERRORS

Show all samples that have been submitted:

curl -X GET path/samples?status=SUBMITTED

All calls return a list of JSON objects, one for each EGA object, that contains all attributes of the object.


2.2.1 Filtering the Results

The identifiers described in section Identifiers can be used to filter out the results for a given object. If no parameter is specified, the filtering is done by either of the two internal IDs indistinctly:

curl -X GET path/samples/{sampleId}

Notice that “{sampleId}” must be an identifier of an already existent sample (in this example). To filter by EGA accession Id or alias, ‘?idType=[name of the id]’ must be added at the end. Example:

curl -X GET path/samples/EGAN00001176841?idtype=ega_stable_id

curl -X GET path/samples/this_is_the_alias?idType=ALIAS 


2.2.2 Filtering by submission

You can list any objects filtering by submission Id.

All analyses that belong to this submission:

curl -X GET path/submissions/{submissionId}/analyses

All experiments that belong to this submission:

curl -X GET path/submissions/{submissionId}/experiments

As when retrieving metadata, you can add '?status=[name of the status]' at the end to also filter by status. All analyses from this submission that are submitted:

curl -X GET path/submissions/{submissionId}/analyses?status=SUBMITTED

Remember that if no status is provided, an aggregation of DRAFT, VALIDATED, VALIDATED_WITH_ERRORS, PARTIALLY_SUBMITTED is returned (that is, all except SUBMITTED).


2.3 Retrieving information about files

Apart from metadata objects the API also provides a way to obtain information about the uploaded files. This may come from different sources, which must be specified in the call: CRG_INBOX : Files uploaded to CRG's box which have not been archived yet. EBI_INBOX: files that the user has uploaded to their box at EBI that have not been archived yet. A process runs every X hours and copies all files uploaded to CRG boxes to EBI boxes. So, all files that are seen using CRG_INBOX are also seen using EBI_INBOX (be aware of the delay).

Example:

curl -X GET path/files?sourceType=CRG_INBOX 

Output formatting

Additional data like parameters for pagination (skip and limit) can be added to all data retrieval queries in order to format the output. They must be specified as query parameters (i. e. skip=1&limit=5).

IMPORTANT!By default, only 10 results are shown and no result is skipped (skip=0&limit=10). If you want to retrieve ALL data set both skip and limit parameters to 0 (i. e. skip=0&limit=0).


2.4 Enumerations

Enumerations contain list of terms that are used as available options at different points of the API. They are all available under the /enums path and are public, so you don't need to be authenticated to call them. Some of them refer to the metadata whereas other provide parameters for calls or methods:


2.4.1 APi details

path/enums/actions: Action values. EDIT/VALIDATE/SUBMIT.
/submission_statuses: Submission status values.
/id_types: List of the different ids accepted at the EGA.
/entity_statuses: Object status values.
 


2.4.2 Metadata

path/enums/analysis_file_types: Analysis file type accepted values. 
/analysis_types: EGA elegible analysis types.
/case_control: Case/control accepted values.
/dataset_types: Dataset type accepted values.
/experiment_types: Experiment types.
/file_types: Run file type accepted values.
/genders: Gender accepted values.
/instrument_models: Platform accepted values
/library_selections: Library Selection accepted values.
/library_sources: Library source accepted values.
/library_strategies: Library Strategy accepted values.
/reference_chromosomes: Chromosome values.
/reference_genomes: Genome values.
/study_types: Study type accepted values.

For instance, this call lists all the experiment types available when registering a new experiment object:

curl -X GET path/enums/experiment_types

"response" : {
    "numTotalResults" : 5,
    "resultType" : "eu.crg.ega.microservice.dto.submitter.AttributeData",
    "result" : [ {
      "tag" : "4",
      "value" : "Curation",
      "label" : null,
      "unit" : null
    }, {
      "tag" : "1",
      "value" : "Exome sequencing",
      "label" : null,
      "unit" : null
    }, {
      "tag" : "2",
      "value" : "Genotyping by array",
      "label" : null,
      "unit" : null
    }, {
      "tag" : "0",
      "value" : "Whole genome sequencing",
      "label" : null,
      "unit" : null
    }, {
      "tag" : "3",
      "value" : "transcriptomics",
      "label" : null,
      "unit" : null
    } ]
  }