Search Results - EGA European Genome-Phenome Archive

How to request data

Request Data Access If you're looking to access data from the EGA for your research, you are in the right documentation page! In this page you will find a step by step guide on how to request data archived at the EGA, as well as some frequently asked questions you might find useful. When requesting data, you'll need to provide information about your research project and intended use of the data. Keep in mind that your data access request may require additional approvals or agreements depending on the datasets and data providers involved. Step by step guide Register yourself as an EGA user. Validate your account. A validation link will be sent to your email to activate your EGA user account. Log into the EGA page. Search the dataset of interest from the dataset catalogue or search bar. Click on “Request access” Add a comment requesting these files and send your request Your request is successfully registered, and the DAC will receive a notification about your request In addition to submitting your request, you should also contact the DAC directly via e-mail. You can access the DAC's contact information from the relevant dataset page. Frequently Asked Questions (FAQ) Which is the role of the EGA in the data access process? The European Genome-phenome Archive (EGA) facilitates the secure distribution under controlled access of personally identifiable genetic and phenotypic data. Data Access Committee (DAC) is a body of one or more individuals who are responsible for data release to external requestors, based on participant consent and/or National Research Ethics terms. How do I request access to multiple datasets? To request access for multiple datasets managed by the same Data Access Committee (DAC), please start by visiting the DAC catalogue to locate the DAC of your choice. Once you've found the relevant DAC, proceed to its dedicated page where you'll find a "Request access" button. After logging in to the EGA website, you'll be presented with a comprehensive list of datasets under the chosen DAC. Select the datasets you're interested in, and then compose a message indicating your access request. Click the "Send" button to submit your request, which will include the chosen datasets. Please be sure to review the data usage policies for the requested datasets, as policies can vary among datasets managed by the same DAC. I am a co-applicant. Will I be given a joint access account? Each applicant needs to create an EGA account themselves, with unique login details. User log-in details are for individual use only and should never be shared, under any circumstances. EGA services are only provided to EGA account holders. Therefore, the co-applicant will send a formal request to the DAC. What happens if a co-applicant leaves or a new member needs to be added to an existing DAC application? The appropriate Data Access Committee (DAC) should be contacted and informed about the required changes so that the application can be updated. I have changed my place of work since my EGA account was created. Can I still have access to data? You must contact the relevant Data Access Committee (DAC). The DAC did not reply to my data access request. How should I proceed? There is no estimated time for DAC response. Please allow up to four weeks for a response. In the meantime, please send them a weekly reminder of the status of your request by emailing them directly.If the DAC becomes unresponsive, meaning that they fail to communicate with a requestor within the specified time frame, please contact the EGA Helpdesk team. I have an account with the EGA and would like access to further datasets. How should I proceed? You need to reapply to the relevant Data Access Committee (DAC). Once the DAC has approved your application your account will be updated and an email will be sent to notify you of a change made to your account. How do I download files of the datasets I have been granted access to? The preferred method for data download and decryption is the EGA Secure Download Client. Please carefully review the provided documentation. How do I download metadata? Metadata is retrieved from the EGA webpage; you will not need to use any download client. Be informed that you will only be able to download metadata of your approved datasets. For such download, you should first log in the EGA webpage with your credentials. Then browse the metadata's dataset you have gained access and you will find a metadata download button. You may also want to have a look our metadata schema. I have forgotten or misplaced my user account password; how do I get a new one? Click on “Log in”, located in the top right corner of the EGA homepage. Then click “Forgot password”, type in your registered institutional email address, and instructions for resetting your password will be sent to you. Please check both regular and junk/spam folders. I was sent a link to set a password for my download account, but it seems like the link has expired. What should I do? The procedure is the same as the one for a forgotten password, explained in the previous question.

Documentation access/request-data/how-to-request-data

Submission FAQ

Submission FAQ Before Submission Is the EGA the right archive for my data? TThe most suitable archive for your data is dependent on the type of data you are wishing to submit and whether the data require public or controlled access. Public access is defined as complete and open access to all submitted data. On the contrary, controlled access, exerted by the EGA, requires formal applications to be made to access the submitted data files and metadata. EGA only accepts human-derived data subject to controlled access. If your submission contains other types of data, please choose the appropriate repository for it (see image below): ENA, EVA, ArrayExpress, BioSD and GWAS catalog. Should your submission be subjected to controlled access? Data access conditions are normally defined in the original informed consent agreements signed by the participants involved in your study. All data submitted to the EGA is subject to controlled access. These consents prevent the derived data files, potentially identifiable, from being dispersed by open and public access. Controlled access data often refers to human data derived from medical research and consortium projects. If in doubt, consult the informed consent agreements that apply to your study The EGA enables you to hold a submission before publication. What data types can be submitted to the EGA? Data types accepted by the EGA can be split into three categories: Sequences: both in generic and platform-specific formats. Array-based: from raw signal files to processed matrices. Phenotypes: all possible phenotype formats are accepted. All manufacturer-specific raw data formats derived from major next generation sequencing platforms are accepted. Also generic sequence formats: flat reads in a FASTQ file, aligned sequences (BAM or CRAM files) as well as sequence variation files in VCF format. All array-based technologies are accepted, including raw data, intensity and analysis files, without any restriction on data formats accepted. We also accept and distribute phenotype data (associated to the samples) in almost any format: from an image to a README file. How long does a submission take? Submissions to EGA come in a variety of formats and sizes, thus it is difficult for us to exactly predict how long a submission will take. We, therefore, advise all of our submitters to allow as much time as possible to make a submission. Based on previous records, we anticipate that the submission process may take at least one month. The submitter’s familiarity with the procedures, possible technical issues that may arise during submission and the amount of data that is being submitted are the main factors that will affect the length of the submission process. However, each step of a regular submission should be considered when estimating the time it would take: Encryption of the files Upload of the files Metadata submission Archival of the files Release of the study and datasets to EGA webpage For example, the upload of the files depends on the submission size, while metadata submission mainly relies on each submitter’s expertise. Further, some steps (e.g. answering to inquiries) depend on the EGA Helpdesk team, which may take some days to be processed during busy times Is data deposited in the EGA secure? The EGA set-up consists of a secure computing facility for data processing, a shared EBI set-up for data submissions and distribution of data via data requests made through the EGA website. Data is also copied in the Barcelona Supercomputing Center (BSC) infrastructure, where all stored and distributed data is encrypted Data is encrypted along the submission process and stored securely, granting its access to authorised users exclusively. During the download process, through our Python Client or Aspera, all requested data is downloaded over secure https connections. All data at the EGA is encrypted, and only accessible (for log-in and download) through secure protocols. For further information please, visit our security overview. What documentation do I need to provide? All submissions require policy’s documentation: 'Data Access Agreement (DAA)', 'Data Processing Agreement (DPA)' and 'Authorized Submitters Formulary'. The data processors (EGA) and the data owners will also sign the DPA. Will all metadata be public? Among the submitted metadata we need to make the distinction between identifiable and unidentifiable metadata: (1) the former may allow the identification of the human the sample derived from (e.g. detailed geographical providence, personal name, family ancestry…); (2) while the latter can be used to interpret the data without compromising the anonymity of the patients. The majority of the metadata submitted to the EGA corresponds to the unidentifiable category (e.g. sequencer's model). This type of metadata is publicly available on the EGA website and other EBI resources/partners’ websites. On the other hand, some parts samples’ metadata are subject to being identifiable, and thus only accessible by authorized data requesters, with the exception of: 5 submitter-defined attributes of the sample: alias, title, subject_id, gender and phenotype. It is the submitter’s responsibility not to submit sensitive metadata in these public fields. 3 anonymised fields that pinpoint the sample record in archivals: sample’s EGA stable ID (EGAN…), BioSample ID (SAMEA…) and submitter’s center name. During Submission Are there any sample specific requirements for EGA? All samples submitted to the EGA must include the attributes of biological sex, subject ID (anonymised individual identifier) and phenotype information. These are critical for data findability and its analysis, and we highly recommend using controlled ontology terms where applicable. For example: defining tumour and non-tumour samples and/or defining disease state. The EGA recommends using the Experimental Factor Ontology Database to find ontologized terms that describe your sample phenotypes. How do I get an accession number to use in my publication? You will receive your study accession number (EGAS…) upon complete your submission, either: Programmatically. As soon as the metadata is submitted and validated your study will be assigned an accession number that will be given in the submission’s response. Manually registering your study and relevant metadata using the online metadata submission tool: the EGA submitter Portal. How are files uploaded to the EGA? Data files are uploaded into private submission drop boxes (i.e. environments to which you are granted access and where you can transfer your files) using INBOX or FTP. These spaces are provided as part of the submission procedure. Before uploading any file, you must encrypt your files, . Only encrypted files shall be uploaded to the drop boxes. Why does data need to to encrypted for my submitted files? It is one of the security steps the EGA has implemented. In case of a security breach, people without the proper encryption key will not be able to read or use the information that could have been leaked. This measure is essential when working with sensitive data, such as controlled access human data. All submitters must use crypt4gh to create EGA compliant files prior to uploading them. This encryption is GPG-based, using EGA’s public key. Why are my files not available if I see them in the INBOX? There exists a time window between the data upload and the availability of such files via the Submitter Portal. For this reason, some metadata (run and analysis objects) cannot be registered until at least 24 hours after the files have been uploaded to your box. Why are MD5 sum values generated for my submitted files? We require pre- and post- encryption MD5 (message-digest) checksum values to be provided for all submitted files. These 128-bit values are computed using the content of each file, creating unique sequences that allow us to ensure that file integrity has been maintained during the transfer process. In other words, if the MD5 checksums we generate and those you generated match, we infer that the content of the transferred files is correct (i.e. files are not corrupted or truncated). MD5 checksums are computed automatically using the crypt4gh tool provided. Your submission will not be accepted and may be significantly delayed if you do not provide MD5 checksum values for all data files in the required format. How can I check if my files are correctly uploaded to the inbox? It is important to check the status pf your file so you know whether your files are in the inbox, being processed, or if there is any issue with one of them. In order to check this, you should: Look for the file locally. Drag and drop it to the file table. Then bars will appear on the table, which means that we are processing it. Green: The files checksum are correct and your file will move to “ingested files”. No further actions are needed from your end.. Red: The files checksum does not match and your file needs to be re-uploaded. Please re-upload relevant files to your inbox using the same path. After Submission How do I use my accession number in my publication? We suggest the use of the below template, using your study accession ID (EGAS…) : Data has been deposited at the European Genome-phenome Archive (EGA), which is hosted by the EBI and the CRG, under accession number EGASXXXXXXXXXXX. Further information about EGA can be found at https://ega-archive.org and "The European Genome-phenome Archive of human data consented for biomedical research" Your study ID will be the one that groups your whole submission, and thus its usage is recommended as such. Nevertheless, all metadata submitted to EGA hold a unique and persistent identifier (starting with EGA…) that can be used to identify specific records. For example, you could reference a specific dataset (EGAD…) or sample (EGAN…) in your publications (see full list of identifiers). How do I make my data searchable? Once you have finalized your submission, you can schedule the data release. Please take into account that the release process needs time for the files to be archived in our system, and for the Helpdesk team to validate your submission. Can I withdraw (meta)data from the EGA? We have methods in place for the secure removal of deposited (meta)data. Contact EGA-helpdesk for further details. EGA complies with FAIRness of (meta)data, and thus, even when the data is removed we keep an entry for their identifiers in our system. In other words, we execute a soft delete on canceled objects (e.g. a study): metadata is still stored in our systems, but it loses all links, cannot be queried and data files cannot be retrieved anymore. The reason behind this behaviour is so that queries using withdrawn data properly respond back (see example of a canceled study). What happens to the data once it has been submitted to the EGA? When the data is submitted, the submitter can choose either keep their data private or schedule the release of their data.

Documentation submission/metadata/submission/FAQ

DAC-2023-07-05-Ritz (DAC-007)

Full description of the dataset and all necessary information on metadata may be found under (https://zenodo.org/records/11237107): DAC-2023-07-05-Ritz (DAC-007), raw data in EGA, metadata in Zenodo (https://zenodo.org/records/11237107)

Dataset EGAD50000001147

DAC for EGA study: "ctDNA residual disease analyses during perioperative nivolumab or nivolumab plus ipilimumab in resectable diffuse pleural mesothelioma"

This DAC regulates access to the whole-exome sequence data for the study "ctDNA residual disease analyses during perioperative nivolumab or nivolumab plus ipilimumab in resectable diffuse pleural mesothelioma".

Dac EGAC50000000706

Hi-C for Junvenile Pilocytic Astrocytomas

5 Hi-C datasets (4 JPA and 1 LGG). Hi-C data for the remaining 5 JPAs used in our paper as well as all controls have been uploaded to EGA under EGAS00001005476.

Dataset EGAD00001009044

The EGA joins EUCAIM, the European project powering AI and imaging in cancer research

EUCAIM (EUropean Federation for CAncer IMages) is the cornerstone of the European Cancer Imaging Initiative, one of the flagships of the Europe’s Beating Cancer Plan (EBCP). The European Genome-Phenome Archive (EGA), as member of the EUCAIM consortium, attended the kick-off meeting organized this week in Brussels for the launch of the Initiative. The EGA will contribute to the project with the experience gained in the deployment of our federated network, the Federated EGA (FEGA), and the experience gained as partners of EuCanImage (AI4HI initiative) contributing to the definition of common data models, ontologies and data standards. The ultimate goal of the project is to unlock the power of imaging and Artificial Intelligence for the benefit of cancer patients, clinicians and researchers. The consortium will deploy a pan-European digital federated infrastructure hosting cancer-related images, and associated pathology, molecular and laboratory data from Real-World. This endeavour will provide a comprehensive dashboard for imaging data discovery, federated search, metadata harvesting, annotation and distributed data processing. EUCAIM joins 76 partners, gathering the experience and previous effort made by the different repositories of the AI4HI initiative, European Research Infrastructures and national/regional repositories. Find out more information about the EUCAIM launch here

Blog the-ega-joins-eucaim

NTNU Data Access Committee for “ProstOmics: spatial and bulk multi-omics of prostate cancer” datasets archived in Federated EGA Norway

The ProstOmics project of NTNU contains spatial and bulk multi-omics datasets of prostate cancer, and access request to these datasets are handled by this Data Access Committee on behalf of NTNU.

Dac EGAC50000000277

Paired Biopsy Project: West Coast Dream Team

This project contains the 16 WGS tumor samples and the corresponding 15 WGS normal tissue samples (germline control) not yet deposited in the public domain (e.g., EGA, dbGaP) at the time of submission of the manuscript (Zhu et al.)

Dataset EGAD50000000473

Boyes Lab - DAC policy

Access to this data is controlled. Upon request from a third party for access to the data, a separate Data Sharing Agreement would be put in place between the University of Leeds and the third party prior to EGA granting access to the data to the third party.

Dac EGAC50000000283

Submitter Portal API

Submitter Portal API The metadata submission process can be difficult and time-consuming. For this reason, the EGA has developed the Submitter Portal, a tool that was created to offer a simplified and user-friendly method of registering metadata. Our portal provides features that are intended to make the input of metadata easier, ensuring that your data is registered correctly and effectively. Our page is divided into logical sections and includes a helpful video instruction to further assist you as you complete the submission process. With the Submitter Portal, you can rest assured that the submission of your data is in good hands. For those that want a more flexible and automated approach, we also provide a programmatic approach using the Submitter Portal AP in addition to the user interface of the Submitter Portal. With the help of our API, you can quickly include the submission of metadata into your own workflow for a more effective and individualised experience. Previous steps Create your EGA account To submit data you first need to create your EGA user. Then, once your account has been verified, you will have to request a submitter role and sign the EGA Data Processing Agreement (DPA). When the DPA is signed, you must send it to EGA Helpdesk for further validation. Please, note that if you already have an EGA account, or you have an ega-box (submission account), you can skip this step. Upload your files Please note that all your files must be encrypted using the Crypt4GH tool before upload them.As soon as you are assigned with a submitter role, you will be able to connect to the EGA inbox and upload your files. Understand the EGA metadata schema It's crucial to comprehend the EGA metadata schema, a set of rules that specify how data is organised, described, and shared inside the EGA, in order to get the most out of this resource. Learn all about EGA metadata schema! Register your DAC and policy There are two objects in the EGA metadata schema that are registered in a separate portal. All DACs and policy objects are registered using the DAC Portal, a tool developed by EGA to help data controllers manage their data stored at the EGA. You can find the relevant information in the DAC Portal Guide. Programmatic submission All the calls to the API need to be Authenticated. We use the OpenID Connect protocol. API Usage Flow: Click here to check the API usage flow schema Obtain Access and Refresh Token: The first step to using the API is to obtain an access and refresh token. These tokens are required for authentication and authorisation of API requests. To obtain these tokens, you need to log in using your EGA credentials. Example: curl "https://idp.ega-archive.org/realms/EGA/protocol/openid-connect/token" \ -d "grant_type=password" \ -d "client_id=sp-api" \ -d "username=your-username" \ --data-urlencode "password=your-password" Use Access Token in API Calls: Once you have obtained the access token, you must use it in all the calls to the API. The access token is valid for a limited time period. When it expires, you will need to use the refresh token to obtain a new access token. Create Submission Object: After authentication, you can start creating a submission object. This object will be used to store all the metadata objects and files related to the submission. Example: curl "https://submission.ega-archive.org/api/submissions" \ -H "Authorization: Bearer your-access-token" \ -H "Content-Type: application/json" \ -d "{ \"title\":\"My submission title\", \"description\":\"My submission description\" }" Create EGA Metadata Objects: Within the submission object, you can create other metadata objects required for the submission. These objects may include information about the submitter, study, sample, experiment, and run. Example of study creation: curl "https://submission.ega-archive.org/api/submissions/{submisison-id}/studies" \ -H "Authorization: Bearer your-access-token" \ -H "Content-Type: application/json" \ -d "{ \"title\":\"My study title\", \"description\":\"My study description\", \"study_type\":\"Metagenomics\" }" If you would like to reuse already registered objects in new submissions, bear in mind that they must have an accession which you must use to reference them. You cannot reuse objects that have not been finalised and accepted yet.For example, when creating a new run which is reusing a previously submitted experiment, instead of using experiment_provisional_id, you must use experiment_accession_id. Enums There are some fields that only accept specific values, i.e. study_type. The API provides several enumerations to list these values. You can list all the enumerations, or a specific one, for example, study_types. Obtain IDs of files You can list the files available in your inbox and filter by a prefix by adding the prefix param, i.e. &prefix=/my_folder/abc Example: curl "https://submission.ega-archive.org/api/files?status=inbox&prefix=/my_folder/abc" \ -H "Authorization: Bearer your-access-token" \ -H "Content-Type: application/json" To improve performance and manageability, we have limited the number of files that the Submitter Portal API can return in a response. If a given call exceeds this limit, user will be asked to narrow down the search by using the query parameters already available in this endpoint. Finalise Submission: Once all the metadata objects are created and linked to the submission, you can finalise the submission. Finalising the submission sends it to the Helpdesk team for review. Please, note that all files linked to the submission must be ingested, and all objects created in a submission must be linked before you can finalise the submission. API Reference: For a full API reference, please check our specification documentation.

Documentation submission/metadata/submission/sequencing-phenotype/submitter-portal-api

23014 results for "ega"

in 60.57 milliseconds.