1248 results for ""halted+early"+"slower-than-anticipated+enrollment"+hospitalized+oral+adaptive+platform"

in 23.87 milliseconds.

LBC1936 fastq files

1075 members of the LBC1936 were sequenced using the Illumina HiSeq X platform. This dataset contains the paired fastq files.
Dataset EGAD00001008774
All you need to know about our new DAC Portal

With the DAC Portal, it is possible to streamline the process of managing access to sensitive data, ensuring that researchers have the necessary resources to advance scientific discovery while maintaining the highest standards of data protection and privacy. We know that, as a new tool, its first use can be complex. For this reason, in this article we will try to show all the elements of interest. Enjoy! Credentials: who needs to create a new user to access the DAC Portal and who doesn't? Only new users (since September the 12th) need to create an EGA account to access the DAC Portal. The EGA team went one step ahead by creating EGA accounts for former DAC members to login to the DAC Portal. These accounts are linked to the personal email used in the DAC structure: To check the linked email, DAC members can paste the DAC ID right after this link: https://metadata.ega-archive.org/dacs/ (e.g. https://metadata.ega-archive.org/dacs/EGAC00001002543) In case of password oblivion, it is possible to set a new one. All the process is channelled through the personal email. Some DAC members may be interested in updating the DAC email. This action can be done contacting our Helpdesk team. For the correct functioning of the DAC Portal, users must add the missing information of the DAC Profile. For a deeper dive, check out our documentation. The first step: knowing your workspace on the Portal Once logged in to the platform using the EGA User credentials, note the sections available: DACs and Policies. In the first one it is possible to check all user's registered DACs by default. It provides a comprehensive overview of each DAC, including its EGA accessions (EGAC), title, status, and the number of data access requests associated with each DAC. Whereas the Policies section is where users can manage and view all their registered policies, along with their associated EGA accessions, links, and titles. A drop-down menu is available with shortcuts to the most useful pages in the top right corner. It is possible to check DACs and policies, as well as creating new ones. In this menu, users can also contact the Helpdesk team using the “Need Help?” section to make inquiries about DACs and policies. Comprehending the DACs list & making the most of all the features Colours are crucial in the DAC's section as they will indicate the user role and the status in every committee, namely: Yellow: you are the administrator of the DAC and have full control to add other contacts, modify the content, and manage data access requests. Orange: you are a member of the DAC and have the authority to manage data access requests but cannot make any changes to the content. Blue: you have been invited to join the DAC and need to either accept or decline the invitation. Grey: the DAC is currently pending validation by the EGA Helpdesk team. Red: the DAC has been declined by the EGA Helpdesk team. Discover more information on how to view the message from Helpdesk containing the declination reason or how to validate or reject a DAC invitation by taking the tour of the DAC Portal. How to create, register and edit a DAC Creating a DAC is an easy process that only requires users to complete the necessary information. After that, the EGA Helpdesk team will review the proposal and validate the proposal. Once the DAC is validated it is possible for authorised users with admin role to edit it at any time, ensuring that all information and contacts associated with the committee is accurate and up to date. Membership management is controlled by searching for the EGA user looking for their username, email, or organisation. Please, note that for a person's details to appear in this search engine, they must have registered as a EGA user. Data Access Committee administrators manage two types of permissions that apply to contacts: Main contact and Role. Both can be modified at any time: Main contact refers to the primary person we firstly contact for any inquiry about the DAC (please, note that we can contact all DAC members, and this refers to the person we will contact first). Role defines the actions that can be taken by each contact. There are two possible status, admin and member, details of which can be found in the Take the Tour. Keep in mind that, to save the modifications, it is always necessary to click on “Update”. Moreover, it is also possible to delete any contact in a DAC whenever it is needed. The data request process: how to manage access requests The data request process is now channelled through the EGA website. Once a user sends a request access, the petition will go directly to the DAC Portal profile of the DAC members liked to the requested dataset. Whenever a request is received, a sand clock symbol will be displayed next to the DAC responsible for the dataset involved. By clicking on the DAC, it is possible to discover more information about requester(s). In this section, DAC Portal users will also be able to modify the display of the data access requests. It is possible to group requests by dataset accession or username, depending on the user preferences. Additionally, it is possible to isolate specific requests by user or dataset. To accept a request, users must slide the button to the right. Once the button turns green, it signifies that the request is accepted; the requester will be granted access to the requested dataset. To make lives easier, we have implemented a “Select all” button to easily accept or revoke multiple requests at once. To decline requests, it is necessary to slide the button to the left. A red button means the requester will not be granted access to the dataset. The DAC Portal will always ask to submit the reason why the data access was denied. Please, note that any decision, grant or denial, must be confirmed clicking the "Apply" button. This will update the status of the request and notify the requester accordingly. Taking control of the activity in the DAC Portal: checking the requests history The History button is a useful feature that allows users to access a list of all the requests that have been granted. This is especially useful for auditing purposes, as it enables to keep track of who has been granted access to the data and when. To facilitate control for those users with several requests we have included a filtering option. It allows to filter requests by user, mail, dataset, EGA ID, and date (these are combinable to boost the search). Don't forget about policies: creation, management, and update Similarly to the DACs section, in the Policies one users can manage and view all their registered policies. By clicking on any policy, it is possible to view its details as well as making any necessary updates or modifications. The sheets symbol and its adjacent number indicate the quantity of objects using a policy. By clicking in the location symbol users will go directly to the DAC page on the DAC Portal to see more information about that DAC. DAC Portal users can create new policies. The first step requires to link the policy to a registered DAC. After defining the title, users can either add a link to an external URL containing terms & conditions or write the policy content directly on the DAC Portal. Keep in mind that both options can be included. Data Use Ontology (DUO) codes can be added to new policies. These codes are used to specify the permitted and prohibited uses of the data. Please, note that some of them require a modifier to provide additional specificity. Find out more by checking out this specific content on the DAC Portal Tour. Policies can be modified at any given time by clicking on one registered policy, which will display the current information.
Blog new-dac-portal
GAW16 Framingham and Simulated Data

Important links to apply for individual-level data Genetic Analysis Workshop Instructions to Request Authorized Access Data Use Certification Requirements (DUC) Apply here for controlled access to individual level data Research Use Statement Questions regarding GAW16 should be directed to Vanessa Olmo at vanessa@business-endeavors.com. Problem 2: Description of the Framingham Heart Study In GAW16, we use data drawn from the Framingham Heart Study. The Framingham Heart Study — under the direction of National Heart, Lung, and Blood Institute; NHLBI — began in 1948 with the recruitment of adults from the town of Framingham, Massachusetts. At the time, little was known about the general causes of heart disease and stroke, but the death rates for cardiovascular disease (CVD) had been increasing steadily since the beginning of the 20th century and had become an American epidemic. The Framingham Heart Study is now conducted in collaboration with Boston University. The objective of the Framingham Heart Study was to identify the common factors or characteristics that contribute to CVD by following its development over a long period of time in a large group of participants who had not yet developed overt symptoms of CVD or suffered a heart attack or stroke. Between 1948 and 1953 the researchers recruited 5,209 subjects (2,336 men and 2,873 women) between the ages of 29 and 62 from the town of Framingham, Massachusetts and began the first round of extensive physical examinations and lifestyle interviews that they would later analyze for common patterns related to CVD development. Subjects were recruited from lists of addresses recorded for the town. Two out of every three households were approached for participation in the study. While there was no intention to recruit families for family studies, the plan was to recruit all household members in the ages 30-60 within each house that was selected for study. Hence, many biologically related individuals were recruited, including 1644 spouse pairs. Since 1948, these participants have returned to the study every two years for a detailed medical history, physical examination, and laboratory tests. Now in 2008 at 60 years of follow up, there remain about 500 participants from this cohort. Between 1971 and 1975 the study enrolled a second-generation group — 5,124 of the original participants' children and the spouses of these children — to participate in similar examinations. 2,616 subjects are offspring of the original spouse pairs and 34 are stepchildren. A total of 898 offspring are children of cohort members where only one parent was a study participant and 1,576 are spouses of the offspring. The Offspring Cohort has been followed every four years through 2001 (except between Exams 1 and 2 with an intervening 8 years) using protocols similar to those used for study of the Original Cohort. Between 2002 and 2005 the study enrolled the third generation (Gen3) of the Framingham Heart Study - 4095 offspring of the second generation. None of their spouses were recruited. An additional 103 parents of this third generation, who were not recruited between 1971 and 1975, were also recruited at this time. The latter group is not included in the GAW16 data. With the recruitment of this third generation, the study has increasingly focused on genetic factors associated with the development of cardiovascular disease and its associated risk factors. To date, there is only one examination of this generation of participants. A description of the recruitment of this third generation and comparison with the earlier generations at their initial recruitment is presented in Splansky GL et al., 2007. Further information on the Study can be found at http://www.nhlbi.nih.gov/about/framingham/index.html. Genome-wide Dense SNP Scan in Framingham Heart Study Genetic studies did not begin in the FHS until the 1990s. In the late 1980s and through the 1990s DNA was extracted from blood samples of surviving FHS participants. In 2007, the FHS entered a new phase with the conduct of genotyping for the FHS SHARe (SNP Health Association Resource) project, for which dense SNP genotyping was performed using approximately 550,000 SNPs (GeneChip® Human Mapping 500K Array Set and the 50K Human Gene Focused Panel) in 10,775 samples (some duplicates) from the three generations of subjects (including over 900 pedigrees). Affymetrix conducted all genotyping for the FHS SHARe project, using the 250K Sty, 250K Nsp, and the supplemental 50K platforms. Eighty-nine percent of the DNA samples were collected during the 1990s. To maximize the power of the study, we also extracted DNA from 1133 blood samples, drawn from subjects who had no DNA, to include in the SHARe project. These samples had been sitting in our refrigerators for some time, a few as far back as the 1970s. We refer to these DNA samples as the legacy samples. These samples had a higher failure rate in the genotyping process (40%) than the other eighty-nine percent (3%). Affymetrix invoked its own criteria for a sample to succeed in genotyping. All non-legacy samples must succeed on all three platforms, while legacy samples needed to pass on at least one platform. When a sample failed, additional attempts were made. Samples that repeatedly failed 2-4 times were called failures. Other samples failed due to issues of genotyped sex identification not matching our records or low SNP concordance among SNPs common across arrays or contamination. Eighty-nine percent of the legacy samples for which genotyping results are available passed all three platforms. The genotyping data from the 10,043 samples from 9354 subjects that passed the Affymetrix criteria were additionally checked for gender consistency and consistency with family structure, resulting in genotyping data for 9,274 participants in FHS SHARe. Genotype calls were made with the BRLMM algorithm. The SHARe database is housed at the National Center for Biotechnology Information database of genotypes and phenotypes (NCBI dbGaP) and contains all ~550,000 SNPs. This genome-wide dense SNP scan and a subset of phenotypes from the Framingham Heart Study are the focus of the Genetic Analysis Workshop 16. Further information on the specific variables in the Problem 2 dataset can be found by clicking on the Documents tab at the top of the page. Problem 3: Description of the Simulated Data Set The focus of this simulation is gene discovery in genome-wide association scans (GWAS). The Framingham Heart Study data set (distributed as "Problem 2") is the basis for the FHS* simulated data. The pedigree structures are derived from the data distributed for Problem 2, and we distribute an accompanying triplet file (triplet_sim) containing person ID, father ID, mother ID, to ensure the identical subjects, pedigrees, and singletons are used in the simulated data analysis. Consistent with standard practice, founders and singletons are designated as subjects with both fshare and mshare equal to zero (missing). The simulated data includes a total of 6,479 subjects with both phenotype and genotype data, in 942 pedigrees distributed among 3 generations and 188 singletons. Data inclusion is consistent with the subjects' consent for use by both for-profit and not-for-profit researchers. The genotypes for all Problem 3 replicates are fixed as measured and distributed for Problem 2 for both the genomewide scan and the additional candidate gene SNPs, for a total of approximately 550,000 SNPs (GeneChip® Human Mapping 500K Array Set and the 50K Human Gene Focused Panel). Thus, to analyze the Problem 3 simulated data you also will need to download the Problem 2 genotypes. Note that there are slight discrepancies in counts between Problems 2 and 3 due to a change in consents between the two datasets. Several phenotypes that contribute to coronary heart disease (CHD) were simulated for all individuals with genotypes across three different time points, 10 years apart. All genotyped individuals have complete data; the effects of missing values can be investigated by user-specified missing value patterns. There are 200 longitudinal datasets created, based on the generating model, and each replication is found in a separate dataset. We suggest that if only one replication is to be analyzed, that it be replication 1 to enable more precise comparisons among analytical approaches. The 'shareid' will allow you to merge the simulated phenotype data with the Problem 2 genotype data, and reconstruction of the pedigrees using the distributed 'triplet_sim' file or for larger families' relationships with the triplet distributed with the Problem 2. The simulated data problem is further described in the associated readme file, and a data dictionary is provided defining all the variables. For disclosure of the generating model for these data, please contact Jean MacCluer at jean@sfbrgenetics.org.
Study phs000128
Submitter Portal

Submitter Portal The Submitter Portal does not support array submissions! Please refer to the array documentationAny new objects registered in the Submitter Portal will not be available for use in an array submission For further information please check our Submission FAQs, submission quickguide as well as submission terms! The metadata submission process can be difficult and time-consuming. For this reason, the EGA has developed the Submitter Portal, a tool that was created to offer a simplified and user-friendly method of registering metadata. Our portal provides features that are intended to make the input of metadata easier, ensuring that your data is registered correctly and effectively. Our page is divided into logical sections and includes a helpful video instruction to further assist you as you complete the submission process. With the Submitter Portal, you can rest assured that the submission of your data is in good hands. Submitter Portal Index Previous steps Access to Submitter Portal Start a Submission Take The Tour Tutorial Previous steps Create your EGA account To submit data, you first need to create your EGA user. Then, once your account has been verified by the Help Desk team, you will be able to request a submitter role and send the EGA Data Processing Agreement (DPA). In the case you already have an EGA account, or an ega-box (submission account), you can skip this step. Upload your files Please note that all your files must be encrypted using the Crypt4GH tool before upload them.As soon as you are assigned with a submitter role and added your public key to your profile, you will be able to connect to the EGA inbox and upload your files. Understand the EGA metadata schema It's crucial to comprehend the EGA metadata schema, a set of rules that specify how data is organised, described, and shared inside the EGA, in order to get the most out of this resource. Learn all about EGA metadata schema! Register your DAC and policy There are two objects in the EGA metadata schema that are registered in a separate portal. All DACs and policy objects are registered using the DAC Portal, a tool developed by EGA to help data controllers manage their data stored at the EGA. You can find relevant information in the DAC Portal Guide. Access to Submitter Portal 1. Request Submitter Role Once you have created your EGA user, request a Submitter Role and complete relevant information such as submission type and size. 2. Signature of the Data Processing Agreement The Data Processing Agreement (DPA) signed by CRG and EMBL-EBI (Joint Data Processors for the EGA) can be downloaded to the submitter for its signature. Further information related to the DPA can be in the Privacy Notice section. The DPA should be signed by a legal representative of the submitter with power of attorney/appointment duly authorised to sign contracts on behalf of the institution. The submitter will return a fully-executed copy of the DPA to us. Please note that due to the high volume of submissions to EGA the DPA is non-negotiable. Once you have completed all the information and attached the DPA signed, send your Submitter request and the EGA Helpdesk Team will validate it and send a confirmation to start submitting your metadata. Start a Submission The steps below will show you how to start a submission and are accompanied with a tutorial and the Take the tour of Submitter Portal. 1. Login Go to the Submitter Portal interface and log in using your EGA account credentials. 2. Create a submission The Submitter Portal interface will show you any already existing open submission. To start a new submission click on “Create a Submission”.In this phase you have the option to add any collaborator(s) as well as their permissions (“Read only” or “Read & Write”). If you want to invite all the collaborators from a previous submission, you can add them by selecting said submission.In the event that the submission has been closed, you will need to reopen it. To do this, you will need to modify one of the objects in the submission, for example, copy and paste the title. Once the submission is open, you can add a collaborator. Further information can be found in the Submission Tour 3. Create Study Click on add Study and complete the form with the relevant information. The fields with an asterisk (*) are mandatory. Further information can be found in the Submission Tour4. Create Sample Sample registration can be done either individually or in batch by using a template. Batch upload: download the template and complete it with information about the sequencing samples. Upload the samples.csv and the sample will be created.Add single sample: complete the mandatory fields with the metadata information of your sample. Further information can be found in the Submission Tour5. Create Experiment Register an experiment with information about the sequencing methods, protocols and machines. Experiments generate the connection between samples and study. Further information can be found in the Submission Tour6. Create Run Samples, experiments and files are linked through runs. To register a run, you will be asked to select the appropriate experiment and the file format. Run registration can be done either individually or in batch by using a template. Batch upload: download the template and complete it with the sample - file linkage. Upload the run.csv and the runs will register.As the INBOX is a directory, bear in mind to include the file path (/). Example: /filename.c4gh. Add single run: select the file format, sample and experiment from the drop box. For re-using submitted samples, please use the EGA accession ID (EGANXXX). Please note that files must be uploaded in your inbox before linking your files and samples. Further information can be found in the Submission Tour7. Create Analysis Analysis should be only used for BAM/BAI or CRAM/CRAI pair, VCF and phenotype linkage to samples. The analysis is an EGA specific metadata object that links samples to files. This object also stores some metadata about your experiments, such as the experiment type, genome reference, or the platform used. **If only BAM or CRAM alignment files are submitted but not the original unaligned FASTQ files, then please make sure that the BAM or CRAM files also contain the unaligned reads. This is critical to enable primary re-analysis and re-alignment of the dataset using new tools or future genome assemblies.** Start by selecting the samples to be linked to the file, and then populate the required attribute fields. Please be aware of the existence of mandatory fields that must be populated.When populating the chromosome field (mandatory), please select the chromosomes from the checkbox. Please take in consideration that EGA allows for the re-use of any registered metadata. Therefore, a previously registered study or samples can be referenced for the analysis in the current data submission.Further information can be found in the Submission Tour 8. Create Dataset Dataset contains the collection of runs/analysis data files subjected to controlled access by being associated with a Policy. Complete the mandatory fields such as title, description, type and policy. Please, take in consideration that DAC and Policy must be registered in advance in the DAC Portal. Similar to other metadata objects, you can re-use previous runs or analyses, using their EGA accessions IDs (runs - EGAR, analyses - EGAZ). Further information can be found in the Submission Tour 9. Finalise submission Once the files are ingested please delete them from your SFTP INBOX before finalising your submission. This will prevent the files from reappearing in your Submitter Portal Inbox. Once you have completed your submission, click on the Finalise button, and schedule an estimated release date. Afterwards, your submission will be sent to Helpdesk for approval. Please note that once the Helpdesk has approved your submission, you will receive the identifiers for your data. See the Frequently Asked Questions (FAQ) for more information. 10. Update closed submission If you want to append more file to a closed submission, you will need to re-open the submission where the dataset was registered. Find the Submission: go to the Submitter Portal Menu -> Datasets section and find the submission id (EGA0XXX). Reopen the submission where the dataset is located: to reopen a submission, you must edit any object within the submission. For example, you can copy and paste the title. Take The Tour The Submitter Portal Take The Tour is a visual guidance of all the steps described previously about how to start a submission. Tutorial In this tutorial we will show you how to use the Submitter Portal to register your metadata. Please note the video focuses on the run-based submission (for raw files - fastq - and aligned data - BAM/CRAM) and analysis-based submission (for your BAM/BAI pairs, variation -VCF - and phenotype files).
Documentation submission/metadata/submission/sequencing-phenotype/submitter-portal
Distribution of data using Live Distribution

Live Distribution Welcome to the documentation for using the Live Distribution feature for distributing data files securely through the EGA platform. This guide will walk you through the process of downloading encrypted files and decrypting them using Crypt4GH. Please follow the steps below to ensure a smooth experience. Before Downloading Create an EGA user. Make sure that you have the permissions to download the dataset of interest. In case you don’t have access, request access to the dataset. Add your Crypt4GH-compatible public key to your EGA account. Please allow a few hours for your public key to be synced with your profile. Afterwards, you will be able to connect to your EGA outbox using the SFTP protocol. Download Graphical User Interface (GUI) You can use any GUI that supports SFTP connections, such as FileZilla, an open-source FTP client. For Filezilla as your GUI, follow these steps to download files: Open FileZilla and access Site Manager (File > Site Manager). Create a new connection with the following settings (Figure 1): Protocol: SFTP - SSH File Transfer Protocol Host: __EGA_OUTBOX_DOMAIN__ Logon Type: Key file User: your EGA username Key file: path/to/your/private_key Figure 1: Process of establishing a new connection to __EGA_OUTBOX_DOMAIN__ using a key file as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the EGA outbox, ensuring seamless data transfers. Click Connect to access your Outbox. This folder serves as your storage space within the EGA cloud, containing files accessible for download in a secure way. Browse the remote directory on the right side of the FileZilla screen. Select the files you wish to download, right-click, and choose Download (Figure 2). Figure 2: Step-by-step process of manually downñoad files from __EGA_OUTBOX_DOMAIN__ using FileZilla, with FileZilla version 3.52.2 operating on IOS v11.2.3. The figure demonstrates how users can downñoad data from the EGA outbox to their local storage by following the depicted steps SFTP command line You can also use the SFTP command line to securely download files from the EGA Outbox. Using SFTP command line client in Linux/Unix Open a terminal window Enter the following command to connect: sftp username@hostname Enter your EGA password To see a list of available sftp commands type help sftp> put – Upload file sftp> get – Download file sftp> cd path – Change remote directory to ‘path’ sftp> pwd – Display remote working directory sftp> lcd path – Change the local directory to ‘path’ sftp> lpwd – Display local working directory sftp> ls – Display the contents of the remote working directory sftp> lls – Display the contents of the local working directory Type get command to download files. For example: get encrypted_file.c4gh Use the bye command to close the connection (SFTP session). Convenient SSH settings Include the following settings in your SSH config file, located in ~/.ssh/config Host __EGA_OUTBOX_DOMAIN__ EGA-outbox hostname __EGA_OUTBOX_DOMAIN__ User username IdentityFile path/to/the/private/key Replace username and path/to/the/private/key with the appropriate settings, and you will be able to connect to the __EGA_OUTBOX_DOMAIN__ simply using sftp EGA-outbox. How to decrypt Files archived at the EGA are encrypted based on Crypt4GH. Hence, to decrypt the files you need to install Crypt4GH. You can install a python implementation of it, with pip install crypt4gh or directly from the Github repository pip install git+https://github.com/EGA-archive/crypt4gh.git After installing Crypt4GH, decrypt files using the following command: crypt4gh decrypt --sk /path/private/key < encrypted_file.c4gh > decrypted_filename The command reads the encrypted file from stdin (with <) and output the decrypted version to stdout (with >). Replace encrypted_file.c4gh and decrypted_filename with the appropriate filenames but make sure to not use the same filename for both reading and writing because your SHELL would then truncate both files before you even read or write. Frequently Asked Questions What username should I use to log in to my outbox? The authentication process for logging in to the EGA website, as well as accessing your inbox and outbox, requires the use of your username, not your email address. Therefore, if you registered a username different from your email address when creating your EGA account, you must use that username to log in. If you have forgotten your registered username, please, contact our Helpdesk team for assistance. I see that some files in my dataset have 'unavailable' as extension. What should I do? Within your Outbox, you'll find a list of all the datasets available for download. Occasionally, certain files may be marked as "unavailable". These unavailable files can be identified by the "unavailable" extension added to their filenames (e.g. filename.fastq.gz.unavailable.c4gh). If you encounter an unavailable file that you need, please reach out to our Helpdesk. We'll promptly work on making the file accessible for download as soon as possible. Specific to using keys Can I access one EGA account from different devices? Yes, you can access your account from different devices by linking several public keys to your EGA account. Each device can generate a unique public-private key pair, and the corresponding public keys can be linked to the same account. This way, you can use different public keys on different devices and still have access to the same account and data. I have several keys and I don't remember which one is which When generating SSH keys, it's a good practice to add a comment using the -C flag. This will allow you to add a descriptive tag to your key, making it easier to identify later on. Here's an example command that generates an SSH key with a comment: ssh-keygen -t ed25519 -C work-pass In this example, we're generating an ed25519 SSH key with the comment work-pass. Once you have multiple keys with different comments, you can use the comments to easily identify each key. To view the comments for your existing SSH keys, you can use the following command: ssh-keygen -l -f /path/to/key This will display the key fingerprint and the associated comment. By checking the comments, you should be able to identify which key is which. What if I can't find my SSH keys for uploading files with a key file, and how can I use new keys? If you can't find your SSH keys, don't worry - you can make new ones. To do this, open your terminal or command prompt and type a command to make a new SSH key. You can pick a name for the key, and choose a password to keep it safe. After making the key, you can add the new key to your account or server where you want to upload files using the key file. This usually involves copying and pasting the key's "public" (e.g. file.pub) part to the right place. If you lose track of the key again, just make a new one and add it again. Keep in mind that SSH keys belong to you and your computer, so if you switch computers or accounts, you'll need to make new keys. I don't want to type the passphrase every time I use the key. What can I do? You can use an ssh-agent to avoid typing the passphrase every time you use the key. An ssh-agent is a program that stores your private keys in memory and provides them to ssh when needed. You can add your key to the ssh-agent using the command ssh-add followed by the path to your key file.Here's an example of the steps to follow: Open a terminal window.Start the ssh-agent by typing the command eval $(ssh-agent).Add your key to the ssh-agent by typing the command ssh-add [key filepath]. For instance, if your key file is located in the home directory with the name mykey, the command will look like this: ssh-add ~/mykey After adding your, key to the ssh-agent, you should be able to use ssh without having to enter your passphrase every time. Can I use my password for authentication (without my private key)? If you prefer to use your username and password for authentication instead of your private key, you can still do so. When using a Graphical User Interface (GUI) such as FileZilla, you can select Ask for password as your Logon Type (Figure 3). This option will prompt you to enter your password when you click Connect, instead of using your private key. Figure 3: This option will prompt you to enter your password when you click "Connect", instead of using your private key. Figure 3: Process of establishing a new connection to __EGA_OUTBOX_DOMAIN__ using your password as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the inbox, ensuring seamless data transfers. It's worth noting that using a password for authentication can be less secure than using an SSH key, as passwords can be more easily compromised through various means. However, if you choose to use your password for authentication, selecting "Ask for password" as your Logon Type is a good way to do so securely via a GUI. Why is it better to use my key and not my password? SSH keys for authentication is generally considered to be more secure and convenient than using passwords. SSH keys are more difficult to crack than passwords, and they can be restricted to specific users and machines, giving you more control over access. Once you set up your SSH keys, you can use them to authenticate quickly and easily, without having to enter a password every time. This makes automation of tasks, such as uploading encrypted files, much simpler. Additionally, SSH keys provide better logging, allowing you to keep track of who is accessing your systems and when. All in all, using SSH keys is a good practice for improving security and convenience in your authentication process.
Documentation access/download/files/live-outbox
Submission FAQ

Submission FAQ Before Submission Is the EGA the right archive for my data? TThe most suitable archive for your data is dependent on the type of data you are wishing to submit and whether the data require public or controlled access. Public access is defined as complete and open access to all submitted data. On the contrary, controlled access, exerted by the EGA, requires formal applications to be made to access the submitted data files and metadata. EGA only accepts human-derived data subject to controlled access. If your submission contains other types of data, please choose the appropriate repository for it (see image below): ENA, EVA, ArrayExpress, BioSD and GWAS catalog. Should your submission be subjected to controlled access? Data access conditions are normally defined in the original informed consent agreements signed by the participants involved in your study. All data submitted to the EGA is subject to controlled access. These consents prevent the derived data files, potentially identifiable, from being dispersed by open and public access. Controlled access data often refers to human data derived from medical research and consortium projects. If in doubt, consult the informed consent agreements that apply to your study The EGA enables you to hold a submission before publication. What data types can be submitted to the EGA? Data types accepted by the EGA can be split into three categories: Sequences: both in generic and platform-specific formats. Array-based: from raw signal files to processed matrices. Phenotypes: all possible phenotype formats are accepted. All manufacturer-specific raw data formats derived from major next generation sequencing platforms are accepted. Also generic sequence formats: flat reads in a FASTQ file, aligned sequences (BAM or CRAM files) as well as sequence variation files in VCF format. All array-based technologies are accepted, including raw data, intensity and analysis files, without any restriction on data formats accepted. We also accept and distribute phenotype data (associated to the samples) in almost any format: from an image to a README file. How long does a submission take? Submissions to EGA come in a variety of formats and sizes, thus it is difficult for us to exactly predict how long a submission will take. We, therefore, advise all of our submitters to allow as much time as possible to make a submission. Based on previous records, we anticipate that the submission process may take at least one month. The submitter’s familiarity with the procedures, possible technical issues that may arise during submission and the amount of data that is being submitted are the main factors that will affect the length of the submission process. However, each step of a regular submission should be considered when estimating the time it would take: Encryption of the files Upload of the files Metadata submission Archival of the files Release of the study and datasets to EGA webpage For example, the upload of the files depends on the submission size, while metadata submission mainly relies on each submitter’s expertise. Further, some steps (e.g. answering to inquiries) depend on the EGA Helpdesk team, which may take some days to be processed during busy times Is data deposited in the EGA secure? The EGA set-up consists of a secure computing facility for data processing, a shared EBI set-up for data submissions and distribution of data via data requests made through the EGA website. Data is also copied in the Barcelona Supercomputing Center (BSC) infrastructure, where all stored and distributed data is encrypted Data is encrypted along the submission process and stored securely, granting its access to authorised users exclusively. During the download process, through our Python Client or Aspera, all requested data is downloaded over secure https connections. All data at the EGA is encrypted, and only accessible (for log-in and download) through secure protocols. For further information please, visit our security overview. What documentation do I need to provide? All submissions require policy’s documentation: 'Data Access Agreement (DAA)', 'Data Processing Agreement (DPA)' and 'Authorized Submitters Formulary'. The data processors (EGA) and the data owners will also sign the DPA. Will all metadata be public? Among the submitted metadata we need to make the distinction between identifiable and unidentifiable metadata: (1) the former may allow the identification of the human the sample derived from (e.g. detailed geographical providence, personal name, family ancestry…); (2) while the latter can be used to interpret the data without compromising the anonymity of the patients. The majority of the metadata submitted to the EGA corresponds to the unidentifiable category (e.g. sequencer's model). This type of metadata is publicly available on the EGA website and other EBI resources/partners’ websites. On the other hand, some parts samples’ metadata are subject to being identifiable, and thus only accessible by authorized data requesters, with the exception of: 5 submitter-defined attributes of the sample: alias, title, subject_id, gender and phenotype. It is the submitter’s responsibility not to submit sensitive metadata in these public fields. 3 anonymised fields that pinpoint the sample record in archivals: sample’s EGA stable ID (EGAN…), BioSample ID (SAMEA…) and submitter’s center name. During Submission Are there any sample specific requirements for EGA? All samples submitted to the EGA must include the attributes of biological sex, subject ID (anonymised individual identifier) and phenotype information. These are critical for data findability and its analysis, and we highly recommend using controlled ontology terms where applicable. For example: defining tumour and non-tumour samples and/or defining disease state. The EGA recommends using the Experimental Factor Ontology Database to find ontologized terms that describe your sample phenotypes. How do I get an accession number to use in my publication? You will receive your study accession number (EGAS…) upon complete your submission, either: Programmatically. As soon as the metadata is submitted and validated your study will be assigned an accession number that will be given in the submission’s response. Manually registering your study and relevant metadata using the online metadata submission tool: the EGA submitter Portal. How are files uploaded to the EGA? Data files are uploaded into private submission drop boxes (i.e. environments to which you are granted access and where you can transfer your files) using INBOX or FTP. These spaces are provided as part of the submission procedure. Before uploading any file, you must encrypt your files, . Only encrypted files shall be uploaded to the drop boxes. Why does data need to to encrypted for my submitted files? It is one of the security steps the EGA has implemented. In case of a security breach, people without the proper encryption key will not be able to read or use the information that could have been leaked. This measure is essential when working with sensitive data, such as controlled access human data. All submitters must use crypt4gh to create EGA compliant files prior to uploading them. This encryption is GPG-based, using EGA’s public key. Why are my files not available if I see them in the INBOX? There exists a time window between the data upload and the availability of such files via the Submitter Portal. For this reason, some metadata (run and analysis objects) cannot be registered until at least 24 hours after the files have been uploaded to your box. Why are MD5 sum values generated for my submitted files? We require pre- and post- encryption MD5 (message-digest) checksum values to be provided for all submitted files. These 128-bit values are computed using the content of each file, creating unique sequences that allow us to ensure that file integrity has been maintained during the transfer process. In other words, if the MD5 checksums we generate and those you generated match, we infer that the content of the transferred files is correct (i.e. files are not corrupted or truncated). MD5 checksums are computed automatically using the crypt4gh tool provided. Your submission will not be accepted and may be significantly delayed if you do not provide MD5 checksum values for all data files in the required format. How can I check if my files are correctly uploaded to the inbox? It is important to check the status pf your file so you know whether your files are in the inbox, being processed, or if there is any issue with one of them. In order to check this, you should: Look for the file locally. Drag and drop it to the file table. Then bars will appear on the table, which means that we are processing it. Green: The files checksum are correct and your file will move to “ingested files”. No further actions are needed from your end.. Red: The files checksum does not match and your file needs to be re-uploaded. Please re-upload relevant files to your inbox using the same path. After Submission How do I use my accession number in my publication? We suggest the use of the below template, using your study accession ID (EGAS…) : Data has been deposited at the European Genome-phenome Archive (EGA), which is hosted by the EBI and the CRG, under accession number EGASXXXXXXXXXXX. Further information about EGA can be found at https://ega-archive.org and "The European Genome-phenome Archive of human data consented for biomedical research" Your study ID will be the one that groups your whole submission, and thus its usage is recommended as such. Nevertheless, all metadata submitted to EGA hold a unique and persistent identifier (starting with EGA…) that can be used to identify specific records. For example, you could reference a specific dataset (EGAD…) or sample (EGAN…) in your publications (see full list of identifiers). How do I make my data searchable? Once you have finalized your submission, you can schedule the data release. Please take into account that the release process needs time for the files to be archived in our system, and for the Helpdesk team to validate your submission. Can I withdraw (meta)data from the EGA? We have methods in place for the secure removal of deposited (meta)data. Contact EGA-helpdesk for further details. EGA complies with FAIRness of (meta)data, and thus, even when the data is removed we keep an entry for their identifiers in our system. In other words, we execute a soft delete on canceled objects (e.g. a study): metadata is still stored in our systems, but it loses all links, cannot be queried and data files cannot be retrieved anymore. The reason behind this behaviour is so that queries using withdrawn data properly respond back (see example of a canceled study). What happens to the data once it has been submitted to the EGA? When the data is submitted, the submitter can choose either keep their data private or schedule the release of their data.
Documentation submission/metadata/submission/FAQ
eMERGE Network's Multi-Center Pilot of Pharmacogenetic Sequencing in Clinical Practice

eMERGE-PGx is a multi-site test of the concept that sequence information can be coupled to electronic medical records (EMRs) for use in healthcare. The promise of personalized medicine - health care guided by each individual's biological characteristics - is being fostered by increasingly powerful and economical methods to acquire clinically relevant biomarkers from large numbers of people. One therapeutic area that seems especially ripe for an early test of the personalized medicine concept is pharmacogenomics (PGx) - the idea that individual variation in drug response includes a genomic component. Drug response variation is an accepted feature of virtually all drug treatments, and contemporary molecular biologic tools continue to identify key genes mediating drug metabolism, transport, and targets. Importantly, common variation in these genes is an increasingly well-recognized contributor, sometimes with large effects, to variation in drug responses. As a result, recommendations for genotype-guided therapy are increasing. These evidence-based recommendations, if implemented in health care practice, could reduce adverse drug events and improve time to therapeutic response. Through eMERGE-PGx, we are developing strategies for the optimal implementation of genetic sequence data into the clinical environment with the ultimate goal of improving patient care. Site and participants include: Children's Hospital of Pennsylvania (CHOP): The Center for Applied Genomics (CAG) at the Children's Hospital of Philadelphia (CHOP) is a high-throughput, highly automated genotyping and sequencing facility equipped with state-of-the-art genotyping and sequencing platforms. Children who are treated at the Children's Hospital Healthcare Network and their parents may be eligible to take part in a major initiative to collect more than 100,000 blood samples, covering a wide range of pediatric diseases. The PGx population selected for sequencing with the PGRNseq panel at CHOP is 1,650 children from CAG's biorepository with well-documented drug-related severe adverse events (SAEs) or EHR-based drug response profiles. SAEs were extracted from EPIC records and from CHOP's Adverse Event (AE) database, which documents every AE at CHOP. These AEs are classified by a medical review panel according to the causal relationship with the suspected drug into 'doubtful', 'possible', and 'probable'. Individuals with events classified as probable, severe and objective, were selected for sequencing. The drugs more frequently associated with adverse events are antibiotics, antineoplastics, immunosuppressants and psychotropic drugs. This cohort constitutes 50% of the target population. The remaining subjects were selected using EHR-based algorithms that we have developed and validated at CAG for identifying patients not responding to ADHD medication (primarily atomoxetine) and patients refractory to antiepileptic treatment from responders. Cincinnati Children's Hospital Medical Center/Boston's Children's Hospital (CCHMC/BCH): 811 CCHMC samples were obtained from children, adolescents or young adults exposed to medication or at risk for needing medication of study interest. 55% of participants were exposed to one or more opioids and their DNA source was a CCHMC study-specific biobank; while 27% of participants were at risk for needing an opioid for surgical pain management and were newly recruited. The remainder of the cohort was exposed to methylphenidate and their DNA samples were obtained from a CCHMC study-specific biobank. The focus of Boston Children's Hospital eMERGE PGx project is on individuals with epilepsy. Samples were taken from a current pharmacogenomics study already in place through which DMET analysis was run and used as confirmation for PGRN-Seq results. A total of 109 samples were sent for PGRN-Seq analysis at University of Washington. The remaining 141 epilepsy samples were from Children's Hospital of Philadelphia and underwent testing with PGRN-Seq at CHOP. Geisinger Health System: A research cohort of adult Geisinger Clinic patients was enrolled from community-based primary care clinics of the Geisinger Health System. Patients were eligible for enrollment if they were a primary care patient of a Geisinger Clinic physician and were scheduled for a non-emergent clinic visit. All data are from Geisinger patients who consent to participate in the MyCode project. MyCode participants agree to provide biological samples for broad research use, including genomic analysis, and for linking of sample data to information in the participant's Geisinger health record. The consent also permits sharing of de-identified data for research purposes. Group Health(GH)/University of Washington (UW): Potential GH participants for the PGx project were enrolled in the eMERGE Network through the Northwest Institute of Genetic Medicine (NWIGM) biorepository, and provided the appropriate consent to receive clinically relevant genetic results (N~6300). Participants were eligible if aged 50 - 65 years old at the time of their enrollment into the NWIGM repository, living, enrolled in GH's integrated group practice, and had completed an online Health Risk Appraisal. The selection algorithm was based on several data sources from the EHR at Group Health: 1. Demographics - participants with self-reported race as Asian or African ancestry were prioritized and selected to enrich for non-European ancestry; 2. Diagnosis and procedure codes - participants were selected if found to have a history of hypertension, atrial fibrillation (AF,) or congestive heart failure (CHF). Participants with a history of arrhythmia were added if the entire selection algorithm did not generate 900 individuals. We also enriched for participants with EHR evidence of actionable indications related to PGRNSeq genes. Participants were selected if found to have an ICD9 code for malignant hyperthermia, hypertension, atrial fibrillation, congestive heart failure or long QT syndrome (LQTS); 3. Laboratory values - if a participant had any laboratory event of creatine kinase (CK) > 1000, and were dispensed statins within 6 months of the event, then they were selected; and 4. Medications - participants were excluded if ever on carbamazepine or had a current regimen of warfarin. Essentia Institute of Rural Health, Marshfield Clinic, Pennsylvania State University (Marshfield): For this study, 750 subjects were selected and enrolled into PGx based on Vanderbilt's algorithm designed to enrich for patients who are most likely to receive one of three common drugs (Clopidogrel, Warfarin or Simvastatin) in the next 2-3 years. These patients were sent a letter of invitation and description of the PGx project. Follow-up phone calls were made, and interested subjects came in for a one time meeting to discuss the project and go through the informed consent with the research coordinator. If they were interested they signed the consent and HIPAA forms and gave blood. Subjects were chosen and enrolled into PGx independently of previous biobank participation. Mayo Clinic: The Right Drug, Right Dose, Right Time - Using Genomic Data to Individualize Treatment (The RIGHT Protocol) enrolled 1013 patients to test the hypothesis that prescribers could deliver genome-guided drug therapy at the point-of-care by using pharmacogenomic data preemptively integrated in the electronic medical record. Complete details regarding the study population have been previously described (Bielinski et al., 2014). Icahn School of Medicine at Mount Sinai School (Mt Sinai): Our study site is the Primary Care Associates (PCA) practice group of the Mount Sinai Faculty Practice Associates (FPA) of the Mount Sinai Medical Center in New York City. This practice has 12 physician providers. All patient encounters are documented and managed with EpicCare ambulatory electronic medical record. Active PCA Patients eligible for enrollment fulfilled the following criteria: a) age 50 or older receiving clinical care at Mount Sinai FPA PCA practice with at least one practice encounter within 18 months prior to commencement of enrollment; b) no history or current use of clopidogrel, warfarin, or simvastatin. Eligible patients were invited to participate through de novo recruitment by letter sent by their provider. Interested patients were screened for eligibility and enrolled to participate in the eMERGE PGX study on site by a dedicated research coordinator. In addition to de novo enrollment from clinical practice, patients of FPA PCA who had previously enrolled in Mount Sinai's BioMe Biobank program AND fulfilled eligibility criteria as stated under a) and b) were identified by chart review and samples sequenced at CIDR using PGRNseq platform (N=300). PGRNseq data from 291 samples passed stringent quality control and are included in the current data set. Furthermore, 56 of these patients carrying known and validated 'actionable' variants affecting prescribing of clopidogrel, warfarin, and/or simvastatin were enrolled in the eMERGE PGX study following invitation through recontacting by the Principal Investigator of the BioMe Program. Northwestern University: Participants for this study were recruited from the General Internal Medicine (GIM) clinic at Northwestern Medical Group (NMG). Patients were selected for invitation to participate if they had been seen a minimum of two times over the last four years, having a high likelihood to receive a prescription for warfarin, Plavix, or a statin, and are seeing a physician who has agreed to allow their patients to be contacted for the study. We utilized an algorithm developed at Vanderbilt and tailored to our population which uses our EHR to estimate the probability that individuals will receive a prescription for warfarin, Plavix, or a statin in the next three years. Participants were sent a letter explaining the study prior to their GIM appointment and offered participation at the time of their visit. Participants were consented on-site and blood drawn after consent was obtained. The GIM clinic consists of 39 primary care physicians who provide approximately 80,000 patient encounters per year. As with any large primary care clinic, a significant proportion of patients in GIM clinic suffer from a variety of chronic health conditions, such as diabetes, hypertension, and coronary artery disease. Over 50,000 individuals have been seen by GIM doctors in the past 5 years; 11,562 of these patients have evidence of a statin prescription in the EHR, 3,436 have evidence of a warfarin prescription, and 1,872 have evidence of a Plavix prescription. Vanderbilt University: The more than 1000 participants enrolled into Vanderbilt's eMERGE PGx study were newly recruited from the Cardiology and Internal Medicine Clinics and the Hillsboro Medical Group within Vanderbilt University Medical Center (VUMC). Patients were selected based on a predictive algorithm estimating the patient's likelihood of receiving Clopidogrel, Warfarin, and/or Simvastatin. The algorithm identifies primarily older middle-aged patients, and the mean age of the study group is 74. The cohort is approximately 45% female with 75% of subjects self-identified as EA and 24% as AA. Subjects were consented in person by study personnel following a routine clinic visit and an introduction to the study staff by their doctor. VUMC is a comprehensive health care facility dedicated to patient care, research, and the education of health care professionals. Translational research into the causes and treatment of disease as well as studying fundamental biological properties is the primary focus of discovery at Vanderbilt. Clinical research is conducted in Vanderbilt University Hospital, the Nashville Veterans Administration Hospital, Meharry General Hospital and in their associated outpatient clinics. These hospitals and clinics, all associated with the Vanderbilt system, each have full time Vanderbilt faculty and medical housestaff and provide clinical care and participate in research programs. The Vanderbilt Clinic is comprised of more than 95 adult outpatient specialty practices and received over 1.5 million ambulatory visits in 2012-13. The Vanderbilt Heart and Vascular Institute offers a comprehensive heart program offering diagnosis, medical treatment, minimally invasive therapies, surgical intervention and disease management, tailored to each individual's unique needs. All programs within the Vanderbilt Clinic have survival figures that surpass the national average.
Study phs000906
Programmatic submission based on XML

Programmatic submissions (XML based) For further information please check our Submission FAQs, submission quickguide as well as submission terms! Introduction Besides the Submitter Portal tool, EGA supports programmatic sequence and clinical data metadata submissions. If you are not sure what this means, you may want to explore our brief metadata introduction. Programmatic submissions are recommended for array-based submission. Moreove, it may be of help if your submission is recurrent or it is difficult to manage manually due to its sheer size. Otherwise, we highly recommend using the Submitter Portal to perform submissions. In this page we will guide you through the required steps to programmatically submit data to the EGA. Programmatic submissions require your metadata to be structured for an easy and straightforward validation and archival. It basically consists in formatting your metadata as Extensible markup language (XML) files and submitting them to the EGA using the WEBIN Before submitting metadata to the EGA, it is important to ensure that the information in your XML files is compliant with our standards. You can see further details on how these standards are maintained at EGA at our EGA Schemas documentation page. Using WEBIN, you can validate your XML files against EGA's schemas to ensure that your metadata is compliant before submission. WEBIN services WEBIN production service WEBIN test service We advise you to submit your metadata to the test service when submitting to the production service for the first time. The test service is identical to the production service except that all submissions will be discarded in the following 24 hours. This allows you to learn about the submission process without having to worry about data being submitted. Authentication Authentication is required each time a submission is made. The submission service uses HTTPS protocol for metadata encryption and identification to provide a secure submission environment. Data file upload Both Runs and Analyses reference files (e.g. FASTQ need to be uploaded to the EGA before these metadata objects are submitted. In other words, if you submit a Run that references a file that we cannot find associated with your account, the metadata submission will fail. See further details on how to upload your files in our File Upload documentation. Metadata model of the EGA Our metadata model is formed by multiple metadata objects. Check further details in our documentation at our EGA Schema documentation page. Working with EGA XMLs files Now that the basic concepts of the EGA metadata have been described, you can start preparing your programmatic submission through XML. Here you will find the guidance on how to prepare the XML files. Programmatic Submission Tutorial Video Take a look at the Programmatic Submission Tutorial Video, which explains the workflow of a programmatic submission and goes over an example metadata submission. Programmatic Submission Tutorial Video. When building your XML files, we recommend using text editors (e.g.Sublime Text or VisualStudio) that allow you to visualise the structure of the XML with ease. Furthermore, these editors constantly check the consistency of the XML structure. Alternatively, and if the submission consists of a big number of objects (specially analyses), you may find the tool star2xml handy. This tool allows for a direct conversion between metadata in a tabular format (e.g. a spreadsheet) into XMLs. Identifying objects: Aliases and center names Every EGA object must be uniquely identified within the submission account using their alias attribute. The aliases can be used in submissions to make references between EGA objects. Let us dig into EGA's use of aliases and center names: alias: every object should have a name that is unique within your submission account. Once submitted successfully, every alias will be assigned a unique and permanent accession (EGA ID). refname: when an object references another by its alias, the alias of the referenced object goes into the "refname" attribute of the referencing object. For example, if a sample has the alias "sample1", and an experiment uses this sample, then the experiment's "EXPERIMENT/SAMPLE/refname" attribute should be "sample1". center_name: The "center_name" attribute is required within the submission XML and, if not provided when the object is submitted, it will be automatically filled using your default EGA account center_name. This element is the "controlled vocabulary acronym or abbreviation that is provided to the account holder when the account is first generated". If the submitter is brokering a submission for another institute, the submitter should use their special broker account name in broker_name while the data centre acronym remains in center_name. Log-in details should have been provided when you requested a submission account. Please contact our Helpdesk team if you have any questions. run_center: Many submitting centers contract out the actual sample sequencing to another center. In these cases, the sequencing center should be acknowledged in the run_center attribute. Again, this is controlled vocabulary and the acronym should be sought from EGA helpdesk before submitting. Please contact our Helpdesk team if you have any questions. Prepare your XMLs The goal of this section is to provide sufficient information to be able to create the metadata XML documents required for programmatic submissions. Please note, the EGA utilises the XML schemas maintained at the European Nucleotide Archive (ENA). It is important due to the fact that by using a similar system, some pieces of documentation from the ENA's programmatic submission can also help you with your programmatic submission to the EGA. For example, you can submit programmatically without using a Submission XML by following the steps at Submission actions without submission XML. A submission does not have to contain all different types of XMLs. For example, it is possible to submit only a few samples; or a study that is later to be referenced. You can submit each object one by one, or submit all in a batch: you choose what method of submission works best for you. We do recommend, nevertheless, that you submit the objects to be referenced (e.g. samples or studies) first, and the objects that reference these (e.g. experiments or datasets) afterwards. You can see a graphical view of these objects and their relationships at our EGA Schemas page. Independently of the submission scenario, you will always require a Dataset XML. The entity of a dataset is what is used to control access to the given data, in the form of runs or analyses. In other words, when a requester is granted access, it is through the dataset and the objects (e.g. runs or analyses) that the dataset contains, granting access to them in one go. Given the nature of the EGA, a dataset XML will always be required for the data access. First, we will differentiate between submissions of "raw" and "processed" data: Runs and Analyses, respectively. Run data submissions Raw data derives from instruments "as is". For example, a plain sequence file (e.g. FASTQ or unaligned BAM files) would be considered raw data. A typical raw (unaligned) sequence read submission consists of 8 XMLs: Submission Study Sample Experiment Run DAC Policy Dataset When technical reads (e.g. barcodes, adaptors or linkers) are included in the submitted raw sequences, a spot descriptor must be submitted to describe the position of the technical reads so that they can be removed. The following data files can be submitted without providing spot descriptor information in the experiment/run XML: BAM files (single reads) SFF files (single reads without barcodes) FastQ files (single reads without any technical reads) Complete Genomics files Analysis data submissions Processed data is, in some way, refined raw data. This includes raw data that has been processed by some form of analysis method (e.g. alignment, noise reduction, etc.). For example, an aligned sequence (e.g. BAM file), that was created using raw FASTQ files, would be a processed file. This category includes most types of data: sequence alignment files (e.g. BAM or CRAM), clinical data (e.g. phenopackets), sequence variation files (e.g. VCF), sequence annotation, etc. A typical EGA analysis data submission consists of 7 EGA XML: Submission Study Sample Analysis DAC Policy Dataset We accept three different types of analysis data submissions: BAM files (for multiple read alignments) VCF files (for sequence variations) Phenotype files (in any format) In anycase, keep in mind that samples must be created in order to be referenced in the analyses. In other words, the provenance of the information within the BAM, VCF and phenotype files Example XMLs Below you can find a non-extensive list of example XMLs with descriptive fields (i.e. explaining what to provide in each field). Furthermore, you can also find real examples (i.e. the true value of the provided fields) in our GitHub repository. Submission XML The submission XML is used to validate, submit or update any number of other objects. The submission XML refers to other XMLs. New submissions use the ADD action to submit new objects. Object updates are done using the MODIFY action and objects can be validated using the VERIFY action. Descriptive submission XML example True values submission XML example Study XML The study XML is used to describe the study containing a title, a study type and abstract as it would appear in a publication. Descriptive study XML example True values study XML example Please use the following notation within the property "STUDY_LINKS" when including PubMed citations in the Study XML: <STUDY_LINKS> <STUDY_LINK> <XREF_LINK> <DB>PUBMED</DB> <ID>18987735</ID> </XREF_LINK> </STUDY_LINK> </STUDY_LINKS> Sample XML The sample XML is used to describe the samples used to obtain the data, whether they were sequenced, measured in any other way, or have an associated phenotype. The mandatory fields include information about the taxonomy of the sample, sex, subject ID and phenotype. For example, the mandatory attribute fields for each sample would look like these, within the array of "SAMPLE_ATTRIBUTES": <SAMPLE_ATTRIBUTES> <SAMPLE_ATTRIBUTE> <TAG>subject_id</TAG> <VALUE>free text!</VALUE> </SAMPLE_ATTRIBUTE> <SAMPLE_ATTRIBUTE> <TAG>sex</TAG> <VALUE>female/male/unknown</VALUE> </SAMPLE_ATTRIBUTE> <SAMPLE_ATTRIBUTE> <TAG>phenotype</TAG> <VALUE>Free text, EFO terms (e.g. EFO:0000574) are recommended</VALUE> </SAMPLE_ATTRIBUTE> </SAMPLE_ATTRIBUTES> Sample is one of the most important objects to be described biologically, it is highly recommended that “TAG-VALUE” pairs are generated as SAMPLE_ATTRIBUTES to describe the sample in as much detail as possible. For example, were we to give the population ancestry of the sample, we could add a new attribute to the array, in which, for example, we would indicate that the sample derives from an individual of "Mende in Sierra Leone" (MSL), with an african ancestry: <SAMPLE_ATTRIBUTE> <TAG>Population</TAG> <VALUE>MSL</VALUE> </SAMPLE_ATTRIBUTE> Given that VALUE and TAG are free text, the combinations are limitless in order to give you full flexibility on the information you want to provide. We recommend you use the Experimental Factor Ontology (EFO) to describe the phenotypes of your samples. You can provide more than one phenotype by adding more items to the array of SAMPLE_ATTRIBUTES. Phenotypes considered essential for understanding the data submission should be provided. Each phenotype described should be listed as a separate sample attribute <SAMPLE_ATTRIBUTE> </SAMPLE_ATTRIBUTE>. There is no limit to the number of phenotypes that can be submitted. If a suitable EFO accession cannot be found for your phenotype attribute, please consider using another controlled ontology database (e.g. HPO, MONDO, etc.) before using free text. Descriptive sample XML example True values sample XML example Experiment XML The experiment XML is used to describe the experimental setup, including instrument platform and model details, library preparation details, and any additional information required to correctly interpret the submitted data. Where any of these values differ between runs, a new experiment object must exist, since runs are grouped by experiments. Each experiment references a study and a sample by alias, or if previously-submitted, by accession. Pooled data must be demultiplexed by barcode for submission. Descriptive experiment ( Illumina paired read ) XML example True values experiment ( Illumina paired read ) XML example Run XML The run XML is used to associate data files with experiments and typically comprises a single data file (e.g. a FASTQ file). Please note that pooled samples should be de-multiplexed prior submission and submitted as different runs. Descriptive run XML example True values run XML example Analysis XML Given that an analysis can be used to submit any type of processed data to the EGA, we will list below an example of each of the three most common types of analysis XMLs submitted to the EGA: sequence alignments (e.g. BAM files); sequence variation (e.g. VCF files); and clinical metadata or phenotypes (e.g. phenopackets). Regardless of the type of processed data submitted in the analysis, the analysis must be associated with a Study and can reference multiple types of other objects, from samples to experiments, if they are available at the EGA. Just like with Runs, whenever a file is submitted to the EGA through an analysis object, the file MD5 checksums must be present, in order for the EGA to validate file integrity upon transfer. This also includes index files when applicable (e.g. .bai.md5 files). Ideally, any analysis that uses a reference sequence for some kind of alignment (e.g. BAM, CRAM or VCF files), would contain metadata about the alignment, such as INSDC reference assemblies and sequences, by either using accessions (e.g. CM000663.1) or common labels (e.g. GRCh37). Read alignment (BAM) Analysis XML The Analysis can be used to submit BAM alignments to EGA. Only one BAM file can be submitted in each analysis and the samples used within the BAM read groups must be associated with Samples. Descriptive bam alignments XML example True values bam alignments XML example Sequence variation (VCF) Analysis XML The Analysis can be used to submit VCF files to EGA. Only one VCF file can be submitted in each analysis and the samples used within the VCF files must be associated with Samples. Download analysis XML (VCF) Phenotype files The Analysis XML can be used to submit phenotype files to the EGA. Only one phenotype file can be submitted in each analysis and the samples used within the phenotype files must be associated with EGA Samples. Download analysis XML (Phenotype) DAC XML The DAC XML describes the Data Access Committee (DAC) affiliated to the data submission. The DAC may consist of a group or a single individual and is responsible for the data access decisions based on the application procedure described in the POLICY.XML. As with any other object, if it was already submitted to the EGA, there is no need to submit it again: you can reference an existing object within the EGA. Hence, A DAC XML does not need to be provided if your submission is affiliated to an existing EGA DAC.. Further information on DACs can be found here, and you can always contact our Helpdesk team if you have further inquiries. Descriptive dac XML example True values dac XML example Policy XML The Policy XML describes the Data Access Agreement (DAA) to be affiliated to the named Data Access Committee. Descriptive policy XML example True values study XML example Dataset XML The dataset XML describes the data files, defined by the Run.XML and Analysis.XML, that make up the dataset and links the collection of data files to a specified Policy. The dataset xml is commonly the last metadata object to be submitted, since it references multiple other entities. Please consider the number of datasets that your submission consists of. For example, a case-control study is likely to consist of at least two datasets. In addition, we suggest that multiple datasets should be described for studies using the same samples but different sequence technologies. Descriptive dataset XML example True values dataset XML example Validating and submitting your EGA Validating EGA's XMLs through Webin After you have ensured that the XMLs are properly formatted and contain all the required information. You can proceed to validate and submit your data. Use the curl command to validate your XML file: Once you have prepared your XML file and asserted you have access to Webin, you can validate your XML file programmatically against EGA's schemas using the curl command. There are multiple ways in which you can validate your XMLs. This variety has to do with the fact that: (1) there are 2 instances of Webin (test and production); and (2) that validation is a default step during submission. In other words, any time that you submit your data through Webin, it will be validated automatically before being accepted. This allows for 4 possible routes of validation, all having the same validation result: validating or submitting to either the production service or the test service of Webin. For example, directly validating a "study" object XML in the testing service (wwwdev…) would look like the following: curl -u <USERNAME>:<PASSWORD> -F "ACTION=VALIDATE" "https://wwwdev.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" In this command, you would need to replace <USERNAME> and <PASSWORD> with your EGA account username and password, respectively. You would also replace <INPUT_FILE> with the path to your XML file. A mock example would look like the following: curl -u ega-test-data@ebi.ac.uk:egarocks -F "ACTION=VALIDATE" "https://wwwdev.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" The validation attempt can have different results depending on the given arguments: If your XML file is valid according to EGA's schemas, you will see a message indicating that your XML file is compliant. For example, see below for our mock example, where the "success" was "true" (i.e. no validation errors found). Nevertheless, notice how the "<STUDY accession=" is empty: it is because we were simply validating, so the study did not get an accession or ID. <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="receipt.xsl"?> <RECEIPT receiptDate="2023-04-11T15:19:28.850+01:00" submissionFile="submission-EBI-TEST_1681222768850.xml" success="true"> <STUDY accession="" alias="Mock example" status="PRIVATE"/> <SUBMISSION accession="" alias="SUBMISSION-11-04-2023-15:19:28:840"/> <MESSAGES> <INFO>VALIDATE action has been specified.</INFO> <INFO>Submission has been rolled back.</INFO> <INFO>This submission is a TEST submission and will be discarded within 24 hours</INFO> </MESSAGES> <ACTIONS>VALIDATE</ACTIONS> <ACTIONS>PROTECT</ACTIONS> If there are any errors or warnings, the tool will display them, allowing you to correct them before submitting your data to EGA. For example, in the following response, it is said that the object we were trying to submit was already existing, and therefore the "success" was "false". <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="receipt.xsl"?> <RECEIPT receiptDate="2023-04-11T15:12:35.609+01:00" submissionFile="submission-EBI-TEST_1681222355609.xml" success="false"> <STUDY alias="Example!_Human Microbiome Project SP56J" status="PRIVATE" holdUntilDate="2023-03-11Z"/> <SUBMISSION alias="SUBMISSION-11-04-2023-15:12:35:576"/> <MESSAGES> <ERROR>In study, alias: "Example!_Human Microbiome Project SP56J". The object being added already exists in the submission account with accession: "ERP127584".</ERROR> <INFO>VALIDATE action has been specified.</INFO> <INFO>Submission has been rolled back.</INFO> <INFO>This submission is a TEST submission and will be discarded within 24 hours</INFO> </MESSAGES> <ACTIONS>VALIDATE</ACTIONS> <ACTIONS>PROTECT</ACTIONS> If the curl command retrieves no response at all, please double check if your username and password are correctly provided. Also notice the "ACTION=..." argument passed to the Curl command. This specifies the action to take during the call to Webin, so we do not need a "Submission" XML just for a validation attempt. See more at submission actions without submission XML. Furthermore, validation of multiple files or objects (e.g. sample, experiment, study…) can be done in a single command by adding more arguments (i.e. '-F'). For example: curl -u <USERNAME>:<PASSWORD> -F "ACTION=VALIDATE" "https://wwwdev.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" -F "SAMPLE=@sample.xml" -F "DATASET=@dataset.xml" As mentioned above, beside "validate" action in the test environment, you can also validate your metadata by three other methods: "Validate" in the production server. From our example above, you simply need to take the "dev" away from the URL. curl -u <USERNAME>:<PASSWORD> -F "ACTION=VALIDATE" "https://www.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" "Add" in the development server. From our example above, you would simply need to replace the action: from "validate" to "add". Whatever is submitted to this service will be discarded in 24h, so whether something gets submitted or not would not matter in the long run. curl -u <USERNAME>:<PASSWORD> -F "ACTION=ADD" "https://wwwdev.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" "Add" in the productionserver. A combination of the previous two methods, which would render this attempt into a submission. This path is just to be taken when you are sure your metadata is compliant and what you want to submit. curl -u <USERNAME>:<PASSWORD> -F "ACTION=ADD" "https://www.ebi.ac.uk/ena/submit/drop-box/submit/" -F "STUDY=@study.xml" What happens after the submission of a dataset XML? Once you have completed the registration of your dataset/s please contact our Helpdesk Team to provide a release date for your study. Please note that all datasets affiliated to unreleased studies are automatically placed on hold until the authorised submitter or DAC contact contact the EGA Helpdesk for the study to be released. We strongly advise you not to delete your data until EGA Helpdesk confirms that your data has been successfully archived.
Documentation submission/metadata/submission/programmatic-submission-xml

10 25 50 100 records per page

First Previous ... 121 122 123 124 125

1248 results for ""halted+early"+"slower-than-anticipated+enrollment"+hospitalized+oral+adaptive+platform"

in 23.87 milliseconds.

LBC1936 fastq files

All you need to know about our new DAC Portal

GAW16 Framingham and Simulated Data

Submitter Portal

Distribution of data using Live Distribution

Submission FAQ

eMERGE Network's Multi-Center Pilot of Pharmacogenetic Sequencing in Clinical Practice

Programmatic submission based on XML