The dataset includes 12 paired FASTQ files (6 samples) with single cell transcriptome sequencing data of normal breast luminal cells from ducts and TDLUs derived from reduction mammoplasties from three patients. Chromium Single Cell 3’ Reagent Kit v2 or v3 (10x Genomics) were used for processing of cells, whereafter sequencing was performed using the Illumina® NextSeq500/550 High Output Kit v2. Cell Ranger was used for generating FASTQ files and files from different lanes were concatenated prior to uploading the data to EGA. "R1" files include the feature barcodes and UMIs, while "R2" files include the reads.
Live Distribution Welcome to the documentation for using the Live Distribution feature for distributing data files securely through the EGA platform. This guide will walk you through the process of downloading encrypted files and decrypting them using Crypt4GH. Please follow the steps below to ensure a smooth experience. Before Downloading Create an EGA user. Make sure that you have the permissions to download the dataset of interest. In case you don’t have access, request access to the dataset. Add your Crypt4GH-compatible public key to your EGA account. Please allow a few hours for your public key to be synced with your profile. Afterwards, you will be able to connect to your EGA outbox using the SFTP protocol. Download Graphical User Interface (GUI) You can use any GUI that supports SFTP connections, such as FileZilla, an open-source FTP client. For Filezilla as your GUI, follow these steps to download files: Open FileZilla and access Site Manager (File > Site Manager). Create a new connection with the following settings (Figure 1): Protocol: SFTP - SSH File Transfer Protocol Host: __EGA_OUTBOX_DOMAIN__ Logon Type: Key file User: your EGA username Key file: path/to/your/private_key Figure 1: Process of establishing a new connection to __EGA_OUTBOX_DOMAIN__ using a key file as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the EGA outbox, ensuring seamless data transfers. Click Connect to access your Outbox. This folder serves as your storage space within the EGA cloud, containing files accessible for download in a secure way. Browse the remote directory on the right side of the FileZilla screen. Select the files you wish to download, right-click, and choose Download (Figure 2). Figure 2: Step-by-step process of manually downñoad files from __EGA_OUTBOX_DOMAIN__ using FileZilla, with FileZilla version 3.52.2 operating on IOS v11.2.3. The figure demonstrates how users can downñoad data from the EGA outbox to their local storage by following the depicted steps SFTP command line You can also use the SFTP command line to securely download files from the EGA Outbox. Using SFTP command line client in Linux/Unix Open a terminal window Enter the following command to connect: sftp username@hostname Enter your EGA password To see a list of available sftp commands type help sftp> put – Upload file sftp> get – Download file sftp> cd path – Change remote directory to ‘path’ sftp> pwd – Display remote working directory sftp> lcd path – Change the local directory to ‘path’ sftp> lpwd – Display local working directory sftp> ls – Display the contents of the remote working directory sftp> lls – Display the contents of the local working directory Type get command to download files. For example: get encrypted_file.c4gh Use the bye command to close the connection (SFTP session). Convenient SSH settings Include the following settings in your SSH config file, located in ~/.ssh/config Host __EGA_OUTBOX_DOMAIN__ EGA-outbox hostname __EGA_OUTBOX_DOMAIN__ User username IdentityFile path/to/the/private/key Replace username and path/to/the/private/key with the appropriate settings, and you will be able to connect to the __EGA_OUTBOX_DOMAIN__ simply using sftp EGA-outbox. How to decrypt Files archived at the EGA are encrypted based on Crypt4GH. Hence, to decrypt the files you need to install Crypt4GH. You can install a python implementation of it, with pip install crypt4gh or directly from the Github repository pip install git+https://github.com/EGA-archive/crypt4gh.git After installing Crypt4GH, decrypt files using the following command: crypt4gh decrypt --sk /path/private/key < encrypted_file.c4gh > decrypted_filename The command reads the encrypted file from stdin (with <) and output the decrypted version to stdout (with >). Replace encrypted_file.c4gh and decrypted_filename with the appropriate filenames but make sure to not use the same filename for both reading and writing because your SHELL would then truncate both files before you even read or write. Frequently Asked Questions What username should I use to log in to my outbox? The authentication process for logging in to the EGA website, as well as accessing your inbox and outbox, requires the use of your username, not your email address. Therefore, if you registered a username different from your email address when creating your EGA account, you must use that username to log in. If you have forgotten your registered username, please, contact our Helpdesk team for assistance. I see that some files in my dataset have 'unavailable' as extension. What should I do? Within your Outbox, you'll find a list of all the datasets available for download. Occasionally, certain files may be marked as "unavailable". These unavailable files can be identified by the "unavailable" extension added to their filenames (e.g. filename.fastq.gz.unavailable.c4gh). If you encounter an unavailable file that you need, please reach out to our Helpdesk. We'll promptly work on making the file accessible for download as soon as possible. Specific to using keys Can I access one EGA account from different devices? Yes, you can access your account from different devices by linking several public keys to your EGA account. Each device can generate a unique public-private key pair, and the corresponding public keys can be linked to the same account. This way, you can use different public keys on different devices and still have access to the same account and data. I have several keys and I don't remember which one is which When generating SSH keys, it's a good practice to add a comment using the -C flag. This will allow you to add a descriptive tag to your key, making it easier to identify later on. Here's an example command that generates an SSH key with a comment: ssh-keygen -t ed25519 -C work-pass In this example, we're generating an ed25519 SSH key with the comment work-pass. Once you have multiple keys with different comments, you can use the comments to easily identify each key. To view the comments for your existing SSH keys, you can use the following command: ssh-keygen -l -f /path/to/key This will display the key fingerprint and the associated comment. By checking the comments, you should be able to identify which key is which. What if I can't find my SSH keys for uploading files with a key file, and how can I use new keys? If you can't find your SSH keys, don't worry - you can make new ones. To do this, open your terminal or command prompt and type a command to make a new SSH key. You can pick a name for the key, and choose a password to keep it safe. After making the key, you can add the new key to your account or server where you want to upload files using the key file. This usually involves copying and pasting the key's "public" (e.g. file.pub) part to the right place. If you lose track of the key again, just make a new one and add it again. Keep in mind that SSH keys belong to you and your computer, so if you switch computers or accounts, you'll need to make new keys. I don't want to type the passphrase every time I use the key. What can I do? You can use an ssh-agent to avoid typing the passphrase every time you use the key. An ssh-agent is a program that stores your private keys in memory and provides them to ssh when needed. You can add your key to the ssh-agent using the command ssh-add followed by the path to your key file.Here's an example of the steps to follow: Open a terminal window.Start the ssh-agent by typing the command eval $(ssh-agent).Add your key to the ssh-agent by typing the command ssh-add [key filepath]. For instance, if your key file is located in the home directory with the name mykey, the command will look like this: ssh-add ~/mykey After adding your, key to the ssh-agent, you should be able to use ssh without having to enter your passphrase every time. Can I use my password for authentication (without my private key)? If you prefer to use your username and password for authentication instead of your private key, you can still do so. When using a Graphical User Interface (GUI) such as FileZilla, you can select Ask for password as your Logon Type (Figure 3). This option will prompt you to enter your password when you click Connect, instead of using your private key. Figure 3: This option will prompt you to enter your password when you click "Connect", instead of using your private key. Figure 3: Process of establishing a new connection to __EGA_OUTBOX_DOMAIN__ using your password as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the inbox, ensuring seamless data transfers. It's worth noting that using a password for authentication can be less secure than using an SSH key, as passwords can be more easily compromised through various means. However, if you choose to use your password for authentication, selecting "Ask for password" as your Logon Type is a good way to do so securely via a GUI. Why is it better to use my key and not my password? SSH keys for authentication is generally considered to be more secure and convenient than using passwords. SSH keys are more difficult to crack than passwords, and they can be restricted to specific users and machines, giving you more control over access. Once you set up your SSH keys, you can use them to authenticate quickly and easily, without having to enter a password every time. This makes automation of tasks, such as uploading encrypted files, much simpler. Additionally, SSH keys provide better logging, allowing you to keep track of who is accessing your systems and when. All in all, using SSH keys is a good practice for improving security and convenience in your authentication process.
WTCCC2 Psychosis Endophenotype samples from UK, Germany, Holland, Spain and Australia using the Affymetrix 6.0 array
Coeliac disease cases and control samples. (1958BC samples excluded)
1958BC control samples
SNP 6.0 arrays of small cell lung cancer
Paroxysmal neurological disorders
A Comprehensive Catalogue of Somatic Mutations from a Human Cancer Genome
UK10K_RARE_FIND REL-2012-01-13
UK10K_RARE_FIND REL-2012-02-22
UK10K_RARE_FIND REL-2012-07-05
UK10K_RARE_FIND REL-2012-11-27
ICGC MMML-seq Data Freeze March 2013 transcriptome sequencing
Genotyping by array and Transcriptome profiling by high-throughput sequencing
This data set includes the following summary level data file used for the exome chip analysis: exome_chip.sv.assoc.txt: results from single variant association analysis in exome chip
Identification of fusion transcripts by RNA-sequencing and Whole genome sequencing of a breast cancer patient sample (METABRIC ID MB-0152)
Whole exome seqeuncing from primary human JMML samples
Intensity files for Immunochip genotypes from blood
Genome-wide DNA methylation profiles of MZ twins clinically discordant for MS generated using Illumina’s Infinium MethylationEPIC BeadChip assay (EPIC array)
Binary Plink files for post-GWAS quality control in 7409 samples genotyped using Axiom 815K Spanish Biobank array (Thermo Fisher)
Freeze 1 of the RP3 project
