File preparation
Due to the processes used at the EGA for file archival the use of non-alphanumeric characters in a filename will cause issues in archival. By convention whitespaces in filenames are to be avoided and should be replaced with the underscore character (_). Before encrypting your files please make sure that any files that will be uploaded to EGA do not use special characters such as # ? ( ) [ ] / \ = + < > : ; " ' , * ^ | &
Files encrypted with EGACryptor must be uploaded via FTP
EGACryptor
The EGACryptor v.2.0.0 is a JAVA-based application which enables submitters to produce EGA compliant encrypted files along with files for the encrypted and unencrypted md5sum for each file to be submitted. The application will generate an output folder that will by default mirror the directory structure containing the original files. This output folder can subsequently be uploaded to the EGA FTP staging area via an FTP or Aspera client.
Download EgaCryptor
Using the EgaCryptor
Troubleshooting
Download EGACryptor
The required jar files can be obtained by downloading EgaCrytptor jar file
After the file has been downloaded, extraction of the zipped archive is required. The EGACryptor has been built to work with Java Runtime Environments from version 6 and above and with the OpenJDK Environment. Please refer to the relevant resources for installation guidance. Installing the latest version of the OpenJDK will include the JCE files. If your installation of Java JRE is less than 1.8.0_151 will require the manual installation of the JCE Policy Files. You can verify the version of the Java SE Runtime Environment (JRE) installed by using the command:
$ java -version
If you need to install the JCE please follow the instructions below:
Installing the JCE policy files (due to licensing terms and conditions the required policy files must be downloaded direct from the ORACLE website) :
- Download the unlimited strength JCE policy files (JRE 6 / JRE 7/ JRE 8)
- Uncompress and extract the downloaded file.
- This will create a subdirectory called JCE. This directory contains the following files: README.txt, COPYRIGHT.html, local_policy.jar and US_export_policy.jar
- Install the two policy JAR files by replacing the existing ones in your java home directory.
- Install the two policy JAR files by replacing the existing ones in your directory.
- The standard place for JCE jurisdiction policy JAR files is: /lib/security [Unix] or \lib\security [Win32]
Notes: refers to the directory where the Java SE Runtime Environment (JRE) was installed.
Additional performance enhancements that have been included in the EGACryptor V2.0.0:
- The ability to parallelise the processing of datasets through the use of the resources on a system. Multicore systems will allow the user to specify n-1 cores for an n-core system. The use of this feature on clusters may speed up the processing of datasets that have large file numbers but consult your local cluster guide to ensure that there are not monopolising resources that are needed by other system users. The default for this process remains single threaded.
- 3 levels of system usage can be specified. Full usage within the limits detailed above. A limited mode that will ensure that 50% of the system resources are available for other tasks. Maximum mode is limited to 75% of system resources, this allows encryption to be prioritised but allows for the system to be usable for light alternate tasks. Finally there is a throttling mode that allows you to specify the exact number of computational threads to be used.
- the EGACryptor is able to ingest a structured directory and will output a directory with the same structure containing the encrypted files along with the md5checksums for the plain and encrypted files. The entire output directory can then be uploaded to the EGA for archival.
- as with the input path, it is now possible to specify the output path.
- the options have been updated inline with the upgraded functionality.
The tool can only be used via the command line. The EGACryptor is designed to perform a single task, encrypting your data, for upload of these files please refer to our uploading guide
Using the EgaCryptor
Below are the three ways on how the EGACryptor tool can be used:
java -jar ../EGA-Cryptor_2_0_0.jar -i example1.bam
java -jar ../EGA-Cryptor_2_0_0.jar -i "example1.bam,example2.bam"
Encrypting all the files within a folder
java -jar ../EGA-Cryptor_2_0_0.jar -i path/to/target/directory
By default the EGACryptor v2 will create a new output directory containing all encrypted files and the relevant checksums within the target directory. If a specific directory is desired this can be specified by using the -o flag. This can be achieved in a similar manner to the following example:
java -jar ../EGA-Cryptor_2_0_0.jar -i /path/to/target/directory -o
/path/to/output/directory
The tool will output three files per input file:
- file.gpg ( encrypted file )
- file.md5( file md5 sum value file )
- file.gpg.md5 ( encrypted file md5 sum value file )
All output files must then be uploaded to your submission account using Aspera or FTP. Further documentation on how to upload files: FTP and Aspera.
- Remember to provide the path to EgaCryptor.jar and run the command from within the directory the file/s are located.
- ECryptor writes files to the source directory of your local file system, as a result you must have write-access permissions for the source directory.
Troubleshooting
If in doubt about the function of the EGACryptor it is recommended to first consult the built-in documentation. This can be accessessed by using the -h flag as stated in the following table.
Built-in Commands
Table: list of the command line options built into EGA-Cryptor v2.0.0.
Command line Option Action --------------------- ----------- -f Set this option to allow application to create maximum threads to utilise full capacity of cores/processors available on machine -h Use this option to view the bult-in help menu * -i File(s) to encrypt. Provide file/folder path or comma separated file path if multiple files in double quotes -l Set this option to allow application to create maximum threads equals to 50% capacity of cores/processors available on machine -m Set this option to allow application to create maximum threads equals to 75% capacity of cores/processors available on machine -o Path of the output file. This is optional. If not provided then output files will be generated in the same path as that of source file (default: output- files) -t Set this option if user wants to control application to create maximum threads as specified. Application will calculate no. of cores/processors available on machine & will create threads accordingly
Encryption Errors
UnixFileSystem.createFileExclusively (Native Method)
The error is thrown by UNIX ("UnixFileSystem.createFileExclusively(Native Method)"). It appears that the user does not have write-access to the file system where the file to be uploaded is located. EGACryptor always writes MD5 checksums into files before uploading them to the server, and these files are created in the same location where the uploaded file itself resides.Solution: address your directory permission issue and re-run the command.
Install JCE Unlimited Strength Jurisdiction Policy files
The JCE policy unlimited strength jurisdiction files should be installed according to your current java version
If you are still facing difficulty with the EGACryptor v.2 after having consulted the documentation please contact the EGA Helpdesk.