EGACryptor

The Ega-Cryptor v.2.0.0 is a JAVA-based application which enables submitters to produce EGA compliant encrypted files along with files for the encrypted and unencrypted md5sum for each file to be submitted. The application will generate an output folder that will by default mirror the directory structure containing the original files. This output folder can subsequently be uploaded to the EGA FTP staging area via an FTP or Aspera client. This mirroring of the directory structure is intended to help with the metadata association within the Submitter Portal.

Download EgaCryptor Using the EgaCryptor Troubleshooting

Download EgaCryptor

The required jar files can be obtained by downloading EgaCrytptor jar file

After the file has been downloaded, extraction of the zipped archive is required.

The new Cryptor has been built to work with Java Runtime Environments from version 6 and above and with the OpenJDK Environment. Please refer to the relevant resources for installation guidance. Installing the latest version of the OpenJDK will include the JCE files. If your installation of Java JRE is less than 1.8.0_151 will require the manual installation of the JCE Policy Files. You can verify the version of Java installed by using the command:

$ java -version

in your terminal, PowerShell or Command Prompt. The output you will need to tale note of will be the build version for Java(TM) SE Runtime Environment.

If you need to install the JCE please follow the instructions below.

Installing the JCE policy files (due to licensing terms and conditions the required policy files must be downloaded direct from the ORACLE website) :

1. Download the unlimited strength JCE policy files (JRE 6 / JRE 7/ JRE 8)
2. Uncompress and extract the downloaded file.
3. This will create a subdirectory called JCE. This directory contains the following files: README.txt, COPYRIGHT.html, local_policy.jar and US_export_policy.jar
4. Install the two policy JAR files by replacing the existing ones in your java home directory.
5. The standard place for JCE jurisdiction policy JAR files is:
6. Install the two policy JAR files by replacing the existing ones in your directory.
7. The standard place for JCE jurisdiction policy JAR files is: /lib/security [Unix] or \lib\security [Win32]

Notes: refers to the directory where the Java SE Runtime Environment (JRE) was installed.

Due to the processes used at the EGA for file archival the use of non-alphanumeric characters in a filename will cause issues in archival. By convention whitespaces in filenames are to be avoided and should be replaced with the underscore character (_).

Additional performance enhancements that have been included in the new version of the EGA-Cryptor V2.0.0:

• the ability to parallelise the processing of datasets through the use of the resources on a system. Multicore systems will allow the user to specify n-1 cores for an n-core system. The use of this feature on clusters may speed up the processing of datasets that have large file numbers but users should consult their local cluster guide to ensure that they are not monopolising resources that are needed by other system users. The default for this process remains single threaded.
• 3 levels of system usage can be specified. Full usage within the limits detailed above. A limited mode that will ensure that 50% of the system resources are available for other tasks. Maximum mode is limited to 75% of system resources, this allows encryption to be prioritised but allows for the system to be usable for light alternate tasks. Finally there is a throttling mode that allows the user to specify the exact number of computational threads are to be used.
• the Cryptor is able to ingest a structured directory and will output the a directory with the same structure containing the encrypted files along with the md5checksums for the plain and encrypted files. The entire output directory can then be uploaded to the EGA for archival.
• as with the input path, it is now possible to specify the output path.
• the options have been updated inline with the upgraded functionality.

Please note that the Graphical User Interface that was present in the previous version of the EgaCryptor has been removed. The tool can only now be used via the command line. The Cryptor is designed to perform a single task, encrypting your data, for upload of these files please refer to our guidance on page: https://ega-archive.org/submission/tools/ftp-aspera

Using the EgaCryptor

Before encrypting your files please make sure that any files that will be uploaded to EGA do not use special characters such as # ? ( ) [ ] / \ = + < > : ; " ' , * ^ | & as the use of these can interfere with the archiving process in addition to causing problems within end users' pipelines.

Below we are presenting 3 possible worked examples on the usage of the EgaCryptor tool

Encrypting a single file :

java -jar ega-cryptor-2.0.0.jar -i example1.bam

Encrypting multiple files :

java -jar ega-cryptor-2.0.0.jar -i "example1.bam,example2.bam"

Encrypting all the files within a folder

java -jar ega-cryptor-2.0.0.jar -i path/to/target/directory

By default the EGA-Cryptor v2 will create a new output directory containing all encrypted files and the relevant checksums within the target directory. If a specific directory is desired this can be specified by using the -o flag. This can be achieved in a similar manner to the following example:

java -jar ega-cryptor-2.0.0.jar -i /path/to/target/directory -o /path/to/output/directory

The tool will output three files per input file:

1. *file.gpg ( encrypted file )
2. *file.md5( file md5 sum value file )
3. *file.gpg.md5 ( encrypted file md5 sum value file )

All output files must then be uploaded to your submission account using Aspera or FTP. To perform such operation please follow the documentation present in the FTP and Aspera pages.

Points to note

1. Remember to provide the path to EgaCryptor.jar and run the command from within the directory the file/s are located.
2. EgaCryptor writes files to the source directory of your local file system, as a result you must have write-access permissions for the source directory.

Troubleshooting

If in doubt about the function of the Cryptor it is recommended to first consult the built-in documentation. This can be accessessed by using the -h flag as stated in the following table.

Built-in Commands

Table: list of the command line options built into EGA-Cryptor v2.0.0.

Command line Option    Action
---------------------  -----------
-f                     Set this option to allow application to create maximum
                         threads to utilize full capacity of cores/processors
                         available on machine
-h                     Use this option to view the bult-in help menu
* -i                   File(s) to encrypt. Provide file/folder path or comma
                         separated file path if multiple files in double quotes
-l                     Set this option to allow application to create maximum
                         threads equals to 50% capacity of cores/processors
                         available on machine
-m                     Set this option to allow application to create maximum
                         threads equals to 75% capacity of cores/processors
                         available on machine
-o                     Path of the output file. This is optional. If not
                         provided then output files will be generated in the
                         same path as that of source file (default: output-
                         files)
-t                     Set this option if user wants to control application to
                         create maximum threads as specified. Application will
                         calculate no. of cores/processors available on machine
                         & will create threads accordingly

Encryption Errors

UnixFileSystem.createFileExclusively (Native Method)

The error is thrown by UNIX (“UnixFileSystem.createFileExclusively(Native Method)”). It appears that the user does not have write-access to the file system where the file to be uploaded is located. EgaCryptor always writes MD5 checksums into files before uploading them to the server, and these files are created in the same location where the uploaded file itself resides.Solution: address your directory permission issue and re-run the command.

Install JCE Unlimited Strength Jurisdiction Policy files

The JCE policy unlimited strength jurisdiction files should be installed according to your current java version

If you are still facing difficulty with the EGA-Cryptor v.2 after having consulted the documentation please contact the EGA Helpdesk.