Sequence submission

Supported EGA data submission formats are described below. If you have data in any other format or have any questions please contact the EGA helpdesk.

Data files should be de-multiplexed prior to submission so that each run is submitted with files containing data for a single sample only.

Signal data is no longer accepted for Illumina GA/Hiseq and SOLiD platforms but continues to be supported for the 454 platform.  

Generic formats
Format  
CRAM format (all platforms)  
BAM format (all platforms)  
Fastq format (all platforms)  

 

Platform specific formats
Format Recommended  
SFF Format (454 and Ion Torrent) Yes  
SOLiD csfasta/qual format  Yes  
Complete Genomics format Yes  
PacBio HDF format Yes  
Illumina Qseq format

No (please convert to Fastq)

 
Illumina Scarf format  No (please convert to Fastq)  
SRF Format (Illumina)  No  

 

BAM format (all platforms)

The BAM format is our recommended primary sequence data submission format. All submitted BAM files must be readable with SAMtools and Picard.  BAM files must be de-multiplexed prior to submission. However, multiple sample BAMs may be submitted as analysis.

Please note that color space BAM submissions are not supported.

The ArchiveCRAM specification outlines the requirements for BAM and CRAM submissions ... more information.

 

Fastq format (all platforms)

Primary sequence data submissions of single and paired reads are accepted as Fastq files that meet the following the requirements:

  • Quality scores must be in Phred scale. For example, quality scores from early Solexa pipelines must be converted to use this scale. Both ASCII and space delimitered decimal encoding of quality scores are supported. We will automatically detect the Phred quality offset of either 33 or 64.
  • No technical reads (adapters, linkers, barcodes) are allowed.
  • Single reads must be submitted using a single Fastq file and can be submitted with or without read names.
  • Paired reads must split and submitted using either one or two Fastq files. The read names must have a suffix identifying the first and second read from the pair, for example '/1' and '/2' (regular expression for the reads "^(.*)([\\.|:|/|_])([12])$").
  • The first line for each read must start with '@'.
  • The base calls and quality scores must be separated by a line starting with '+'.
  • The Fastq files must be compressed using gzip or bzip2.

Example of Fastq file containing single reads:

@read_name
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
...

Example of Fastq file containing paired reads: 

@read_name/1
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
@read_name/2
GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65
...

where <cycle> indicates the cycle number that starts the second read.

 

SFF format (454 and Ion Torrent)

The SFF format is the recommended primary data submission format for the 454 and Ion Torrent platforms.

 

SOLiD csfasta/qual format

The Csfasta/qual format is supported as a primary data submission format for the SOLiD platform. Please note that paired reads require the title to be identical in both csfasta files in order to associate the reads into pairs. Both csfasta and qual files should be compressed using gzip or bzip2.

 

PacBio HDF5 format

PacBio data submissions are supported in the PacBio HDF5 format. 

 

Complete Genomics format

Complete Genomics data should be submitted as the full Complete Genomics data package containing the ASM, LIB and MAP subfolders. Please contact ega-helpdesk@ebi.ac.uk for further information and guidance on file preparation.

 

Illumina qseq format

We accept but do not recommend primary data submissions in Illumina qseq format. Currently, qseq data submissions are not processed or made available in any other formats. We recommend that submitters convert their data from qseq format to Fastq format prior submission. If submitted, qseq files should be compressed using gzip or bzip2.

 

Illumina scarf format

We accept but do not recommend primary data submissions in Illumina scarf format. Currently, scarf data submissions are not processed or made available in any other formats. We recommend that submitters convert their data from scarf format to Fastq format prior submission. Please note, that scarf format typically uses log-odds qualities that should be converted into Phred qualities when preparing the Fastq files. If submitted, scarf files should be compressed using gzip or bzip2.

 

SRF format (Illumina)

The SRF format continues to be supported as historical primary data submission format for existing submitters only.

 

Preparing SRF files

The *_seq.txt files can be converted into SRF files using the illumina2srf utility available from the DNA Sequence Read Toolkit.

Each Illumina lane should be submitted as a separate SRF file and runs should be demultiplexed prior SRF file generation.

To produce a SRF submission file for a non-paired lane, change the working directory to the run folder and run:

illumina2srf -R -P -N <run>:%l:%t: -n %x:%y -o <center_name>_<run>_<lane>.srf s_<lane>_*_seq.txt

The -R, -P options are used to exclude intensity, noise and signal data from the generated SRF files. These data series are no longer supported for new data submissions.

The recommended format for the SRF file names is <center_name>_<run>_<lane>.srf, where <center_name> is the center name abbreviation assigned to all submitters, and the <run> and <lane> are the run and the lane identifiers.

To produce a SRF submission file for paired lane, change the working directory to the run folder and run:

illumina2srf -R -P -N <run>:%l:%t: -n %x:%y -2 <cycle> -o <center_name>_<run>_<lane>.srf s_<lane>_*_seq.txt

 

Analysis submission formats 

The EGA supports two different types of analyses: reference alignments and sequence variations. Reference alignments are accepted in BAM/CRAM format and sequence variations in VCF format.

If only BAM or CRAM alignment files are submitted but not the original unaligned FASTQ files, then please make sure that the BAM or CRAM files also contain the unaligned reads.  This is critical to enable primary re-analysis and re-alignment of the dataset using new tools or future genome assembilies