CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Kamil Khanipov; Georgiy Golovko; Mark Rojas; Levent Albayrak; Otto Dobretsberger; Maria Pimenova; Nels Olson; Sergei Chumakov; Yuriy Fofanov

doi:10.1109/BIBM.2015.7359838

CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Kamil Khanipov, Georgiy Golovko, Mark Rojas, Levent Albayrak, Otto Dobretsberger, Maria Pimenova, Nels Olson, Sergei Chumakov, Yuriy Fofanov

Pharmacology & Toxicology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

Original language	English (US)
Title of host publication	Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1117-1122
Number of pages	6
ISBN (Print)	9781467367981
DOIs	https://doi.org/10.1109/BIBM.2015.7359838
State	Published - Dec 16 2015
Event	IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 - Washington, United States Duration: Nov 9 2015 → Nov 12 2015

Other

Other	IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015
Country/Territory	United States
City	Washington
Period	11/9/15 → 11/12/15

Keywords

File Formats
HTS Data
HTS File Converter

ASJC Scopus subject areas

Software
Artificial Intelligence
Health Informatics
Biomedical Engineering

Access to Document

10.1109/BIBM.2015.7359838

Cite this

Khanipov, K., Golovko, G., Rojas, M., Albayrak, L., Dobretsberger, O., Pimenova, M., Olson, N., Chumakov, S., & Fofanov, Y. (2015). CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. In Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 (pp. 1117-1122). [7359838] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIBM.2015.7359838

CoCo : An application to store High-Throughput Sequencing data in compact text and binary file formats. / Khanipov, Kamil ; Golovko, Georgiy; Rojas, Mark et al.

Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 1117-1122 7359838.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Khanipov, K , Golovko, G, Rojas, M, Albayrak, L, Dobretsberger, O, Pimenova, M, Olson, N, Chumakov, S & Fofanov, Y 2015, CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. in Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015., 7359838, Institute of Electrical and Electronics Engineers Inc., pp. 1117-1122, IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015, Washington, United States, 11/9/15. https://doi.org/10.1109/BIBM.2015.7359838

Khanipov K , Golovko G, Rojas M, Albayrak L, Dobretsberger O, Pimenova M et al. CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. In Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 1117-1122. 7359838 doi: 10.1109/BIBM.2015.7359838

@inproceedings{0e1ed099f68a43f9b3986cfc4e86e211,

title = "CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats",

abstract = "The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.",

keywords = "File Formats, HTS Data, HTS File Converter",

author = "Kamil Khanipov and Georgiy Golovko and Mark Rojas and Levent Albayrak and Otto Dobretsberger and Maria Pimenova and Nels Olson and Sergei Chumakov and Yuriy Fofanov",

year = "2015",

month = dec,

day = "16",

doi = "10.1109/BIBM.2015.7359838",

language = "English (US)",

isbn = "9781467367981",

pages = "1117--1122",

booktitle = "Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

note = "IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 ; Conference date: 09-11-2015 Through 12-11-2015",

}

TY - GEN

T1 - CoCo

T2 - IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

AU - Khanipov, Kamil

AU - Golovko, Georgiy

AU - Rojas, Mark

AU - Albayrak, Levent

AU - Dobretsberger, Otto

AU - Pimenova, Maria

AU - Olson, Nels

AU - Chumakov, Sergei

AU - Fofanov, Yuriy

PY - 2015/12/16

Y1 - 2015/12/16

N2 - The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

AB - The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

KW - File Formats

KW - HTS Data

KW - HTS File Converter

UR - http://www.scopus.com/inward/record.url?scp=84962360836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962360836&partnerID=8YFLogxK

U2 - 10.1109/BIBM.2015.7359838

DO - 10.1109/BIBM.2015.7359838

M3 - Conference contribution

AN - SCOPUS:84962360836

SN - 9781467367981

SP - 1117

EP - 1122

BT - Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 9 November 2015 through 12 November 2015

ER -

CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this