CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Kamil Khanipov; Georgiy Golovko; Mark Rojas; Levent Albayrak; Otto Dobretsberger; Maria Pimenova; Nels Olson; Sergei Chumakov; Yuriy Fofanov

doi:10.1109/BIBM.2015.7359838

CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Kamil Khanipov, Georgiy Golovko, Mark Rojas, Levent Albayrak, Otto Dobretsberger, Maria Pimenova, Nels Olson, Sergei Chumakov, Yuriy Fofanov

Pharmacology & Toxicology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

Original language	English (US)
Title of host publication	Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015
Editors	lng. Matthieu Schapranow, Jiayu Zhou, Xiaohua Tony Hu, Bin Ma, Sanguthevar Rajasekaran, Satoru Miyano, Illhoi Yoo, Brian Pierce, Amarda Shehu, Vijay K. Gombar, Brian Chen, Vinay Pai, Jun Huan
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1117-1122
Number of pages	6
ISBN (Electronic)	9781467367981
DOIs	https://doi.org/10.1109/BIBM.2015.7359838
State	Published - Dec 16 2015
Event	IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 - Washington, United States Duration: Nov 9 2015 → Nov 12 2015

Publication series

Name	Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

Other

Other	IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015
Country/Territory	United States
City	Washington
Period	11/9/15 → 11/12/15

Keywords

File Formats
HTS Data
HTS File Converter

ASJC Scopus subject areas

Software
Artificial Intelligence
Health Informatics
Biomedical Engineering

Access to Document

10.1109/BIBM.2015.7359838

Cite this

Khanipov, K., Golovko, G., Rojas, M., Albayrak, L., Dobretsberger, O., Pimenova, M., Olson, N., Chumakov, S., & Fofanov, Y. (2015). CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. In L. M. Schapranow, J. Zhou, X. T. Hu, B. Ma, S. Rajasekaran, S. Miyano, I. Yoo, B. Pierce, A. Shehu, V. K. Gombar, B. Chen, V. Pai, & J. Huan (Eds.), Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 (pp. 1117-1122). Article 7359838 (Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIBM.2015.7359838

CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. / Khanipov, Kamil ; Golovko, Georgiy; Rojas, Mark et al.
Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015. ed. / lng. Matthieu Schapranow; Jiayu Zhou; Xiaohua Tony Hu; Bin Ma; Sanguthevar Rajasekaran; Satoru Miyano; Illhoi Yoo; Brian Pierce; Amarda Shehu; Vijay K. Gombar; Brian Chen; Vinay Pai; Jun Huan. Institute of Electrical and Electronics Engineers Inc., 2015. p. 1117-1122 7359838 (Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Khanipov, K , Golovko, G, Rojas, M, Albayrak, L, Dobretsberger, O, Pimenova, M, Olson, N, Chumakov, S & Fofanov, Y 2015, CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. in LM Schapranow, J Zhou, XT Hu, B Ma, S Rajasekaran, S Miyano, I Yoo, B Pierce, A Shehu, VK Gombar, B Chen, V Pai & J Huan (eds), Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015., 7359838, Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015, Institute of Electrical and Electronics Engineers Inc., pp. 1117-1122, IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015, Washington, United States, 11/9/15. https://doi.org/10.1109/BIBM.2015.7359838

Khanipov K , Golovko G, Rojas M, Albayrak L, Dobretsberger O, Pimenova M et al. CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats. In Schapranow LM, Zhou J, Hu XT, Ma B, Rajasekaran S, Miyano S, Yoo I, Pierce B, Shehu A, Gombar VK, Chen B, Pai V, Huan J, editors, Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 1117-1122. 7359838. (Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015). doi: 10.1109/BIBM.2015.7359838

Khanipov, Kamil ; Golovko, Georgiy ; Rojas, Mark et al. / CoCo : An application to store High-Throughput Sequencing data in compact text and binary file formats. Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015. editor / lng. Matthieu Schapranow ; Jiayu Zhou ; Xiaohua Tony Hu ; Bin Ma ; Sanguthevar Rajasekaran ; Satoru Miyano ; Illhoi Yoo ; Brian Pierce ; Amarda Shehu ; Vijay K. Gombar ; Brian Chen ; Vinay Pai ; Jun Huan. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 1117-1122 (Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015).

@inproceedings{0e1ed099f68a43f9b3986cfc4e86e211,

title = "CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats",

abstract = "The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.",

keywords = "File Formats, HTS Data, HTS File Converter",

author = "Kamil Khanipov and Georgiy Golovko and Mark Rojas and Levent Albayrak and Otto Dobretsberger and Maria Pimenova and Nels Olson and Sergei Chumakov and Yuriy Fofanov",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015 ; Conference date: 09-11-2015 Through 12-11-2015",

year = "2015",

month = dec,

day = "16",

doi = "10.1109/BIBM.2015.7359838",

language = "English (US)",

series = "Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1117--1122",

editor = "Schapranow, {lng. Matthieu} and Jiayu Zhou and Hu, {Xiaohua Tony} and Bin Ma and Sanguthevar Rajasekaran and Satoru Miyano and Illhoi Yoo and Brian Pierce and Amarda Shehu and Gombar, {Vijay K.} and Brian Chen and Vinay Pai and Jun Huan",

booktitle = "Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015",

}

TY - GEN

T1 - CoCo

T2 - IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

AU - Khanipov, Kamil

AU - Golovko, Georgiy

AU - Rojas, Mark

AU - Albayrak, Levent

AU - Dobretsberger, Otto

AU - Pimenova, Maria

AU - Olson, Nels

AU - Chumakov, Sergei

AU - Fofanov, Yuriy

PY - 2015/12/16

Y1 - 2015/12/16

N2 - The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

AB - The storage, manipulation, and especially internet transfer of large amounts of data produced by High-Throughput Sequencing (HTS) instruments present major obstacles to utilizing the full potential of this promising technology. The current standard is based on storing all data, which are produced in text (FASTQ and FASTA) and often stored in binary (SRA and BAM) formats. To date, significant effort has been devoted to efficiently compressing these cumbersome sequencing data sets in their existing formats. However, given the substantial improvements in the quality of HTS data, we believe that if one can afford to exclude low quality data and read headers, new much more compressed data formats can be used to reduce the size of HTS data files by at least two orders of magnitude. Here we present several examples of file formats specifically designed to store only high quality sequencing reads in space efficient text and binary form.

KW - File Formats

KW - HTS Data

KW - HTS File Converter

UR - http://www.scopus.com/inward/record.url?scp=84962360836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962360836&partnerID=8YFLogxK

U2 - 10.1109/BIBM.2015.7359838

DO - 10.1109/BIBM.2015.7359838

M3 - Conference contribution

AN - SCOPUS:84962360836

T3 - Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

SP - 1117

EP - 1122

BT - Proceedings - 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015

A2 - Schapranow, lng. Matthieu

A2 - Zhou, Jiayu

A2 - Hu, Xiaohua Tony

A2 - Ma, Bin

A2 - Rajasekaran, Sanguthevar

A2 - Miyano, Satoru

A2 - Yoo, Illhoi

A2 - Pierce, Brian

A2 - Shehu, Amarda

A2 - Gombar, Vijay K.

A2 - Chen, Brian

A2 - Pai, Vinay

A2 - Huan, Jun

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 9 November 2015 through 12 November 2015

ER -

CoCo: An application to store High-Throughput Sequencing data in compact text and binary file formats

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this