Deep learning-based NLP data pipeline for EHR-scanned document information extraction

Enshuo Hsu, Ioannis Malagaris, Yong Fang Kuo, Rizwana Sultana, Kirk Roberts

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Objective: Scanned documents in electronic health records (EHR) have been a challenge for decades, and are expected to stay in the foreseeable future. Current approaches for processing include image preprocessing, optical character recognition (OCR), and natural language processing (NLP). However, there is limited work evaluating the interaction of image preprocessing methods, NLP models, and document layout. Materials and Methods: We evaluated 2 key indicators for sleep apnea, Apnea hypopnea index (AHI) and oxygen saturation (SaO2), from 955 scanned sleep study reports. Image preprocessing methods include gray-scaling, dilating, eroding, and contrast. OCR was implemented with Tesseract. Seven traditional machine learning models and 3 deep learning models were evaluated. We also evaluated combinations of image preprocessing methods, and 2 deep learning architectures (with and without structured input providing document layout information), with the goal of optimizing end-to-end performance. Results: Our proposed method using ClinicalBERT reached an AUROC of 0.9743 and document accuracy of 94.76% for AHI, and an AUROC of 0.9523 and document accuracy of 91.61% for SaO2. Discussion: There are multiple, inter-related steps to extract meaningful information from scanned reports. While it would be infeasible to experiment with all possible option combinations, we experimented with several of the most critical steps for information extraction, including image processing and NLP. Given that scanned documents will likely be part of healthcare for years to come, it is critical to develop NLP systems to extract key information from this data. Conclusion: We demonstrated the proper use of image preprocessing and document layout could be beneficial to scanned document processing.

Original languageEnglish (US)
Article numberooac045
JournalJAMIA Open
Issue number2
StatePublished - Jul 1 2022


  • electronic health records
  • natural language processing
  • optical character recognition
  • polysomnography
  • scanned document

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'Deep learning-based NLP data pipeline for EHR-scanned document information extraction'. Together they form a unique fingerprint.

Cite this