UK surveillance: Provision of quality assured information from combined datasets

G. A. Paiba, S. R. Roberts, C. W. Houston, E. C. Williams, L. H. Smith, J. C. Gibbens, S. Holdship, R. Lysons

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Surveillance information is most useful when provided within a risk framework, which is achieved by presenting results against an appropriate denominator. Often the datasets are captured separately and for different purposes, and will have inherent errors and biases that can be further confounded by the act of merging. The United Kingdom Rapid Analysis and Detection of Animal-related Risks (RADAR) system contains data from several sources and provides both data extracts for research purposes and reports for wider stakeholders. Considerable efforts are made to optimise the data in RADAR during the Extraction, Transformation and Loading (ETL) process. Despite efforts to ensure data quality, the final dataset inevitably contains some data errors and biases, most of which cannot be rectified during subsequent analysis. So, in order for users to establish the 'fitness for purpose' of data merged from more than one data source, Quality Statements are produced as defined within the overarching surveillance Quality Framework. These documents detail identified data errors and biases following ETL and report construction as well as relevant aspects of the datasets from which the data originated. This paper illustrates these issues using RADAR datasets, and describes how they can be minimised.

Original languageEnglish (US)
Pages (from-to)117-134
Number of pages18
JournalPreventive Veterinary Medicine
Volume81
Issue number1-3 SPEC. ISS.
DOIs
StatePublished - Sep 14 2007
Externally publishedYes

Fingerprint

monitoring
Information Storage and Retrieval
animals
stakeholders
United Kingdom
Datasets
extracts
Data Accuracy

Keywords

  • Data presentation
  • Metadata
  • Quality
  • Reporting
  • Surveillance

ASJC Scopus subject areas

  • Animal Science and Zoology
  • veterinary(all)

Cite this

Paiba, G. A., Roberts, S. R., Houston, C. W., Williams, E. C., Smith, L. H., Gibbens, J. C., ... Lysons, R. (2007). UK surveillance: Provision of quality assured information from combined datasets. Preventive Veterinary Medicine, 81(1-3 SPEC. ISS.), 117-134. https://doi.org/10.1016/j.prevetmed.2007.04.006

UK surveillance : Provision of quality assured information from combined datasets. / Paiba, G. A.; Roberts, S. R.; Houston, C. W.; Williams, E. C.; Smith, L. H.; Gibbens, J. C.; Holdship, S.; Lysons, R.

In: Preventive Veterinary Medicine, Vol. 81, No. 1-3 SPEC. ISS., 14.09.2007, p. 117-134.

Research output: Contribution to journalArticle

Paiba, GA, Roberts, SR, Houston, CW, Williams, EC, Smith, LH, Gibbens, JC, Holdship, S & Lysons, R 2007, 'UK surveillance: Provision of quality assured information from combined datasets', Preventive Veterinary Medicine, vol. 81, no. 1-3 SPEC. ISS., pp. 117-134. https://doi.org/10.1016/j.prevetmed.2007.04.006
Paiba GA, Roberts SR, Houston CW, Williams EC, Smith LH, Gibbens JC et al. UK surveillance: Provision of quality assured information from combined datasets. Preventive Veterinary Medicine. 2007 Sep 14;81(1-3 SPEC. ISS.):117-134. https://doi.org/10.1016/j.prevetmed.2007.04.006
Paiba, G. A. ; Roberts, S. R. ; Houston, C. W. ; Williams, E. C. ; Smith, L. H. ; Gibbens, J. C. ; Holdship, S. ; Lysons, R. / UK surveillance : Provision of quality assured information from combined datasets. In: Preventive Veterinary Medicine. 2007 ; Vol. 81, No. 1-3 SPEC. ISS. pp. 117-134.
@article{18c085de6b0b48fba6c3e33dd33991d5,
title = "UK surveillance: Provision of quality assured information from combined datasets",
abstract = "Surveillance information is most useful when provided within a risk framework, which is achieved by presenting results against an appropriate denominator. Often the datasets are captured separately and for different purposes, and will have inherent errors and biases that can be further confounded by the act of merging. The United Kingdom Rapid Analysis and Detection of Animal-related Risks (RADAR) system contains data from several sources and provides both data extracts for research purposes and reports for wider stakeholders. Considerable efforts are made to optimise the data in RADAR during the Extraction, Transformation and Loading (ETL) process. Despite efforts to ensure data quality, the final dataset inevitably contains some data errors and biases, most of which cannot be rectified during subsequent analysis. So, in order for users to establish the 'fitness for purpose' of data merged from more than one data source, Quality Statements are produced as defined within the overarching surveillance Quality Framework. These documents detail identified data errors and biases following ETL and report construction as well as relevant aspects of the datasets from which the data originated. This paper illustrates these issues using RADAR datasets, and describes how they can be minimised.",
keywords = "Data presentation, Metadata, Quality, Reporting, Surveillance",
author = "Paiba, {G. A.} and Roberts, {S. R.} and Houston, {C. W.} and Williams, {E. C.} and Smith, {L. H.} and Gibbens, {J. C.} and S. Holdship and R. Lysons",
year = "2007",
month = "9",
day = "14",
doi = "10.1016/j.prevetmed.2007.04.006",
language = "English (US)",
volume = "81",
pages = "117--134",
journal = "Preventive Veterinary Medicine",
issn = "0167-5877",
publisher = "Elsevier",
number = "1-3 SPEC. ISS.",

}

TY - JOUR

T1 - UK surveillance

T2 - Provision of quality assured information from combined datasets

AU - Paiba, G. A.

AU - Roberts, S. R.

AU - Houston, C. W.

AU - Williams, E. C.

AU - Smith, L. H.

AU - Gibbens, J. C.

AU - Holdship, S.

AU - Lysons, R.

PY - 2007/9/14

Y1 - 2007/9/14

N2 - Surveillance information is most useful when provided within a risk framework, which is achieved by presenting results against an appropriate denominator. Often the datasets are captured separately and for different purposes, and will have inherent errors and biases that can be further confounded by the act of merging. The United Kingdom Rapid Analysis and Detection of Animal-related Risks (RADAR) system contains data from several sources and provides both data extracts for research purposes and reports for wider stakeholders. Considerable efforts are made to optimise the data in RADAR during the Extraction, Transformation and Loading (ETL) process. Despite efforts to ensure data quality, the final dataset inevitably contains some data errors and biases, most of which cannot be rectified during subsequent analysis. So, in order for users to establish the 'fitness for purpose' of data merged from more than one data source, Quality Statements are produced as defined within the overarching surveillance Quality Framework. These documents detail identified data errors and biases following ETL and report construction as well as relevant aspects of the datasets from which the data originated. This paper illustrates these issues using RADAR datasets, and describes how they can be minimised.

AB - Surveillance information is most useful when provided within a risk framework, which is achieved by presenting results against an appropriate denominator. Often the datasets are captured separately and for different purposes, and will have inherent errors and biases that can be further confounded by the act of merging. The United Kingdom Rapid Analysis and Detection of Animal-related Risks (RADAR) system contains data from several sources and provides both data extracts for research purposes and reports for wider stakeholders. Considerable efforts are made to optimise the data in RADAR during the Extraction, Transformation and Loading (ETL) process. Despite efforts to ensure data quality, the final dataset inevitably contains some data errors and biases, most of which cannot be rectified during subsequent analysis. So, in order for users to establish the 'fitness for purpose' of data merged from more than one data source, Quality Statements are produced as defined within the overarching surveillance Quality Framework. These documents detail identified data errors and biases following ETL and report construction as well as relevant aspects of the datasets from which the data originated. This paper illustrates these issues using RADAR datasets, and describes how they can be minimised.

KW - Data presentation

KW - Metadata

KW - Quality

KW - Reporting

KW - Surveillance

UR - http://www.scopus.com/inward/record.url?scp=34347349116&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34347349116&partnerID=8YFLogxK

U2 - 10.1016/j.prevetmed.2007.04.006

DO - 10.1016/j.prevetmed.2007.04.006

M3 - Article

C2 - 17482298

AN - SCOPUS:34347349116

VL - 81

SP - 117

EP - 134

JO - Preventive Veterinary Medicine

JF - Preventive Veterinary Medicine

SN - 0167-5877

IS - 1-3 SPEC. ISS.

ER -