Journal Information
Journal ID (publisher-id): OJPHI
ISSN: 1947-2579
Publisher: University of Illinois at Chicago Library
Article Information
©2013 the author(s)
open-access: This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
Electronic publication date: Day: 4 Month: 4 Year: 2013
collection publication date: Year: 2013
Volume: 5E-location ID: e196
Publisher Id: ojphi-05-196

Defining Public Health Situation Awareness – Outcomes and Metrics for Evaluation
Don Olson*
Rob Mathes
Marc Paladini
Kevin Konty
NYC DOHMH, Long Island City, NY, USA
*Don Olson, E-mail:


Review concept of situation awareness (SA) as it relates to public health surveillance, epidemiology and preparedness [1]. Outline hierarchical levels and organizational criteria for SA [2]. Initiate consensus building process aimed at developing a working definition and measurable outcomes and metrics for SA as they relate to syndromic surveillance practice and evaluation.


A decade ago, the primary objective of syndromic surveillance was bioterrorism and outbreak early event detection (EED) [3]. Syndromic systems for EED focused on rapid, automated data collection, processing and statistical anomaly detection of indicators of potential bioterrorism or outbreak events. The paradigm presented a clear and testable surveillance objective: the early detection of outbreaks or events of public health concern. Limited success in practice and limited rigorous evaluation, however, led to the conclusion that syndromic surveillance could not reliably or accurately achieve EED objectives. At the federal level, the primary rationale for syndromic surveillance shifted away from bioterrorism EED, and towards all-hazards biosurveillance and SA [46]. The shift from EED to SA occurred without a clear evaluation of EED objectives, and without a clear definition of the scope or meaning of SA in practice. Since public health SA has not been clearly defined in terms of operational surveillance objectives, statistical or epidemiological methods, or measurable outcomes and metrics, the use of syndromic surveillance to achieve SA cannot be evaluated.


This session is intended to provide a forum to discuss SA in the context of public health disease surveillance practice. The roundtable will focus on defining SA in the context of public health syndromic and epidemiologic surveillance. While SA is often noted in federal level documents as a primary rationale for biosurveillance [1, 46], it is rarely defined or described in operational detail. One working definition presents SA as “real-time analysis and display of health data to monitor the location, magnitude, and spread of an outbreak”, yet it does not elaborate on the methods, systems or evaluation requirements for SA in public health or biosurveillance [3]. In terms of translating SA into public health surveillance practice [1], we will discuss and define the requirements of public health SA based on its development and practice in other areas [2]. The proposed theoretical framework and evaluation criteria adapted and applied to public health SA [2] follow:

  • - Level 1: Perceive relevant surveillance data and epidemiological information.
  • - Level 2: Integrate surveillance and non-surveillance data in conjunction with operator goals to provide understanding of the meaning of the information.
  • - Level 3: Through perceiving (Level 1) and integrating and understanding (Level 2) provide prediction of future events and system states to allow for timely and effective public health decision making.


Sample questions for discussion: What is the relevance of syndromic surveillance and biosurveillance in the SA framework? Where does it fit within the current public health surveillance environment? To achieve the roundtable discussion objectives, the participants will work towards a consensus definition of SA for public health, and will outline measureable outcomes and metrics for evaluation of syndromic surveillance for public health SA.


This work was carried out in conjunction with a grant from the Alfred P. Sloan Foundation (#2010-12-14). We thank the members of the New York City Department of Health and Mental Hygiene Syndromic Surveillance Unit.

1.. Thacker SB, Qualters JR, Lee LM. Public Health Surveillance in the United States: Evolution and ChallengesMMWR 2012;61:3–9.
2.. Endsley MR. Towards a Theory of Situation AwarenessHuman Factors 1995;37:32–64.
3.. Fricker RD. Some Methodological Issues in BiosurveillanceStat Med 2011;30:403–15.
4.. 109th Congress of the United States, Amendment to the Public Service Act. Pandemic and All-Hazards Preparedness Act (2006). Pub L No. 109-417, 101 et seq.
5.. Homeland Security Presidential Directive 21 (HSPD-21), “Public Health and Medical Preparedness,” 18 Oct 2007
6.. White HouseNational Strategy for Biosurveillance. July;2012

Article Categories:
  • ISDS 2012 Conference Abstracts

Keywords: evaluation, biosurveillance, situational awareness, syndromic surveillance, local public health.