TY - JOUR AU - Law, Royal K. AU - Burkom, Howard AU - Schier, Josh PY - 2017/05/02 Y2 - 2024/03/28 TI - Evaluation of Exposure-Type Stratification to Improve Poison Center Surveillance JF - Online Journal of Public Health Informatics JA - OJPHI VL - 9 IS - 1 SE - Novel algorithms, statistical or mathematical methods DO - 10.5210/ojphi.v9i1.7595 UR - https://ojphi.org/ojs/index.php/ojphi/article/view/7595 SP - AB - <div style="left: 90px; top: 335.758px; font-size: 14.1667px; font-family: sans-serif; transform: scaleX(1.08129);" data-canvas-width="63.778333333333336">Objective</div><div style="left: 105px; top: 350.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.959577);" data-canvas-width="379.72049999999996">Our objective was to determine if the detection performance of</div><div style="left: 90px; top: 367.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00763);" data-canvas-width="393.8305">current surveillance algorithms to detect call clusters is improved by</div><div style="left: 90px; top: 384.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00579);" data-canvas-width="186.49000000000007">stratifying by exposure category.</div><div style="left: 90px; top: 415.758px; font-size: 14.1667px; font-family: sans-serif; transform: scaleX(1.11768);" data-canvas-width="82.63416666666666">Introduction</div><div style="left: 105px; top: 430.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03611);" data-canvas-width="379.63125">The Centers for Disease Control and Prevention (CDC) uses the</div><div style="left: 90px; top: 447.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.986563);" data-canvas-width="395.71041666666656">National Poison Data System (NPDS) to conduct surveillance of</div><div style="left: 90px; top: 464.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.969809);" data-canvas-width="395.16499999999974">calls to United States poison centers (PCs) to identify clusters of</div><div style="left: 90px; top: 480.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.990863);" data-canvas-width="395.43699999999995">reports of hazardous exposures and illnesses. NPDS stores basic</div><div style="left: 90px; top: 497.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03842);" data-canvas-width="394.82358333333303">information from PC calls including call type (information request</div><div style="left: 90px; top: 514.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00123);" data-canvas-width="393.5655833333334">only or call reporting a possible chemical exposure), exposure agent,</div><div style="left: 90px; top: 530.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00095);" data-canvas-width="246.6841666666667">demographics, clinical, and other variables.</div><div style="left: 105px; top: 547.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.972121);" data-canvas-width="378.04175000000004">CDC looks for anomalies in PC data by using automated algorithms</div><div style="left: 90px; top: 564.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.950093);" data-canvas-width="394.70033333333316">to analyze call and clinical effect volume, and by identifying calls</div><div style="left: 90px; top: 580.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00199);" data-canvas-width="393.5131666666668">reporting exposures to pre-specified high priority agents. Algorithms</div><div style="left: 90px; top: 597.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.99083);" data-canvas-width="393.1448333333333">analyzing call and clinical effect volume identify anomalies when the</div><div style="left: 90px; top: 614.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.98471);" data-canvas-width="393.047083333333">number of calls exceeds a threshold using the historical limits method</div><div style="left: 90px; top: 630.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02909);" data-canvas-width="394.4850000000001">(HLM). Clinical toxicologists and epidemiologists at the American</div><div style="left: 90px; top: 647.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.01614);" data-canvas-width="394.21299999999997">Association of Poison Control Centers and CDC apply standardized</div><div style="left: 90px; top: 664.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02993);" data-canvas-width="394.51616666666627">criteria to determine if the anomaly is a potential incident of public</div><div style="left: 90px; top: 680.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00496);" data-canvas-width="395.9725000000004">health significance (IPHS) and then notify the respective health</div><div style="left: 90px; top: 697.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.981943);" data-canvas-width="392.9734166666667">departments and PCs as needed. Discussions with surveillance system</div><div style="left: 90px; top: 714.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03663);" data-canvas-width="394.54166666666674">users and analysis of past IPHS determined that call volume-based</div><div style="left: 90px; top: 730.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.01791);" data-canvas-width="394.22008333333326">surveillance results in a high proportion of false positive anomalies.</div><div style="left: 90px; top: 747.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.04051);" data-canvas-width="396.98824999999994">A study assessing the positive predictive value (PPV) of this</div><div style="left: 90px; top: 764.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.98679);" data-canvas-width="393.0385833333334">approach determined that fewer than four percent of anomalies over a</div><div style="left: 90px; top: 780.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02503);" data-canvas-width="162.79199999999997">five-year period were IPHS.</div><div style="left: 252.893px; top: 781.084px; font-size: 8.5px; font-family: serif;">1</div><div style="left: 257.155px; top: 780.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03558);" data-canvas-width="227.188">A low PPV can cause an unnecessary</div><div style="left: 90px; top: 797.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.968956);" data-canvas-width="392.57250000000016">waste of staff time and resources. We hypothesized that first stratifying</div><div style="left: 90px; top: 814.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.01552);" data-canvas-width="394.0741666666664">call volume by exposure category would reduce the number of false</div><div style="left: 90px; top: 830.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.993878);" data-canvas-width="395.84499999999986">positives. With the help of medical toxicologists, we created 20</div><div style="left: 90px; top: 847.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00093);" data-canvas-width="380.4883333333332">toxicologically-relevant exposure categories to test this hypothesis.</div><div style="left: 90px; top: 879.091px; font-size: 14.1667px; font-family: sans-serif; transform: scaleX(1.07287);" data-canvas-width="58.23916666666666">Methods</div><div style="left: 105px; top: 894.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.04106);" data-canvas-width="382.17416666666674">To compare cluster detection performance between the two</div><div style="left: 90px; top: 910.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.990084);" data-canvas-width="395.9512499999999">approaches, we used a historical testbed of hourly exposure call</div><div style="left: 90px; top: 927.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.969138);" data-canvas-width="394.8108333333335">counts with and without initial stratification by exposure category</div><div style="left: 90px; top: 944.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.05746);" data-canvas-width="394.7867499999999">from 10 selected PCs from Jan 1, 2006 - Jul 31, 2015. We ran the</div><div style="left: 90px; top: 960.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.961775);" data-canvas-width="394.93691666666655">HLM for both non-stratified and stratified testbeds to estimate the</div><div style="left: 90px; top: 977.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.985723);" data-canvas-width="392.72833333333324">monthly number of anomalies triggered (i.e., alert burden). Our target</div><div style="left: 90px; top: 994.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.977839);" data-canvas-width="392.89550000000014">signals to assess detection performance consisted of call samples from</div><div style="left: 90px; top: 1010.71px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00155);" data-canvas-width="393.5740833333333">three large public health events: the 2009 Salmonella food poisoning</div><div style="left: 90px; top: 1027.38px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.0301);" data-canvas-width="390.22083333333325">event from contaminated peanut butter, the 2012 Hurricane Sandy-</div><div style="left: 90px; top: 1044.04px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00462);" data-canvas-width="393.6250833333333">associated carbon monoxide poisonings in New Jersey, and the 2014</div><div style="left: 90px; top: 1060.71px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00184);" data-canvas-width="396.1425000000002">Elk River contaminated water spill in West Virginia (WV). For</div><div style="left: 90px; top: 1077.38px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02868);" data-canvas-width="394.3291666666664">each event, we chose 30 random calls one thousand times to obtain</div><div style="left: 90px; top: 1094.04px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.985845);" data-canvas-width="393.0399999999997">1000 random sets of inject clusters. Each inject cluster was iteratively</div><div style="left: 90px; top: 1110.71px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.01425);" data-canvas-width="396.5391666666664">added into the testbed with and without initial stratification by</div><div style="left: 90px; top: 1127.38px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.990006);" data-canvas-width="393.01591666666656">exposure category. We then applied the HLM for each iteration to see</div><div style="left: 90px; top: 1144.04px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.0229);" data-canvas-width="394.2073333333332">if the algorithm identified the inject cluster. The sensitivity for each</div><div style="left: 90px; top: 1160.71px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02876);" data-canvas-width="394.35891666666686">approach for each PC was calculated as the proportion of iterations</div><div style="left: 90px; top: 1177.38px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00234);" data-canvas-width="393.3686666666667">where the algorithm identified the inject cluster. We reported median</div><div style="left: 90px; top: 1194.04px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03165);" data-canvas-width="394.383">sensitivities from the ten PCs for each of the time windows of 1, 2,</div><div style="left: 90px; top: 1210.71px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00041);" data-canvas-width="105.05999999999999">4, 8, and 24 hours.</div><div style="left: 510px; top: 335.758px; font-size: 14.1667px; font-family: sans-serif; transform: scaleX(1.08488);" data-canvas-width="51.17">Results</div><div style="left: 525px; top: 350.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00851);" data-canvas-width="381.5536666666666">Figure 1 summarizes results for the WV event with markers</div><div style="left: 510px; top: 367.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03119);" data-canvas-width="394.4977500000001">showing anomaly burden (x-axis) and sensitivity (y-axis) using the</div><div style="left: 510px; top: 384.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.997901);" data-canvas-width="61.22125">stratified (</div><div style="left: 571.21px; top: 386.733px; font-size: 14.1667px; font-family: sans-serif;">Δ</div><div style="left: 579.984px; top: 384.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.961467);" data-canvas-width="324.9918333333333">) and the non-stratified (o) approach by different time</div><div style="left: 510px; top: 400.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.01989);" data-canvas-width="394.1690833333332">windows (hrs). The results from the other two events are not shown</div><div style="left: 510px; top: 417.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.971307);" data-canvas-width="395.40299999999996">but established similar patterns. Anomaly burden is shown as the</div><div style="left: 510px; top: 434.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03128);" data-canvas-width="394.42833333333346">estimated monthly anomaly count for each approach. For example,</div><div style="left: 510px; top: 450.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.03345);" data-canvas-width="394.4495833333334">markers linked by the arrow show that with a 4-hour time window,</div><div style="left: 510px; top: 467.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.04099);" data-canvas-width="394.725833333333">the stratified approach achieves nearly perfect sensitivity with ~10</div><div style="left: 510px; top: 484.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.960052);" data-canvas-width="394.96525">anomalies as the monthly anomaly burden while sensitivity of the</div><div style="left: 510px; top: 500.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02613);" data-canvas-width="394.34333333333353">non-stratified approach is below 20% with ~40 monthly anomalies.</div><div style="left: 510px; top: 517.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.02875);" data-canvas-width="394.3560833333335">The stratified approach gave improved overall sensitivity across all</div><div style="left: 510px; top: 534.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.947831);" data-canvas-width="394.73999999999984">time windows, and reduced anomaly burden for 1-, 2-, and 4-hour</div><div style="left: 510px; top: 550.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00113);" data-canvas-width="83.4275">time windows.</div><div style="left: 510px; top: 582.425px; font-size: 14.1667px; font-family: sans-serif; transform: scaleX(1.10336);" data-canvas-width="85.01416666666667">Conclusions</div><div style="left: 525px; top: 597.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00363);" data-canvas-width="380.9104999999997">We found a consistent detection advantage (higher sensitivity</div><div style="left: 510px; top: 614.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00262);" data-canvas-width="390.4191666666665">and lower anomaly burden) for the stratified vs traditional non-</div><div style="left: 510px; top: 630.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.992097);" data-canvas-width="396.0121666666666">stratified approach for 1-, 2-, and 4-hour time windows. Further</div><div style="left: 510px; top: 647.376px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.986126);" data-canvas-width="395.55174999999997">research should focus on refining the stratified approach and the</div><div style="left: 510px; top: 664.043px; font-size: 14.1667px; font-family: serif; transform: scaleX(0.990867);" data-canvas-width="393.02725000000004">specific surveillance parameters (such as time windows) that increase</div><div style="left: 510px; top: 680.709px; font-size: 14.1667px; font-family: serif; transform: scaleX(1.00071);" data-canvas-width="133.7616666666667">algorithm performance.</div><div style="left: 510px; top: 879.594px; font-size: 12.5px; font-family: serif; transform: scaleX(1.02263);" data-canvas-width="395.82499999999976">Figure 1: Detection performance comparison: stratified vs non-stratified</div><div style="left: 510px; top: 892.928px; font-size: 12.5px; font-family: serif; transform: scaleX(1.00626);" data-canvas-width="386.0499999999998">approach; 2014 Elk River contaminated water spill in West Virginia scenario</div> ER -