TY - JOUR AU - Tseng, Marion AU - Raketich, Natalie AU - Simmons, Cristal PY - 2017/05/02 Y2 - 2024/03/29 TI - Evaluation of chlamydia and gonorrhea electronic provider reports data quality JF - Online Journal of Public Health Informatics JA - OJPHI VL - 9 IS - 1 SE - Data sources, standards, exchange, visualization, and quality DO - 10.5210/ojphi.v9i1.7616 UR - https://ojphi.org/ojs/index.php/ojphi/article/view/7616 SP - AB - <div style="left: 89.1818px; top: 317.842px; font-size: 14.0379px; font-family: sans-serif; transform: scaleX(1.08186);" data-canvas-width="63.1985303030303">Objective</div><div style="left: 104.045px; top: 332.657px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.957033);" data-canvas-width="376.1140787878788">To describe the evaluation process to assess data quality during</div><div style="left: 89.1818px; top: 349.172px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00359);" data-canvas-width="390.04526969696957">development of an electronic case report application, and to describe</div><div style="left: 89.1818px; top: 365.688px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00259);" data-canvas-width="119.30793181818181">the evaluation results</div><div style="left: 89.1818px; top: 397.115px; font-size: 14.0379px; font-family: sans-serif; transform: scaleX(1.11811);" data-canvas-width="81.88294696969696">Introduction</div><div style="left: 104.045px; top: 411.93px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.962057);" data-canvas-width="371.4984242424243">Electronic case reporting (eCR) is defined as the fully or semi-</div><div style="left: 89.1818px; top: 428.445px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.03128);" data-canvas-width="393.22625303030304">automated generation and electronic transmission of reportable</div><div style="left: 89.1818px; top: 444.96px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.02724);" data-canvas-width="390.883331060606">disease case reports from an electronic health record (EHR) system</div><div style="left: 89.1818px; top: 461.475px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.982593);" data-canvas-width="391.62874242424243">to public health authorities, replacing the historically paper-based</div><div style="left: 89.1818px; top: 477.991px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.01293);" data-canvas-width="42.5600409090909">process</div><div style="left: 131.728px; top: 478.362px; font-size: 8.42273px; font-family: serif;">1</div><div style="left: 135.978px; top: 477.991px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.04388);" data-canvas-width="344.15123257575726">. ECR has been reported to increase the number, accuracy,</div><div style="left: 89.1818px; top: 494.506px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.03487);" data-canvas-width="326.86498863636336">completeness and timeliness of surveillance case reports</div><div style="left: 415.971px; top: 494.877px; font-size: 8.42273px; font-family: serif;">2</div><div style="left: 420.23px; top: 494.506px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.04874);" data-canvas-width="59.98806742424242">. Chicago</div><div style="left: 89.1818px; top: 511.021px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.991424);" data-canvas-width="392.0751469696971">Department of Public Health (CDPH) collaborated with Alliance</div><div style="left: 89.1818px; top: 527.536px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.03794);" data-canvas-width="390.9240409090913">of Chicago (AOC) to develop an application to generate electronic</div><div style="left: 89.1818px; top: 544.051px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.995358);" data-canvas-width="389.7490704545454">provider reports (ePR) for chlamydia (CT) and gonorrhea (GC) cases</div><div style="left: 89.1818px; top: 560.566px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00062);" data-canvas-width="389.9245439393939">from the EHR system managed by AOC and send ePR records to the</div><div style="left: 89.1818px; top: 577.082px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.995299);" data-canvas-width="389.59325">Illinois National Electronic Disease Surveillance System (I-NEDSS).</div><div style="left: 89.1818px; top: 593.597px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.97628);" data-canvas-width="389.454275">This application was tested in the EHR database of Health Center A in</div><div style="left: 89.1818px; top: 610.112px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.986205);" data-canvas-width="389.55113636363615">AOC’s network. It is essential to ensure ePR data are accurate, so that</div><div style="left: 89.1818px; top: 626.627px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.02608);" data-canvas-width="390.8496401515148">public health receives correct information to take actions if needed.</div><div style="left: 89.1818px; top: 643.142px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.0044);" data-canvas-width="373.87082575757563">Therefore, evaluation is needed to assess ePR records data quality.</div><div style="left: 89.1818px; top: 674.569px; font-size: 14.0379px; font-family: sans-serif; transform: scaleX(1.07334);" data-canvas-width="57.70971969696969">Methods</div><div style="left: 104.045px; top: 689.385px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00467);" data-canvas-width="377.87162121212106">CDPH developed a five step evaluation plan to validate ePR</div><div style="left: 89.1818px; top: 705.9px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.947829);" data-canvas-width="391.4532689393938">records data quality. Step 1 was to validate the ePR file format to</div><div style="left: 89.1818px; top: 722.415px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.04071);" data-canvas-width="390.8033151515152">ensure all I-NEDSS required fields are present, required value sets</div><div style="left: 89.1818px; top: 738.93px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00926);" data-canvas-width="390.11124772727277">were used, and file format did not vary across files generated. Step 2</div><div style="left: 89.1818px; top: 755.445px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.0149);" data-canvas-width="390.3807749999999">was to validate the algorithm accuracy. Chart review was conducted</div><div style="left: 89.1818px; top: 771.96px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.994348);" data-canvas-width="389.7181871212121">to ensure the ePR records do not include non-reportable cases. Step 3</div><div style="left: 89.1818px; top: 788.476px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.97998);" data-canvas-width="389.42760303030286">was to review ePR records loaded in I-NEDSS to make sure all values</div><div style="left: 89.1818px; top: 804.991px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00373);" data-canvas-width="351.25580303030307">in ePR raw files appeared correctly on the I-NEDSS front end.</div><div style="left: 104.045px; top: 821.506px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.976159);" data-canvas-width="374.56850833333317">After the application passed steps 1 to 3, it moved to step 4, parallel</div><div style="left: 89.1818px; top: 838.021px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.983302);" data-canvas-width="389.5188492424242">validation. The first phase of parallel validation was to review historic</div><div style="left: 89.1818px; top: 854.536px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.04375);" data-canvas-width="391.0391515151514">cases. Test ePR records for CT and GC cases diagnosed by Health</div><div style="left: 89.1818px; top: 871.051px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.953863);" data-canvas-width="391.62453106060593">Center A in 2015 (n=510) were compared to the same 510 cases’</div><div style="left: 89.1818px; top: 887.566px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.961365);" data-canvas-width="391.0672272727272">closed surveillance case reports in I-NEDSS. The completeness of</div><div style="left: 89.1818px; top: 904.082px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.970115);" data-canvas-width="391.48976742424253">treatment, race, and ethnicity was examined. The application then</div><div style="left: 89.1818px; top: 920.597px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.991373);" data-canvas-width="389.59324999999984">moved into testing daily data feed. Daily ePR records were compared</div><div style="left: 89.1818px; top: 937.112px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.997821);" data-canvas-width="392.45978484848507">with EHR charts and paper provider reports received by CDPH</div><div style="left: 89.1818px; top: 953.627px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.984762);" data-canvas-width="392.1158568181817">to assess completeness and timeliness. Step 5 was to re-evaluate</div><div style="left: 89.1818px; top: 970.142px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.04401);" data-canvas-width="393.69792575757486">algorithms. EPR records were validated against the electronic</div><div style="left: 89.1818px; top: 986.657px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00317);" data-canvas-width="390.01438636363633">laboratory reports (ELR) records, which were used as gold standards</div><div style="left: 89.1818px; top: 1003.17px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00418);" data-canvas-width="318.926568181818">of all reportable CT and GC cases, to find missing cases.</div><div style="left: 89.1818px; top: 1034.6px; font-size: 14.0379px; font-family: sans-serif; transform: scaleX(1.08498);" data-canvas-width="50.70481818181818">Results</div><div style="left: 104.045px; top: 1049.41px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.01927);" data-canvas-width="375.6508287878788">The first three steps of evaluation occurred from January to April</div><div style="left: 89.1818px; top: 1065.93px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.967447);" data-canvas-width="391.1388204545452">2016. Test ePR files containing historic cases from Health Center</div><div style="left: 89.1818px; top: 1082.45px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.942297);" data-canvas-width="391.05318939393925">A were vetted weekly. A total of 14 test ePR files were reviewed.</div><div style="left: 89.1818px; top: 1098.96px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.0415);" data-canvas-width="390.7022424242424">This process identified required fields not present (patient address,</div><div style="left: 89.1818px; top: 1115.48px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.987091);" data-canvas-width="392.2042954545451">treatment date, treatment, and race), race value sets not returned</div><div style="left: 89.1818px; top: 1131.99px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.974732);" data-canvas-width="391.6961242424244">correctly, and additional logic statements needed to return correct</div><div style="left: 89.1818px; top: 1148.51px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.981283);" data-canvas-width="389.4711204545455">pregnancy status at the time of diagnosis. These issues were discussed</div><div style="left: 89.1818px; top: 1165.02px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00655);" data-canvas-width="385.5924545454545">with the project team, and the application was modified accordingly.</div><div style="left: 104.045px; top: 1181.54px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.998597);" data-canvas-width="375.00649015151515">The historic case review found ePR data were more complete than</div><div style="left: 89.1818px; top: 1198.05px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.01299);" data-canvas-width="390.473425">closed surveillance reports. Compared to closed surveillance reports</div><div style="left: 505.364px; top: 316.142px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.964121);" data-canvas-width="391.51363181818186">in I-NEDSS, 18% (94/510) of the cases had incomplete treatment</div><div style="left: 505.364px; top: 332.657px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.956139);" data-canvas-width="390.9015803030303">information in the ePR records compared to 78% (400/510), 0.2%</div><div style="left: 505.364px; top: 349.172px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00095);" data-canvas-width="389.9526196969697">(1/510) of the cases did not have race information in the ePR records</div><div style="left: 505.364px; top: 365.688px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.03902);" data-canvas-width="390.75839393939407">compared to 47% (240/510), and 0.7% (4/510) of the cases had no</div><div style="left: 505.364px; top: 382.203px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.990043);" data-canvas-width="389.5160416666667">ethnicity information in the ePR records compared to 50% (253/510).</div><div style="left: 505.364px; top: 398.718px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00615);" data-canvas-width="392.3474818181821">These preliminary evaluation results suggest that eCR improves</div><div style="left: 505.364px; top: 415.233px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.989618);" data-canvas-width="389.5300795454544">surveillance case reports data quality. The daily data feed data quality</div><div style="left: 505.364px; top: 431.748px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.02938);" data-canvas-width="390.9759810606059">evaluation is still on-going, and ePR data quality will be monitored</div><div style="left: 505.364px; top: 448.263px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.01481);" data-canvas-width="76.04318939393939">continuously.</div><div style="left: 505.364px; top: 479.691px; font-size: 14.0379px; font-family: sans-serif; transform: scaleX(1.10288);" data-canvas-width="84.24131060606061">Conclusions</div><div style="left: 520.227px; top: 494.506px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.01289);" data-canvas-width="375.58063939393935">Evaluation plays an integral role in developing and implementing</div><div style="left: 505.364px; top: 511.021px; font-size: 14.0379px; font-family: serif; transform: scaleX(0.99219);" data-canvas-width="389.58342348484837">the eCR process in Chicago. The stepwise evaluation process ensures</div><div style="left: 505.364px; top: 527.536px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.02769);" data-canvas-width="390.7275106060607">ePR data quality meeting public health requirements, so that public</div><div style="left: 505.364px; top: 544.051px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.02909);" data-canvas-width="390.8482363636363">health will be able to act on more complete information to improve</div><div style="left: 505.364px; top: 560.566px; font-size: 14.0379px; font-family: serif; transform: scaleX(1.00229);" data-canvas-width="101.38156060606062">population health.</div> ER -