Development and Validation of a Standardized Tool for Prioritization of Information Sources

How to Cite

Akwar, H., Kloeze, H., & Mukhi, S. N. (2016). Development and Validation of a Standardized Tool for Prioritization of Information Sources. Online Journal of Public Health Informatics, 8(2).


Purpose: To validate the utility and effectiveness of a standardized tool for prioritization of information sources for early detection of diseases.

Methods: The tool was developed with input from diverse public health experts garnered through survey. Ten raters used the tool to evaluate ten information sources and reliability among raters was computed. The Proc mixed procedure with random effect statement and SAS Macros were used to compute multiple raters’ Fleiss Kappa agreement and Kendall's Coefficient of Concordance.  

Results: Ten disparate information sources evaluated obtained the following composite scores: ProMed 91%; WAHID 90%; Eurosurv 87%; MediSys 85%; SciDaily 84%; EurekAl 83%; CSHB 78%; GermTrax 75%; Google 74%; and CBC 70%. A Fleiss Kappa agreement of 50.7% was obtained for ten information sources and 72.5% for a sub-set of five sources rated, which is substantial agreement validating the utility and effectiveness of the tool.  

Conclusion: This study validated the utility and effectiveness of the standardized criteria tool and was used to identify five information sources suited for use by the KIWI system for a pilot project focusing on emerging and zoonotic diseases. The tool can be used in prioritizing a plethora of information sources to improve early detection of diseases.
Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Share-alike: when posting copies or adaptations of the work, release the work under the same license as the original. For any other use of articles, please contact the copyright owner. The journal/publisher is not responsible for subsequent uses of the work, including uses infringing the above license. It is the author's responsibility to bring an infringement action if so desired by the author.