Quantifying Model Form Uncertainty of Epidemic Forecasting Models from Incidence Data

Abstract

Objective

We present a mathematical framework for non-parametric estimation of the force of infection, together with statistical upper and lower confidence bands. The resulting estimates allow to assess how well simpler models, such as SEIR, fit the observed time series of incidence data.

Introduction

Uncertainty Quantification (UQ), the ability to quantify the impact of sample-to-sample variations and model misspecification on predictions and forecasts, is a critical aspect of disease surveillance. While quantifying the impact of stochastic uncertainty in the data is well understood, quantifying the impact of model misspecification is significantly harder. For the latter, one needs a "universal model" to which more restrictive parametric models are compared too.

Methods

This talk presents a useful modeling framework for time series of incidence data from contagious diseases that enables one to identify and quantify the impact of model form uncertainty. Specifically, we propose to focus on estimating the time
dependent force of infection. The latter is a universal parameters for all contagious disease model. Using a machine learning technique for estimating monotone functions, i.e., isotonic regression and its variants, one can estimate the force of
infection without addtional assumptions. We note that most contagious disease model do satisfy this monotonicity assumption, due to a combination of factors: depletion of susceptibles, implementation of mitigation strategies, behavior change, etc. Comparing the resulting "non-parametric" estimate with parametric estimates, obtained by fitting an SEIR for example, can reveal model deficiencies and help quantify model form uncertainties.

Finally, we discuss how ideas from "strict bound theory" can be used to develop upper and lower uncertainty bands for force of infection that acknowledge the intrinsic stochasticity in the data.

Results

We demonstrate the application of the methodology to weekly Influenza Like Illness (ILI) incidence data from France and
compare the results to fitted SIR and SEIR models. This comparison can be seen as a nonparametric goodness of fit test, providing one with tools to do simple model selection.

Conclusions

We present a novel and flexible model to statistically describe the force of infection as a function of time. Comparing the fit to incidence data of that model with the fit of simpler parametric models enables the quantification of model form uncertainty and associated model selection.