Forecasts, scenarios, projections: how do we anticipate the trend of the epidemic

When analyzing the trend of an epidemic, it is important to distinguish between “forecast”, “scenario analysis” and “projection”. In thefocus on the website of the Istituto Superiore di Sanità you can find the respective definitions and explanations, published in collaboration with Fondazione Bruno Kessler.

“We drafted this note”, explains epidemiologist Stefano Merler, who directs the Health Emergencies Center at FBK, “to clarify the use of mathematical models in the study of epidemics, and in COVID-19 in particular. In fact, mathematical models, unlike what is probably common belief, are not only used to make predictions and certainly are not used primarily for making predictions. Especially with COVID-19, whose degree of predictability is extremely low. These are tools mainly used for analyzing the trend of the epidemic, for estimating parameters that cannot be estimated in any other way, for evaluating the effectiveness of possible intervention strategies. All things, together with many others that can be investigated with models, which help us to better understand the epidemic we are facing”

You can find the definitions explained by the Italian Institute of Helath (ISS) and Fondazione Bruno Kessler below.

Forecast:   estimate, with relative uncertainty (forecast range), of the future trend of the epidemic (e.g. in terms of number of cases, hospital admissions and expected deaths over time).

Scenario analysis: estimate, with relative uncertainty, of the future trend of the epidemic, under the condition that the events that define the scenario occur.

Projection: estimate, with relative uncertainty, of the future trend of the epidemic, under the condition that the parameters that regulate the epidemic (e.g. R number) do not change in the future.

Forecast

estimate, with relative uncertainty (forecast range), of the future trend of the epidemic (e.g. in terms of number of cases, hospital admissions and expected deaths over time).

A forecast requires, in addition to knowing the parameters of the natural history of the pathogen, to know in advance and in quantitative terms all the events that may affect the future epidemic dynamics. In addition to many others, social distancing measures that will be adopted (which and when), the vaccination coverage that will gradually be achieved, the individual behavior of citizens, can be of particular relevance. These factors can be more or less decisive depending on the epidemic being analyzed. In fact, the degree of predictability of an epidemic depends a lot on the severity of the infection.

In the case of SARS-CoV-2, for these reasons it is currently extremely complex to make predictions on the progress of the epidemic. Instead, they can be done for viruses whose behavior is known, such as the West Nile virus, transmitted by mosquitoes, of which one can reasonably think that it will have a summer peak, or the flu, which usually has a similar trend all the years (except in the 2020-2021 season, in which social distancing significantly influenced the infection rate curve, and in the 2009-2010 pandemic season).

Scenario analysis

estimate, with relative uncertainty, of the future trend of the epidemic, under the condition that the events that define the scenario occur.

As knowing in advance all the events that may affect the future epidemic dynamics is not possible, we may formulate various hypotheses (which define the scenario) and analyze the corresponding dynamics.

Scenario analises is particularly useful for constructing risk estimates, associated, for example, with the different levels of actions that can be adopted (both in terms of imposing and issuing restrictive measures) or the different levels of vaccination coverage that will be achieved.

For example, in the event that different intervention options are under discussion, scenario analysis enable us to identify those actions with a high chance of keeping the epidemic under control and that enable us to provide estimates of the epidemic risk (e.g. in terms of number of cases, hospital admissions and expected deaths over time) associated with actions that do not guarantee epidemic control.

In general, when retrospectively evaluating the quality of a scenario analysis, we should not mistake it for a forecast. First of all, we need to accurately identify the actions that were actually taken (for example in terms of measures), then verify whether there were any scenarios that hypothesized what actually happened (again in terms of those same actions) and, if so, compare what actually observed with the simulated epidemics in that specific scenario. It is quite clear that there can only be one scenario compatible with what is observed.

Scenario analyses related to SARS-CoV-2 was conducted in April 2020 in anticipation of reopening. On that occasion, the possible impact on some parameters (hospitalizations, deaths) corresponding to different degrees of restrictions lifting was estimated. The scenario to be compared with the epidemic observed is that in which it is assumed: i) not to reopen schools, ii) gradually reopening work activities as of May 4 (almost totally as May 18) while maintaining high levels of agile work, especially in essential services, iii) to maintain restrictions on social gathering contexts. The results of this scenario are extremely consistent with the epidemic observed during the summer of 2020. Obviously, scenarios that included early reopening or more permissive reopenings in terms of social contacts (e.g. total reopening as before the epidemic) calculated an impact on the health system much greater than what was actually observed.

Projection

estimate, with relative uncertainty, of the future trend of the epidemic, under the condition that the parameters that regulate the epidemic (e.g. R number) do not change in the future.

A projection is a particular scenario analysis (status quo scenario) and is a useful risk indicator.

Projections are usually a useful tool in the short run. When the analysis period is particularly short (thus it is not reasonable to expect substantial changes in the parameters in that period), a projection can also be considered as a short-term forecast.

An example of projection is that on hospital beds capacity (in medicine wards and intensive care units) which is usually presented in the weekly monitoring of the epidemic, in which it is assumed that the R number remains constant and that there are no substantial changes in the other parameters. The calculation method, based on the renewal equation and epidemiological estimates on admissions in the medicine wards and intensive care units, is described in [3,4]. These 30-day projections cannot be interpreted as short-term forecasts.

Projections of the SARS-CoV-2 outbreak were also made in summer 2020 to assess the possible course of the outbreak in the fall. The mitigation measuress, and in particular the DPCM of November 3, 2020 which established the yellow, orange and red risk zones, made it possible to greatly limit the impact of the epidemic [4].

Edited by Iss and Fondazione Bruno Kessler (www.fbk.eu)

[1] M. Ajelli, S. Merler, A. Pugliese, C. Rizzo. Model predictions and evaluation of possible control strategies for the 2009 A/H1N1v influenza pandemic in Italy. Epidemiology & Infection, 139: 68 – 79, 2011.

[2] V. Marziano, G. Guzzetta, BM Rondinone, F. Boccuni, F. Riccardo, A. Bella, P. Poletti, F. Trentini, P. Pezzotti, S. Brusaferro, G. Rezza, S. Iavicoli, M. Ajelli, S. Merler. Retrospective analysis of the Italian exit strategy from COVID-19 lockdown. PNAS, 118: e2019617118, 2021.

[3] G. Guzzetta, S. Merler. Stime della trasmissibilità di SARS-CoV-2 in Italia. Disponibile online https://www.epicentro.iss.it/coronavirus/open-data/rt.pdf

[4] M. Manica, G. Guzzetta, F. Riccardo, A. Valenti, P. Poletti, V. Marziano, F. Trentini, .X Andrianou, A. Mateo-Urdiales, M. del Manso, M. Fabiani, M.F. Vescio, M. Spuri, D. Petrone, A. Bella, S. Iavicoli, M. Ajelli, S. Brusaferro, P. Pezzotti, S. Merler. Impact of tiered restrictions on human activities and the epidemiology of the second wave of COVID-19 in Italy. Nature Communications, 12: 4570, 2021.