a Twitter thread from @manlius84
@TwitterVid_bot1.
I'd like to comment about the impossibility to forecast an epidemics, at least in the mid-to-long terms.
Let's stress that models should not be used to forecast, but to test what-if scenarios, like
'what could happen if I make a travel ban?Assuming that my model is correct' 1/
2.
Models depend on parameters that are unknown and must be fitted by statistical procedures.
Even if you do it in the best way, our inability to collect perfect data leads to missing/incomplete information and, thus, to instability in the parameters, needing recalibration. 2/
3.
The system is stochastic: random mutations of the virus lead to emergent variants, ie a new recalibration of the models, assuming no changes in the assumed biology/epidemiology, which can only be understood after lot of data analysis (ie, too late) at the geno/phenotype level 3/
4.
Don't forget that an epidemic is a collective phenomenon, ie it depends on us and the environment. Behavioral shifts (positive or negative ones) can't be simply predicted in the mid-term, often not even in the short-term.
But.. the worst is described in the next tweets. 4/
5.
Assume that we know *everything* and are able to measure *everything* about this virus: its epi parameters, its clinical effects, etc, with 100% accuracy. Keep only a very small uncertainty on the initial conditions (eg, the exact date of the case zero). What'd happen? 5/
6.
As other collective phenomena, epidemics amplify fluctuations in an exponential way (also related to the infamous exponential growth that anyone is trying to measure in each country with doubling times, etc). When its rate is positive, chaos emerges, forecast has a horizon. 6/
7.
It's not our fault, it's just how nature works. Above this temporal threshold, known as Lyapunov horizon, we can't say anything about the behavior of the system. It's the same reason behind the mid-term unpredictability of weather, the math works in the same way.
7/
8.
In a recent paper by @omeuxeito & co, it has been shown how the turning point and end of an expanding epidemic cannot be precisely forecast, even if your model is simple, parameter-free or whatever: we are limited in the predictions we can do. 8/
https://www.pnas.org/content/117/42/26190
9.
The main message here is that instead of forecasting, our best thing to do is to estimate the probability of a specific scenario to happen under the what-if-then paradigm. 9/
10.
We have a work to be submitted soon, and this is a plot from there. Using forecasts for the US n-weeks-ahead from tens of the most brilliant epi labs in the world, it's clear that the horizon is no more than 2-3 weeks (red=data, cyan=forecasts).
What does it mean for policy?
10/
11.
It means that we have to consider the effects of our actions in the mid-to-long-term, but we can only act and monitor our response in the short-term.
Small errors today can have huge consequences in 2-3 weeks from now, that would be out of our control. Let's not give up.
end/
12.
Supplementary info 1:
A fit It’s the statistical procedure used to find the best values of a model’s parameters when one collects data consisting of an independent variable (eg, time) and a dependent one (eg, covid19 cases in one country).
More: https://en.wikipedia.org/wiki/Curve_fitting
➖➖➖➖➖➖➖➖➖➖➖
Manlio De Domenico (@manlius84)
Models depend on parameters that are unknown and must be fitted by statistical procedures.
Even if you do it in the best way, our inability to collect perfect data leads to missing/incomplete information and, thus, to instability in the parameters, needing recalibration. 2/
➖➖➖➖➖➖➖➖➖➖➖
13.
Supplement 2:
Modeling in epidemiology is usually more difficult than just fitting a function. For the ones who want to start reading about epi, I recommend this recent paper about the SIR approach.
https://www.nature.com/articles/s41592-020-0822-z
14.
There is a follow up too, discussing the more sophisticated SEIR model and it’s variants.
Here you can find SEIRS, used when there is loss of immunity (as for covid19 after 5-6 months), and epidemic waves:
https://www.nature.com/articles/s41592-020-0856-2#change-history
15.
If you want to play with epidemics, this online interactive tool is for you:
https://shiny.bcgsc.ca/posepi2/
You can change the value of parameters and see the epi waves, the curves of cases, etc.
16.
Supplement 3:
I cited a paper about covid19, but @svscarpino & @lordgrilo have recently shown a similar issue for a variety of diseases.
The study is from 2019: https://www.nature.com/articles/s41467-019-08616-0
It’s relevant because they show predictability limits regardless of the fitting model.
➖➖➖➖➖➖➖➖➖➖➖
Manlio De Domenico (@manlius84)
In a recent paper by @omeuxeito & co, it has been shown how the turning point and end of an expanding epidemic cannot be precisely forecast, even if your model is simple, parameter-free or whatever: we are limited in the predictions we can do. 8/
https://www.pnas.org/content/117/42/26190
➖➖➖➖➖➖➖➖➖➖➖
17.
Supplement 4:
Let me point to the wiki page about the Lyapunov exponent, responsible for the exponential divergence I mention: https://en.wikipedia.org/wiki/Lyapunov_exponent
It is a smart way to study, quantitatively, how fast uncertainty will propagate in a dynamical system (like epi models)
➖➖➖➖➖➖➖➖➖➖➖
Manlio De Domenico (@manlius84)
As other collective phenomena, epidemics amplify fluctuations in an exponential way (also related to the infamous exponential growth that anyone is trying to measure in each country with doubling times, etc). When its rate is positive, chaos emerges, forecast has a horizon. 6/
➖➖➖➖➖➖➖➖➖➖➖
18.
More generally, this is known as sensitivity to initial conditions and falls within the study of chaos theory: https://en.wikipedia.org/wiki/Chaos_theory#Sensitivity_to_initial_conditions
This is one of the most beautiful and interesting topics I have ever studied. It’s crucial to understand the basics of complexity science.
19.
Let’s assume a double pendulum (ie a pendulum attached to another pendulum). Its movements are described by differential equations (like epi models).
Simulate it, starting from 50 slightly different initial conditions and you’ll see divergence in action:
20.
Want still to know more? I have learned about chaos from this fantastic book: https://www.amazon.com/Chaos-Theory-Steven-H-Strogatz-Physics/s?rh=n%3A13567%2Cp_lbr_one_browse-bin%3ASteven+H.+Strogatz
It’s by the great @stevenstrogatz and I am sure there are also his own lectures about this on YouTube. I could not think of a better place to start learning about chaos theory.
21.
Supplement 5:
🦠 variants are not detected as they appear. We have to find them, first.
It happens when we do a lot of testing+sequencing, and only once we collect enough samples to assess the presence of new relevant mutations.
That is, when it’s too late, usually.
➖➖➖➖➖➖➖➖➖➖➖
Manlio De Domenico (@manlius84)
The system is stochastic: random mutations of the virus lead to emergent variants, ie a new recalibration of the models, assuming no changes in the assumed biology/epidemiology, which can only be understood after lot of data analysis (ie, too late) at the geno/phenotype level 3/
➖➖➖➖➖➖➖➖➖➖➖
Read this thread on Twitter
Made by @TwitterVid_bot