"Recalculating the statistics can't save the current work"

The uncertainty of the data was too great, the procedure unsuitable for statistical evaluation - and led to an error with consequences.

A guest article by ALEXANDER S. KEKULÉ

Kekulé is a doctor and biochemist. He holds the chair of Medical Microbiology and Virology at the Martin Luther University Halle-Wittenberg and is director of the Institute of Medical Microbiology at the University Hospital Halle.

The controversy surrounding the Berlin violologist Christian Drosten and his current study is so explosive because it is not really about Christian Drosten and his study. Instead, the public debate has ignited on two battle lines where powder kegs were buried before.

Firstly, there is the increasingly emotional debate between proponents and opponents of quick relaxation measures. Secondly, there is the question of whether the "Bild" newspaper should be allowed to attack a scientist as sharply as it occasionally does the politicians it advises. A critical look at the controversial study can help to cool both conflicts a little.

In their work, which was published on 29th April as a preprint, Drosten and his team wanted to investigate whether there is a correlation between the virus concentration (virus load) on the pharyngeal mucosa and age. The background is that day-care centres and primary schools are regarded as continuous flow heaters during outbreaks of respiratory diseases.

It is known from influenza viruses and other pathogens that children are often contagious, even if they only show mild symptoms. Moreover, as children do not adhere to distance or hygiene rules, pandemic plans include closing daycare centres and schools as early as possible.

Whether this basic observation also applies to Covid-19 has not yet been proven. This could be contradicted by the fact that children are clearly underrepresented worldwide in the infections that have been proven so far. On the other hand, Covid-19 was first spread in all affected countries by people in middle age, who are particularly socially active.

Since daycare centres and schools were closed almost everywhere immediately after the outbreak became known, children were better protected. In addition, until recently, testing was almost only done for typical Covid-19 symptoms, which are significantly less common in children than in adults.

The question of whether children play a role in the transmission of Covid-19 is also of particular personal importance for Drosten. As an advisor to the federal government, he had initially spoken out against the closure of daycare centres and schools. Later he revised his opinion after reading an older publication on the Spanish flu of 1918.

This change of direction, which earned him considerable criticism (not only from the "Bild" newspaper), was apparently able to provide scientific support for the current study. In the summary, the authors write: "These data prove that the viral load between very young people does not differ significantly from adults. Based on these results, we must warn against unlimited reopening of schools and kindergartens in the current situation".

But at the latest one week after the publication it was clear in the professional world that Drosten had been mistaken. In a meticulous analysis, Leonhard Held, a biostatistician from Zurich, demonstrated several methodological errors that make the proclaimed results of the Charité study untenable. The criticism has now been confirmed by at least three other statisticians.

For their study, Drosten and his co-authors evaluated data from 3712 samples that tested positive for Covid-19 from January to April 26. This indicated that older patients were more likely to have higher virus concentrations in their throat than younger ones - in other words, the opposite of what the authors had hoped for. So far, there has been no significant criticism from the statisticians.

In a further step, however, the Berlin researchers then compared pairs of two age groups with each other. They divided the patients into ten age groups of ten years (0 to 10, 11 to 20 ... 91-100 years). The resulting 44 pairwise comparisons did not show any statistically significant difference in viral load.

From this, Drosten and colleagues concluded that the virus concentration in the throat of children does not differ (significantly) from that of adults, i.e. children can play an important role as virus carriers. This part is flawed in several ways, both methodologically and in terms of the conclusion.

Study has methodological errors

Firstly, the sample quantities taken with swabs are not comparable. The doctor gets more or less mucus and thus different amounts of virus. In children who are less cooperative in such procedures, it is rather less material. In addition, different swabs and even different analysis devices were used.

Secondly, the samples were taken at different times in the course of the disease. Because the virus concentration already decreases significantly a few days after the onset of symptoms, the samples are not comparable for this reason either. In order to statistically compensate for the inevitable, considerable fluctuations in the measured values, the sample would have to be very large. However, of all the relevant under eleven-year-olds, there were only 49 patients.

In addition, the 44 pairwise comparisons of the ten age groups do not provide any meaningful results here. The question of whether there is a difference between the 80-90 year-olds and the 40-50 year-olds is irrelevant. However, the summary consideration of all 44 couples, independent of their absolute age, statistically neutralises the age-dependent effect that was indicated in the preliminary analysis.

The fact that no significant difference was found with this statistical method cannot be taken as evidence that there is no difference between children and adults. In biostatistical introductions there is a mnemonic for this: "The absence of evidence is not an evidence of absence".

In his podcast Christian Drosten defends himself against the criticism and defends his findings, but at the same time admits possible errors of the study. However, the study cannot be compared with an otherwise well-built garden shed, where only the joints were not bricked up as nicely. If, as here, the uncertainties of the data used are too great and an unsuitable method for statistical evaluation was used, the claimed result lacks the scientific basis.

Drosten provides the "picture" with an unnecessary target

Drosten now wants to evaluate further data and recalculate the statistics. But this cannot save the current work. It is difficult to understand why the experienced researcher and political consultant, who is well-versed in dealing with the media, did not simply withdraw the pre-publication and instead give the 'Bild' an unnecessary target for attack.

The hotly disputed question of whether day-care centres and primary schools should now be opened quickly still has no clear scientific answer. Meanwhile, Baden-Württemberg's Minister President Winfried Kretschmann refers to another study that allegedly justifies the immediate opening. The example of the Charité pre-publication (as well as the Heinsberg study before it) shows that rush jobs in the end do neither serve politics nor science.

However, there is a simple, political solution for the schools and daycare centres: the children would have to be examined regularly in groups in pooled tests - at least once before reopening. In such pools, the swabs of up to 30 children can be tested together in the laboratory in just one run. Only if one of the "pools" is positive would all have to be retested individually.

This saves time and money, we have the capacities and the safety of the population is significantly improved. You don't need a scientist as a key witness for this. A little money and the will to take political responsibility is enough.

Translated and reblogged Version - Original here