How the Wehealth App Was Tested
How testing Covid Watch led to the most sophisticated exposure notification app available anywhere– a digital tool capable of fighting the...
5 min read
Joanna Masel : Mar 19, 2021 5:21:50 PM
Q: How should exposure notifications account for new, more infectious strains?
The purpose of exposure notification is to get people who are infected with SARS-CoV-2 to quarantine as soon as possible, before they know for sure that they are infected, and before they infect others. Like traditional contact tracing, many of the people who are asked to quarantine will turn out not to be infected. We believe that it is a matter of ethics to do the best possible job of assessing the risk of being infected, so that we quarantine as many infected people as possible, while avoiding inflicting the harms of quarantine on the uninfected. We put a lot of research into this risk assessment, much of it described in this preprint, now accepted pending minor revisions at the journal Risk Analysis. But the new B.1.1.7 strain is more transmissible than past strains, and so we are now revising the rules we use to decide who we ask to quarantine.
How likely someone is to be infected depends on the dose of virus they were exposed to. An exposure notification app has access to three sources of information about the risk. The most informative is how long the exposure lasted. There is no magic about 15 minutes: a 60 minute exposure is 10 times worse than a 6 minute exposure, and a 10 hour exposure is 10 times worse again. The second most informative factor is timing. People are 10 times more infectious around the time of symptom onset than they are at the margins of their infectious period, so it matters a lot exactly what day you had contact with someone who was infected. Unfortunately, it is difficult (but not impossible) to capture this level of risk information in the Google/Apple system, whose API provides only two levels of infectiousness. After rounding each day’s risk up or down to one of those two levels (or down to zero), we are left with only an average 3-fold difference between the two non-zero levels.
The least quantitative of the three types of information is knowing how strong the BlueTooth signal is. BlueTooth is great for telling whether two people were anywhere near each other at all. But it’s not very good at saying exactly how far apart they are. Worse, BlueTooth signals are sometimes still detected through walls. Even if BlueTooth could measure distance perfectly, because the virus spreads through the air, the relationship between distance and infection risk, while real, isn’t that strong. E.g. in one superspreader event on a bus, there was no relationship between how far someone sat from the infected person and whether they got infected. In our risk analysis, we modeled short-range aerosol transmission, i.e. that two people are near each other, and one breathes in the air that the other person just breathed out. A lot of the infection risk information in distance is based on social science, not physics: people who are interacting face to face tend to position themselves close together, e.g. an average of 0.81m. apart in China. If our model and our calibration experiments are accurate (a big if), then we can capture a 4-fold difference in risk using BlueTooth. But the evidence to support this is much weaker than for our calibrations of how duration and timing of exposure set infection risk.
The easiest way to think about risk thresholds for exposure notifications is in terms of how many minutes of exposure, on a given day and at a given BlueTooth signal strength, are needed to trigger a notification. Below are the settings WeHealth uses before B.1.1.7 hits a community with BlueTooth signal attenuation given in decibels:
You can compare this to CDC guidelines below:
As you can see above, the CDC definition of close contacts excludes those whose only contact was more than two days before symptoms began. This seems to be based on an influential paper from April 2020, in which an error was later found, leading the authors to issue a formal correction in August 2020 acknowledging a greater amount of early transmission. This is science working how it is supposed to, as a self-correcting community of peers. The part that is not working the way it is supposed to is that the WHO and CDC never revised their definitions of who is a close contact in the light of this correction by the scientific community. WeHealth, with the approval of the public health departments it delivers apps for, has chosen to use the best available evidence on the timing of transmission, rather than following outdated CDC guidance. I realize that a change like this creates a lot of work in issuing revised guidance and re-training contact tracers. But it is past time.
Indeed, the rise of the B.1.1.7 variant has made this issue more pressing than ever, because it has an even longer pre-symptomatic infectious period. For this reason, we are revising our risk settings so that the following numbers of minutes are needed to trigger a quarantine recommendation (changes in red):
Many other jurisdictions are using risk settings recommended at the “Risk Scoring Symposium International” (RSSI), which I took part in. This symposium was held under the Chatham House Rule, which means that I am allowed to talk about what was said there, but not who said what. Discussion in the subgroup tasked with recommending which days would be assigned which risk revolved around the question of whether the right thing to do was to follow the science, or whether it was necessary to propagate the recommendations of the CDC even though they were based on a scientific paper for which a correction had been issued. The outcome of that discussion was a compromise between those two positions. This is why WeHealth has not followed the RSSI recommendations — we continue to follow the science, without compromise. We are sharing our settings here in the interests of the public good (although we would appreciate acknowledgement if you adopt them).
The “narrow net” recommendations from RSSI (below), which were the most popular choice at first, are mostly in line with our pre-B.1.1.7 risk settings, except for a smaller window of infectious days. This smaller window of only the most infectious days was chosen as a compromise to ensure that we did not stray too far from the CDC’s recommendation to go back only two days before symptom onset.
The alternative “wide net” settings below, which many jurisdictions are now switching to, have a new problem that I want to bring people’s attention to. The Google/Apple Exposure Notification API scans BlueTooth at intervals that range from once every 2 minutes to once every 5 minutes, depending at least in part on whether the phone is in motion and/or using CPU at the time. This means that a single BlueTooth scan, from a fleeting interaction, might register as 4-5 minutes and be enough to trigger a recommendation for quarantine.
This problem is fixable in a custom app - you simply add an additional logical step not to issue a quarantine recommendation unless there are at least two BlueTooth “scaninstances” logged. WeHealth is in the process of making this change. We want to cast a wider net soon, in part because B.1.1.7 infected individuals have higher viral load and so are more infectious, but mostly because the socially optimal policy has a lower threshold for quarantine when there are fewer cases of a disease, and disease prevalence is falling as vaccinations roll out. But BlueTooth is not a highly reliable technology, and a single scan seems insufficient cause to inflict the harms of quarantine. The unreliability of BlueTooth is also why we don’t extend past 70dB signal attenuation - we want to avoid recommending quarantine to people whose phone picked up a signal on the other side of a wall. If a user knows that on the day in question, they were never in the same airspace as someone else, that kind of incident could undermine support for this technology.
To cast a wider net, one thing we do is have separate messaging for “Low Exposures” that do not meet the threshold for quarantine. Our experience is that users value this early warning signal that the virus is circulating in their vicinity. Hopefully it prompts them to take better precautions prior to getting infected themselves. Below are our thresholds for getting this early warning messaging in a community with B.1.1.7:
I hope this blog post is helpful to others in the process of widening the net cast by their exposure notifications, and trying to follow the latest science while doing so, including the new dangers posed by the B.1.1.7 variant. The virus is evolving, and our science and public health recommendations must too.
Have a question? Ask the professor: contact@wehealth.org
How testing Covid Watch led to the most sophisticated exposure notification app available anywhere– a digital tool capable of fighting the...
Update: The app is still active and continues to deliver pubic health communication and alerts. Apple and Google have decided to turn off the...
Q: What does the success of the NHS app tell us about other EN apps?