MIT Lincoln Laboratory: EN in Simulated Real-World Environments

Wehealth has worked closely with our academic partners and collaborators to evaluate and improve the technology we use and the solutions we build. MIT Lincoln Laboratory just released a video highlighting the CDC funded work they did to evaluate COVID-19 exposure notifications using our University of Arizona app in simulated real-world environments.

Source: Private Automated Contact Tracing (PACT), MIT Lincoln Laboratory, 2023. 

Abstract from “COVID-19    Exposure Notification in    Simulated    Real-World Environments.” Schiefelbein, Curran, Gervin, Germain, and Mazzola.  Project Report, April 15, 2022.

Privacy-preserving contact tracing mobile applications, such as those that use the Google-Apple
Exposure Notification (GAEN) service, have the potential to limit the spread of COVID-19 in communities, but the privacy-preserving aspects of the protocol make it difficult to assess the performance of the apps in real-world populations. To address this gap, we exercised the CovidWatch app on both Android and iOS phones in a variety of scripted real-world scenarios, relevant to the lives of university students and employees. We collected exposure data from the app and from the lower-level Android service, and compared it to the phones’ actual distances and durations of exposure, to assess the sensitivity and specificity of the GAEN service configuration as of February 2021. Based on the app’s reported ExposureWindows and alerting thresholds for Low and High alerts, our assessment is that the chosen configuration is highly sensitive under a range of realistic scenarios and conditions. With this configuration, the app is likely to capture many long-duration encounters, even at distances greater than six feet, which may be desirable under conditions with increased risk of airborne transmission.

Excerpt from “Automated Contact Tracing Assessment” (Schiefelbein, Curran, Charlie Ishikawa, Emily Shen, Raphael Yahalom, and Adam Norige.  Technical Report. MIT Lincoln Laboratory, November 17, 2022.)

MIT LL conducted data collection, in partnership with the University of Arizona’s Campus
Health Service, in February-March 2021. The University had deployed an Android- and iOS-based EN app to the campus population, and had conducted preliminary benchtop testing for app functionality. They wished to obtain an independent assessment of the app’s performance in more realistic situations, using the same configuration of attenuation weights and thresholds, and two levels of risk scores, already in use on campus. CDC funded MIT LL to conduct the data collection and analysis with the objective of enriching the BLE exposure dataset, using various models of phones on both operating systems. The MIT LL team drew upon their experience with the Guardian data collection to develop a test plan for nine unique scenarios across social, professional, and public transit environments (Figure 13), and extended the software tools developed for the first data collection. The test scenarios included scripted “natural” mobility, up to 16 phones per scenario, and were repeated three times each. The team’s earlier experience with Android EN data collection techniques enabled them to carefully construct test scenarios that included iPhones, and reconstruct in post-analysis which BLE messages came from which iPhones. The details, results, and practice implications of the second data collection are discussed in the associated program report, ACTA3.