How UB’s COVID-19 modeling workgroup helped change behaviors, save lives

Winkelstein: The groups came together in the early days of COVID as an organic response to, “This thing is coming and we don’t know what it will look like.” We were watching New York City with overwhelmed hospitals, and everyone was imagining that Buffalo was next. I was asked to take a look, and said, “What data do we have and what can we get?” We barely had anything.

In response to wanting to use the numbers to help us understand what was coming, we put together these two groups. The analytics group ended up being a cross-organizational group with representatives from the various health systems and other organizations. This group was trying to take a broad look at any data we might have or could get our hands on as a community.

The modeling group grew out of looking specifically at one set of data: hospital admissions. We were very concerned about the hospitals being overwhelmed. At that time, it was hard to figure out how much COVID we had. There was no testing. We were flying blind, and the hospital data was our best probe into the community prevalence of COVID. It was data that could help inform local decision-making.

Anaya: When we first got involved, our task was mostly to try to understand what was happening and what could happen here. We looked at other places that already had COVID and what were the hospitalization, ICU and mortality rates that they were seeing. During that search, we came across different models of predictive analytics from across the U.S. and also from other countries.

Some of these models are designed for bigger cities, or for the whole country, but we wanted to focus on Erie County. That is when we started to analyze the models and incorporated some of the data that we had for Erie County. Once we started to get COVID cases in Erie County, that’s when we began to fine-tune our model. We saw that the amount of cases here was not making sense compared to the national or international numbers. There was something a little bit different in Erie County.

Mullin: Locally, we saw more stringent social distancing and facemask-wearing measures. We did not see that giant peak [that presented in other areas] because of the mitigation factors in place. How do you model that? This became a huge question for our group. We have not had this size of a pandemic since 1918. So, as a group, we started to use different mathematical functions.

Bonner: The model that we started with is a standard Susceptible-Exposed-Infected-Recovered (SEIR) model that was invented back in the early 1910s, and it was never intended to be a predictive model of what is happening in real time. It was developed to help understand what intervening in some sort of way might do to an emerging epidemic. It became clear that this simplified model was not going to capture the complexity of the biology and the behavioral aspects to transmission of SARS-CoV-2.

So, we started looking at what other people were doing and at the information that we could gather about how many people were actually doing things like social distancing. But it is a big challenge because these models were never designed to make predictions like, “What is coming in the next three weeks? What is the peak going to be?” These kinds of models were almost historical in nature. They are great models for afterward, when this epidemic is subsided and you can go back and do your serosurveillance and find out how many people were actually infected. Doing it in real time is much more challenging.

Winkelstein: We started with a baseline epidemiologic model and some code behind it, but basically the model was pretty standard epidemiologic stuff. And it became very clear that was not going to cut it. It did not produce what we were talking about.

First we needed to understand that COVID did not lend itself naturally to a baseline epidemiologic model. We had to figure out how to make the model more complicated. Then we had to add in all these other factors, like Sarah mentioned: What do we do about social distancing and face-masking? We know it makes the virus less likely to go from one [person] to another. How do you put that into the equations? There is a huge amount of work in trying to understand how to take what was going on in the real world and turn that into the equations that drove the model.

Mullin: The first three months were pretty much anxiety and panic all the time because we saw that if it actually got to that peak, our hospital systems would be overwhelmed. In addition, there was a lot of stress because what if we were estimating too low and Erie County did not prepare well enough?

Anaya: The first months were very stressful because in other places, the number of infections was very dramatic. We were very lucky that we had different interventions implemented for all of New York State, so we did not have the full impact in Erie County. We were happy that [our most dire predictions] were wrong. As science people, we always want to be right, but in this case we did not want to see the worst-case scenario.

Winkelstein: The short answer is, yes, they paid attention — and they are still paying attention. Nearly every Tuesday I brief the Erie County Department of Health and hospital leaders on the data. In some sense, our numbers seem to have stabilized.

As Gabe said, we wound up initially overestimating the number of cases we might see. I would not phrase that that we were wrong. I actually think that those projections wound up making us wrong. They helped us be wrong because people saw them and recognized what we might be facing if we did not change our behavior. People did change their behavior. It was not just us alone convincing them, but I think we were a part of that. I consider that to be a real significant success. We dodged a big bullet in April, partly because we and others were saying, “This is going to blow up, and blow up fast.”

I was very anxious during the early days; it looked bad to me. I had many uncomfortable conversations — things like trying to advise the Department of Health on how many body bags to buy. It was a tough time, but I think our work made a difference, and it is still making a difference.

As Matt explained, our model is more of a what-if tool, an exploratory tool. It is not necessarily a predictive tool. I am not going to be able to tell you how many patients are going to be in the hospital at the end of February because it depends on things like human behavior. But I can tell you what is going to happen if people burn their masks.

For the second wave [last] fall, we were able to use our model to go backward, too. We got the initial spring curve, and we were able to understand it with the model. It went up, it went down, it stayed low, and things were trucking along. Then it spiked up again in the fall. How did it get like that? You needed a pretty sudden change in rate of transmission of the virus at the end of October. The only way we could get the model to follow this fall spike was to say that something dramatically changed, and it had to be at the end of October. And what changed was transmission. Why transmission changed dramatically at the end of October, the model will not be able to tell us. We will probably never know. I think it is because that is exactly when the weather got cold and everybody moved indoors. Maybe it was a new variant that hit us at that time. The point is that the model helped inform the fact that something happened at the end of October that was different, and it happened very suddenly and very dramatically.

Winkelstein: Our fate is in our own hands to a large extent. What are now common sense measures — facemasks, social distancing, small crowds, not having Buffalo Bills parties at your house — these things are still our best tools by far, and they work. One of the things the model illustrated very successfully is these things really work. Facemasks, social distancing, these are not little things. They are actually the biggest things. Behavior drives the shape of the curve, and that is one of the messages I keep trying to get across: Everything about the curve represents behavior. So, we cannot take our foot off the gas. We have got to stay vigilant, especially in the face of the potential of a more infectious variant coming into the area.

That is the direction I have asked the team to start thinking about: How do we model a new variant? I cannot tell you when a new variant might drop out of the sky and get us in Buffalo, but if and when it does, what might that look like in the curve? That is a what-if scenario that we might be able to model. Also, what impact will vaccines have on that curve, and how quickly? This is what we are exploring now. [In the face of changing societal trends, re-openings, new variants, and vaccines], personal behavior is going to be absolutely key for the foreseeable future. Not only do we have to not take our foot off the gas, we probably have to press harder.