Breaking down the COVID-19 numbers: Should we be comparing countries?
TORONTO -- In a little more than two months, SARS-CoV-2, the virus that causes COVID-19, raced around the world and turned a handful of known cases to more than three quarters of a million, with 36,000 lives lost and counting - reported figures that the scientific and health communities widely agree are too low.
The spreading virus has pushed numerous countries to scramble to lock down cities, shutter non-essential businesses, and close their borders to all but their own citizens, adopting some of the extraordinary measures executed in China that might have previously been unthinkable elsewhere.
Researchers and armchair epidemiologists alike are analyzing the trove of data to create models, find patterns and clues on whether curves are being flattened, which country is on a faster or slower trajectory, why death rates and ages vary, what measures seemed to work, and when the pandemic might end.
The flood of numbers and questions they raise can be overwhelming for the average person trying to make sense of the data.
Epidemiologists and an infectious disease expert who spoke with CTVNews.ca said it was too early to make predictions or draw conclusions from the data, but stressed the importance of understanding the context surrounding the numbers.
Some of the key variables to consider when looking at the data include:
Some countries and jurisdictions are testing broadly and aggressively, while others are testing based on a stricter criteria of travel and symptoms. There is no global criteria to follow, though the World Health Organization told governments earlier this month to “test, test, test”, even as experts were divided over which method is more effective and a better use of resources. Scientists do agree that how broadly or narrowly a country is testing is a factor in the wide range of case fatality rates from country to country.
Correlation between capacity and results:
If testing capacity increases, does the number of positive results also increase proportionally over the same period? Or does an increase in new positives result in a rise or fall in the overall percentage of positive cases?
Developing countries may be more vulnerable to the disease due to a number of factors and may not have the resources for wider testing. Conversely, countries with better access to care could also be a driver in the higher case count due to a greater number of tests being conducted, one epidemiologist said.
How ‘confirmed’ tests are defined and counted:
Some countries include symptomless positive cases under “confirmed”, while others, like China, specifically do not. Meanwhile, some estimates indicate that patients who show little or no symptoms could account for 60 per cent of the cases. The criteria for the type of cases included can also change in the middle of reporting. It is a “break” in the data that can result in a sudden jump or fall. This happened in China, which caused confusion and controversy, and also with Quebec, for example. The province recently saw a surge in confirmed cases after it began including tests administered at hospitals.
A government’s reputation for transparency can also impact the level of trust one has in the data. There has been skepticism around Russia’s numbers, for example, which has so far reported fewer cases than the tiny country of Luxembourg.
The population density, demographics, smoking rate, healthcare system capacity at the time of the outbreak, even cultural habits (e.g. greeting with a kiss vs a handshake vs a bow), are also examples of country-specific factors that may or may not play into how an outbreak develops.
IT’S NOT JUST ABOUT THE TOTAL NUMBER OF CASES
While most of the focus has been on the daily tally of new cases, epidemiologists say that other data points are more useful.
Cynthia Carr, a Winnipeg-based epidemiologist with two decades of experience interpreting and developing protocols for gathering and analyzing health data, said the daily focus on new cases can be a distraction and spark unnecessary panic.
“[The public was] not listening to the information. They were in a store with 1,000 people at Costco buying toilet paper” when that was the last place they should be, said Carr.
The total number of tests administered, infections, hospitalizations, intensive care patients, and deaths are all key indicators for different reasons, explained Erin Strumpf, an epidemiologist and associate professor at McGill University.
“It’s more about the rate of change in those numbers than it is about the actual numbers on a given day,” she said.
The mortality and hospitalization rates - and whether they are increasing or decreasing over time - gives more context and balance to the data, Carr noted.
“You should never just look at one piece of information,” Carr said.
“I have said from the beginning, when we increase our testing capacity, you would quickly see an increase in cases... we’re getting more of an accurate denominator, an accurate representation of the number of people with the illness.”
COUNTRY VS COUNTRY: TO COMPARE OR NOT TO COMPARE?
A number of countries reported their first cases around the same time, in late January, but the timing and containment measures taken by each government have, at times, been vastly different.
Both the United States and South Korea reported their first case around January 20, for example. The United States, whose response to the pandemic has been widely criticized, has since reported more than 140,000 cases and recorded more than 2,500 deaths, while South Korea, which has been widely praised for its swift containment measures, has recorded some 9,600 cases and 158 deaths.
But differences in testing protocols, testing availability, and a host of other factors, combined with circumstances unique to each country make apples-to-apples comparison complicated.
Still, with 181 countries and regions around the world reporting cases of COVID-19, those comparisons will be inevitable.
In the short term, Strumpf said it should be made with caution, with the understanding that much of the data is generated quite differently from jurisdiction to jurisdiction, and even at different points in time.
Yet, those very differences in methods and approaches are why comparisons will eventually need to be made, she added. “Ultimately, the comparisons become very, very important...That’s how we’re going to learn from this whole experience.”
SOUTH KOREA AND ITALY AS CASE STUDIES?
South Korea has been widely lauded for its efforts to quickly control the outbreak. Authorities there wasted little time launching broad and innovative testing methods like drive-through testing sites, quarantining infected patients in government shelters, and using GPS data on cell phones and credit card information to retrace a patient’s movements in detail and alert those who may have been nearby. The public tracking is not without controversy, but South Korea’s aggressive tactics resulted in daily case numbers soaring quickly, but also declining quickly, with 158 reported deaths. Its death rate stood around 1.6 per cent as of March 30. And, unlike many other places with significant outbreaks, it was done without dramatic shut-down measures or enforced physical distancing requirements for most of South Korea’s population of 51.5 million.
India’s 1.3 billion people went into “complete” lockdown last Tuesday amid fears the country’s low infection count, which stood at 1,024 cases and 29 deaths as of March 30, did not paint an accurate picture given how few people had been tested so far. Even more concerning, the lockdown has sent hundreds of thousands of people fleeing from cities back home to villages unprepared to handle potential outbreaks. India’s medical community and epidemiologists like Strumpf say it is only a matter of time before the virus sweeps the country with potentially devastating consequences.
Meanwhile, Italy is the worst-hit country in the world in terms of total number of reported deaths, with a mortality rate of 12 per cent as of March 30. The extraordinarily high death rate could be attributed in part to how many tests were administered in some of the hardest hit regions, but overall fatality figures still more than doubled in a week, and accounted for nearly a third of the global deaths as of March 30. Last week, Italy recorded 919 deaths in a single day. The government instituted various containment measures including ordering school closures, for example, when the death toll reached 100. When the number of deaths surpassed 450, the government enacted the “largest clampdown against the coronavirus outbreak in the Western world” according to the New York Times, putting the entire country - some 60 million people - under quarantine, with additional measures added in the subsequent days and weeks.
In contrast, New Zealand urged its citizens to come home and closed its borders to all foreigners when the total number of identified cases stood at less than 50 and zero deaths. A nationwide lockdown was put in place when the country still had fewer than 300 cases, and still no deaths.
“Part of the story about why we’re seeing such bad outcomes in Italy and Spain are linked to the fact that they were always behind the curve in putting those measures in place,” said Strumpf, describing it as “too little, too late.”
But Carr cautioned against simplistic comparisons of the most extreme examples.
“The last person you want to compare yourself to is Italy, because they are an outlier,” she said, pointing to their exceptionally high death rate compared to Canada’s 1 per cent, or even the WHO’s estimate of 3.4 per cent.
“That is not the scenario we should be planning based on because they have something very different going on. And to just simplify it and go, ‘They ignored it until it was too late’ - do we know that?”
Instead, it may be more useful to learn what factors are contributing to its outlier status and what can be learned from it, she said.
Potential non-COVID-19 variables Carr suggested considering include the number of physicians and hospital beds per 1,000 people, the vaccination rates for those over 65, the age distribution of the population, the population density of the country, the smoking rate, whether there were “stressors” currently in the country’s health system, and the poverty gap, among other factors.
According to 2018 data from the OECD, Italy has more doctors than South Korea, for example, but South Korea’s overall population density is more than double Italy’s. Meanwhile, the smoking rates for the two countries are somewhat similar.
The influenza vaccination rate for those over 65 in Italy, however, stood at just over 52 per cent, significantly lower than the nearly 83 per cent rate for South Korea. Italy also has one of the highest percentages of people over 65 in the world, second only to Japan, according to the World Bank. Data indicated that some 65 per cent of the COVID-19 deaths in Italy were among those over the age of 70.
Yet how does this explain Japan’s seemingly low reported case count (so far), low number of reported deaths and limited government response to the pandemic until last week? Japan is more densely populated than Italy, smoking rates and low flu vaccination rates are comparable, according to OECD data, and 27 per cent of its population are over 65.
“It could be that none of these factors are impacting the outbreak, but it’s just always a good idea to get the context,” Carr added in an email later.
Like South Korea, Japan’s proximity and contact with China would seemingly make it more vulnerable. Some analysts speculated whether the low reported figures were due to its low rate of testing, but if that were the case, hospitals would still see an increase in patients, for example. However, that has not been the case. Others have pointed to Japan’s etiquette of bowing, rather than a kiss or a handshake, mask-wearing, and hygiene education as potential differentiating factors. Masks and bowing are also common in South Korea.
Strumpf said these cultural elements may play a part, but SARS, which hit in early 2003, was also fresh for many of the regions surrounding China, and likely motivates people’s willingness to follow the rules.
“They understand both how bad it can get and the importance of everybody pitching in and changing their behaviour to try to limit the spread,” she said.
All these uncertainties - and current lack of answers - underscore both the challenge in making comparisons but also its value.
In Italy’s case, Carr noted that the region saw a “massive” flu outbreak, a key factor that put pressure on their healthcare system even before COVID-19 struck.
“We have to really understand what’s going on in a population - always - before we ever compare ourselves to it,” she said.
“Context is key right now.”