Do the Numbers Add Up? Questions about an Afghan Health Study

Questions about an Afghan Health Study

The news coming out of Afghanistan is rarely good—widespread government corruption, human rights violations, Taliban brutality.

Late last year, when the Afghanistan Mortality Survey revealed dramatic health gains in key indicators— particularly in the health of women and children—media organizations around the world reported that billions of dollars in investments in the country’s health care system were beginning to pay off.

In one piece of shockingly good news, the survey—funded by USAID and other international agencies and carried out by the Afghan Ministry of Public Health and local institutions—reported that from 2004 to 2010 the maternal mortality rate fell from 1,600 deaths for every 100,000 deliveries to 327 deaths for every 100,000 deliveries.

It wasn’t long before public health officials—including some technical advisers who worked on the survey—voiced doubts about the findings and questioned whether the numbers were too good to be true.

David Peters, MD, DrPH '93, MPH '89, is director of the Health Systems Programs and a professor in the School’s Department of International Health, who, for the past nine years, has led a team that has conducted annual health service delivery assessments in Afghanistan. He said the survey data, overall, aren’t convincing.

“I think they’re unlikely to be realistic, but it’s probably the best that can be done under the circumstances in terms of getting good quality data,” said Peters. “While I have no doubt that mortality indicators have improved, I think it’s unlikely that they have improved to the degree they’re showing in this analysis.”

The USAID website describes the survey as a “landmark study,” and the first comprehensive mortality survey ever conducted in Afghanistan. The agency notes that for security reasons, some rural areas in the country’s southern regions with heavy insurgent activity were not included in the study.

Other findings from the survey, which looked at the period between 2004 and 2010, include the following:

• Under 5 child mortality fell from 172 deaths per 1,000 live births to 97 deaths per 1,000 live births.
• Infant mortality fell from 115 deaths per 1,000 live births to 77 deaths per 1,000 live births.  
• Estimated life expectancy rose from 42 years to 62 years.

Peters explained that he isn’t critical of the data analysis, but the quality of the data itself.

Gathering raw numbers in household surveys is fraught with difficulties, he said, including the unwillingness of Afghans to talk to data-collection workers and worsening security, which hinders the supervision of data-collection teams.

“You’re dealing with a highly illiterate, distrustful population, many of whom are not willing to disclose who is in a household,” he said, particularly when it comes to women and girls living in the home.

Despite his misgivings about the survey, Peters said that there have been improvements in Afghan health care, based on Balanced Scorecard assessments that he and Bloomberg School colleagues have conducted across the country since 2004. The Balanced Scorecard is an evaluation and monitoring tool developed by Peters and Gilbert Burhnam, MD, PhD, MS, a professor in International Health, to measure key components of health services. The Scorecard assessments are based on several indicators, including observations of providers with patients, interviews with patients and providers, quality of provider training and availability of drugs and medical equipment.

“Beginning in 2004, we can show very consistent improvements in terms of access, utilization and quality of care, until it levels off in 2010,” said Peters. He attributes the plateau effect to a number of factors, including deteriorating security—a major barrier to people seeking health services—new data from insecure areas where health services are inferior, and the fact that in 2010, several NGOs, which provide approximately 70 percent of all primary health care in the country, renegotiated their contracts.

“Services came to a standstill in many places,” Peters noted, adding that investigators are in the process of completing the 2011 Balanced Scorecard analysis.

According to Peters, meaningful improvement in the health of the Afghan population depends not only on more and better health facilities, care providers and basic public health services, it is inextricably linked to peace, the education of women and girls, as well as addressing government corruption and controlling the illicit drug trade.

“The solutions to that are certainly beyond the health sector,” he said. — Jackie Powder