Two Scoresheets for the Same Deaths

In two Ugandan surveillance cohorts, InterVA-5 and InSilicoVA reproduced a physician panel's population-level cause distribution closely and its individual cause assignment about four times in ten. The gap between those two numbers is where the argument lives.

verbal autopsyvalidationcause of deathfield studies

In Rakai, a district of south-central Uganda where a health and demographic surveillance site has followed whole communities for decades, 1,243 adult deaths were registered between January 2013 and December 2021. A verbal autopsy — the structured interview with a bereaved relative that stands in for medical certification where most deaths happen at home — was completed for 1,037 of them. Two physicians read each record independently. Where they disagreed, a third joined, and the three settled on a cause between them.

That archive of interviews, plus 527 more from a second cohort in Kalungu district in the southwest of the country, is the basis of a comparison published in PLOS Global Public Health in April. Dorean Nabukalu and colleagues at the Rakai Health Sciences Program, the Uganda Virus Research Institute and the London School of Hygiene and Tropical Medicine took the records the physicians had already read and put the same coded answers through two of the field’s standard algorithms: InterVA-5, the deterministic naive-Bayes model developed at Umeå University, and InSilicoVA v1.4, its Bayesian counterpart. Both were run through the openVA toolkit.

What came back is not a verdict on whether software can replace a panel of physicians. It is a split scoresheet, and reading both columns at once is the whole exercise.

Two metrics, two verdicts

InterVA-5 reproduced the physicians’ cause-of-death distribution to within 12 percentage points of perfect: a cause-specific mortality fraction (CSMF) accuracy of 88% (95% CI 85–90), against 83% (79–87) for InSilicoVA. On the same records, counting deaths one at a time, the algorithms matched the panel’s exact cause for 40.5% (34.7–41.8) and 38.2% (34.7–41.8) of deaths on the fifteen-category WHO list, or 41.0% and 44.4% on the fourteen-category version.

Those figures are not rivals, and they do not contradict each other. CSMF accuracy asks whether a population’s proportions are right; percent agreement asks whether a particular death received the same label. One can be excellent while the other is mediocre, and for the purpose that high-volume cause-of-death work exists — a national or district mortality profile — it is the first that matters.

A third number in the same table cuts the other way. Spearman’s rank correlation asks whether the causes are in the right order. Among deaths at ages 15–49 it reached 0.71 (0.64–0.76) for InSilicoVA; among deaths at 50 and over it fell to 0.32 (0.23–0.40) for the same algorithm. The authors describe that as a divergence between proportional accuracy and rank agreement, and note that other comparisons have found the same pattern.

A cause that doubles

The disagreement is not noise scattered evenly across the list. It has a shape, and the shape is HIV and tuberculosis.

On the fifteen-cause list, HIV/TB accounted for 14.1% of male deaths by the physicians’ reading, 31.0% by InterVA-5 and 28.1% by InSilicoVA. It was the leading cause in men under the model and the fourth under the panel. In women the same three figures are 14.2%, 27.2% and 22.3%. InterVA-5’s sensitivity for the category was 80%: when the panel said HIV/TB, the model agreed four times in five. It also assigned the category to more than twice the share of deaths, and that excess can only come from deaths the panel placed somewhere else.

InSilicoVA’s error collects elsewhere. At the six-category level it assigned 24.1% of male deaths to “other communicable” causes against the panel’s 14.8%, while InterVA-5 assigned 11.1%. For non-communicable disease the ordering reverses: 46.9% of male deaths by physician review, 41.6% by InterVA-5, 30.5% by InSilicoVA. Two algorithms, the same interviews, and a sixteen-point spread on the largest cause group in the data.

The deaths nobody named

Physician review produces a second kind of output that rarely reaches a summary table. Of the 876 Rakai interviews that went to the panel, 236 closed without a cause at all: the reviewers could not settle on one, and the record was marked indeterminate. Those deaths were excluded from the agreement calculations, since there was nothing to agree or disagree with, and were analysed separately.

Handed the same records, the algorithms placed a third of the 236 in infectious and parasitic causes and 17% in neoplasms (InterVA-5), or 43% and 8% respectively (InSilicoVA). Two automated readings of the same unexplained deaths, dividing them differently by roughly a factor of two on cancer. Neither figure can be checked against anything: by construction, these are the deaths for which the reference standard declined to give an answer.

The yardstick has its own record

The paper is candid about what its reference standard is worth. Physician review, it says, agrees imperfectly with complete diagnostic autopsy and minimally invasive tissue sampling; it carries inter-reviewer variability and subjective judgment; and its misclassifications are known — HIV/AIDS read as tuberculosis, malaria read as sepsis or meningitis. The interviews were collected with the WHO’s 2012 instrument, and two later versions have been issued since. Both algorithms read only the coded answers; the physicians read the coded answers and the relatives’ own words.

Where the reference is tissue rather than a committee, the picture shifts again. That was the ground covered in this digest’s earlier look at country-specific misclassification rates for computer-coded methods, where the comparison ran against tissue diagnoses from the CHAMPS network rather than against a panel.

The authors’ own recommendation is a tiered arrangement: automation for routine mortality monitoring, physician review held back for priority cases, and no replacement until the algorithms can read the narrative. That is their proposal. This study is a field validation against one reference standard in two Ugandan cohorts; it is not a trial of that arrangement, and no measurement here establishes which of the two readings of a given death is the true one.

What a reader should carry away

The number to remember is not 88%. It is the distance between 88% and four-in-ten, and the fact that both describe the same interviews.

A cause-of-death table built on an algorithm can be nearly right about a district and wrong about a large share of the people inside it. When such a table is quoted — in a burden estimate, a ministry briefing, a news story — those two facts belong in the same breath, because only together do they say what the numbers can and cannot support.

Sources

  1. Nabukalu D, Lutalo T, Mugisha J, Makanga R, Kasamba I, Calvert C, Khan P, Marston M, Todd J. A comparison of verbal autopsy assignment methods to obtain adult cause-specific mortality in two longitudinal studies in Rakai and Kalungu districts of South Central, Uganda. PLOS Global Public Health 2026;6(4):e0006223. doi:10.1371/journal.pgph.0006223
    pmc.ncbi.nlm.nih.gov
  2. openVA — open-source R toolkit implementing InterVA-5 and InSilicoVA
    openva.net
  3. WHO verbal autopsy instrument (2022 revision)
    who.int

← Back to the news digest