One Character Made 775 Reserve Rows Wrong: Inside chainladder's CapeCod.predict Bug
Table of Contents
I spent ten years as a financial advisor. Actuarial work was the room next door. chainladder-python is the loss-reserving library maintained by the Casualty Actuarial Society, and actuaries use it to estimate how much to hold for claims that have happened but are not yet reported. On September 5, 2026 my second PR to it was merged. The change is one character.
This post lays out everything around that character: what the method computes, why the line was wrong, why the existing test did not catch it, and how the fix was agreed with the maintainer before any code existed.
What the Cape Cod method computes
The classic reserving technique is the chain ladder: look at how past years' claims developed over time, apply those development ratios to estimate where this year will end up. The problem is that recent years have little data, and multiplying a small number by a large development factor amplifies error.
Cape Cod starts by estimating an expected loss ratio, the apriori: total developed claims across all years divided by the corresponding earned premium. That ratio then fills in the undeveloped part of recent years. The apriori is the core of the method. Get it wrong and every reserve row downstream is wrong.
In chainladder-python, CapeCod().fit(triangle, sample_weight=premium) produces apriori_, and predict(new_triangle, sample_weight=new_premium) applies the fitted apriori to new data.
Grain: predict at the level you fit at
Actuarial data has grain. The same data can be viewed by line of business (LOB), by company (GRNAME), or both. The public clrd sample is like that: each row is one company on one line.
A common pattern is to fit coarse and predict fine. Fit the apriori by line (summing all companies in the line), then predict per company, so each company uses its line's apriori:
import chainladder as cl
clrd = cl.load_sample("clrd")
tri, prem = clrd["CumPaidLoss"], clrd["EarnedPremDIR"].latest_diagonal
model = cl.CapeCod().fit(tri.groupby("LOB").sum(),
sample_weight=prem.groupby("LOB").sum())
model.predict(tri, sample_weight=prem)
predict() has to decide: is the prediction data finer than the model? If so, aggregate up to the model's grain and use the fitted apriori. If not, use it directly. It decides by counting the index levels the prediction sample_weight carries that apriori_ does not.
The line
if len(set(sample_weight.key_labels) - set(self.apriori_.key_labels)) > 1:
"More than one extra level." In the example above the model was fit on LOB, the prediction data carries LOB and GRNAME, and the difference is one level: GRNAME. One is not greater than one, so it falls to the else branch, recomputes the apriori from the prediction data, and throws the fitted one away.
Measured on the comauto line: the fitted apriori is 0.5689995797 and predict() returned 1.2516635774. clrd has 775 rows, and all 775 differed from the value fitted for their line. After the fix every row carries its fitted apriori, and expectation_ equals premium times apriori exactly across all 775.
The fix: > 1 becomes > 0. Any extra level means aggregate.
Why the existing test missed it
The repository has test_capecod_predict2, and it covers this path. I looked at its dataset: prism. The prism triangle carries five index levels more than the fitted model. Five is greater than one, so it always takes the aggregation branch and never hits "exactly one".
Coverage reports will not tell you this. The line executed, the coverage is green, but it only ever executed on an input that cannot trigger the bug. The test covered the branch, not the boundary. The new regression test uses clrd, which carries exactly one extra level, and asserts explicitly that the extra level is {"GRNAME"}, that the predicted apriori equals the fitted apriori, and that the two ultimate totals agree within 1e-6. It fails on the unpatched line and passes on the patched one. I ran both.
The fix was agreed first, the code came after
The PR came out of issue #1265, where maintainer henrydingliu had already identified > 0 as the fix and asked for a PR. My job was to reproduce, quantify the impact, write a test that catches this boundary, and confirm the documented example was unaffected: the predict() docstring uses ukmotor, fits and predicts at the same grain, so the level difference is empty and neither branch changes. Its documented output still reproduces exactly.
An API design question that surfaced in the issue went into a separate issue, #1274, rather than into this PR.
Before merging the maintainer asked one thing: pick up the ruff lint fixes. I did. capecod.py had an unused numpy import, the test file had whitespace issues, and I removed both files from the per-file-ignores table since its comment says to remove entries as files are cleaned up. I left ruff format alone: it touches 40 of the test file's 107 lines, and that is not this PR's business.
The AI disclosure
casact has an AI usage policy that asks contributors to disclose. What I wrote in the PR:
I used Claude Code on this. It reproduced the bug against clrd, ran the before and after comparison, and drafted the regression test. The one-character fix itself is the one henrydingliu and I settled on in #1265 before any code was written. I reviewed the diff and the test myself, and the suite was run locally on my machine: 1130 passed, 7 skipped.
The maintainer merged without comment on it. My view: the point of disclosure is not "was AI used" but "which judgments were human". Deciding the fix, bounding the scope, leaving ruff format alone: those are human calls. Reproducing and drafting a test, the tool does faster than I do, and there is no reason not to use it.
Three things to take away
- One character can make an entire report wrong without raising. The reserve comes out as a plausible number. No traceback. This class of bug is only visible when you put the fitted value next to the predicted one.
- Green coverage does not mean the boundary was tested. Ask: can the input this test uses actually trigger the condition I am guarding? If not, it is protecting something else.
- Agree in the issue, and the PR stays short. Opened to merged in about 57 hours, because the only thing the maintainer had to review was what I proved, not what I wanted.
The PR: casact/chainladder-python#1275. Three weeks earlier in the same repository, #1205, I made a mistake in the description that the maintainer caught on the spot. That story is in the seven-PR roundup.