Testing
7 articles about "Testing".
What CapeCod.predict Actually Does: chainladder Re-Estimates the Apriori and Returns a Third Number
chainladder-python's CapeCod.predict does not apply the model you saved. On the ukmotor sample, the apriori fitted on the prior diagonal is 0.6572660657, predict on the new diagonal returns 0.6894525318, and a clean refit on the new data gives 0.7062253126. Taking the formula apart, predict uses today's losses and exposure paired with last period's development pattern.
A Validation Check One Line Too Late Doesn't Raise, It Becomes Decoration: Four Silent Failures in chainladder's predict
predict in the actuarial library chainladder-python still returns a result for four kinds of mismatched input: 775 rows go in and 643 come out, 6 rows go in and 775 come out, one row's pattern applied to another row comes out 38% short, and a paid pattern applied to incurred data overstates by 52%, all without an error message. This post unpacks two input-validation lessons from our fix, PR #1310: legitimate and dangerous looseness look identical in the data, so look for a marker the system writes itself; and a check placed after the line that rewrites its input does not raise, it goes blind in exactly the case it most needs to see.
Regex Can't Tell "Said It" From "Warned Against It": False Positives, Misses, and Our Overturned Fix in LLM Output Scanning
Matching LLM replies with regex to catch XSS or SQL injection strings also fails replies that name the hazard in order to warn against it: on an unmerged open-source Giskard PR, all 5 such advice sentences tripped the check. We proposed anchoring the patterns to line start and false positives fell to 0. Then the PR author added one ordinary phrasing, a lead-in on the same line as the payload, and the anchor missed 12 of 24 compliant replies. This post covers why the anchor only held on our sample, how to choose between over-reporting and under-reporting, and why our own output scanner has the same limit.
A Promise That Never Settles: How a Failed IndexedDB Write Left the MITRE ATT&CK Search Spinning
A new Promise(async (resolve) => ...) that never took a reject cannot fail. When the IndexedDB chunk write inside it threw, the promise handed to the caller did not reject, it never settled at all, so the .catch the authors had already written could never fire and the search spinner ran until the tab closed. Using mitre-attack/attack-website #637 to explain Promise executor semantics, why clearing ESLint's no-async-promise-executor would not have fixed it, and how to write a test that tells a hang apart from a rejection.
One Character Made 775 Reserve Rows Wrong: Inside chainladder's CapeCod.predict Bug
chainladder-python, maintained by the Casualty Actuarial Society, is the standard library actuaries use for loss reserving. CapeCod.predict() had a condition written as > 1 that should have been > 0, so the most common usage got the wrong apriori: the comauto line fitted 0.569, predict returned 1.252, and all 775 rows were off. An existing test covered the path and still missed it, because its dataset always walked the other branch. On what the Cape Cod method computes, why the line was wrong, how the test missed, and the AI-use disclosure filed under casact's policy.
A Named Parameter Never Reaches **kwargs: Why Every Disaster Photo in HOT's Drone System Was Stored as the Wrong Type
A function declared content_type in its signature and documented it, then called the downstream SDK with positional arguments plus **kwargs. Because content_type is a named parameter, Python bound the value to it and it never reached kwargs, so every post-disaster aerial photo went into S3 as application/octet-stream and browsers downloaded it instead of showing it. Using hotosm/drone-tm #882 to explain Python's binding rules, why metadata in the same function was fine, and a test technique that binds mocks to the real SDK signature.
Seven PRs Merged Into Six Organizations in 24 Days: What the Maintainers Taught Me
Between August 12 and September 5, 2026, 24 days, maintainers at the Casualty Actuarial Society (twice), CERT/CC, the UK AI Security Institute, FINOS, NIST and Humanitarian OpenStreetMap merged seven of my pull requests. This is a line-by-line account: where each bug was, why the existing tests missed it, and what the reviewer corrected. Six unrelated projects, three root causes: Linux-only CI, tests that pass for the wrong reason, and sentinel or binding semantics that betray intuition. Every PR is linked.