work-blog/articles/drafts/testing-probabilistic-systems.md

title	date	topics	related	abstract
Testing Probabilistic Systems	2026-04-20	philosophy craft	testing-telos.md	The testing pyramid was built for deterministic, functional code, and it breaks on probabilistic systems where "correctness" is a statistical property rather than a per-invocation one. ML components and signal-producing pipelines demand a different shape of test — and the usual telos-shaped diagrams do not quite accommodate them.

Testing Probabilistic Systems. LiverMultiScan has ML components; cardiac T1 mapping produces distributions not binaries. The testing pyramid was built for deterministic, functional code — it breaks on probabilistic systems, where "correctness" is a statistical property, not a per-invocation one. This is a natural sequel to Testing Telos: none of your four shapes quite fits ML. Google's "ML Test Score" paper¹ and Christian Kästner's "Machine Learning in Production"² are good starting points. This is also where your concern about LLMs and your day job most obviously meet.

---

1. https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/ ↩︎

2. https://ckaestne.github.io/seai/ ↩︎