---
title: "Testing Probabilistic Systems"
date: 2026-04-20
topics: [philosophy, craft]
related:
  - testing-telos.md
abstract: >
  The testing pyramid was built for deterministic, functional code, and it breaks on probabilistic systems where "correctness" is a statistical property rather than a per-invocation one. ML components and signal-producing pipelines demand a different shape of test — and the usual telos-shaped diagrams do not quite accommodate them.
---

Testing Probabilistic Systems. LiverMultiScan has ML components; cardiac T1 mapping produces distributions not binaries. The testing pyramid was built for deterministic, functional code — it breaks on probabilistic systems, where "correctness" is a statistical property, not a per-invocation one. This is a natural sequel to Testing Telos: none of your four shapes quite fits ML. Google's "ML Test Score" paper[^1] and Christian Kästner's "Machine Learning in Production"[^2] are good starting points. This is also where your concern about LLMs and your day job most obviously meet.

[^1]: https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/
[^2]: https://ckaestne.github.io/seai/