0fc66fedcb
Standardize draft articles with YAML frontmatter including title, date, topics, related, and abstract. Expand README drafts section into a table listing all drafts with topics. Add "Testing Telos" to published articles.
14 lines
1.2 KiB
Markdown
14 lines
1.2 KiB
Markdown
---
|
|
title: "Testing Probabilistic Systems"
|
|
date: 2026-04-20
|
|
topics: [philosophy, craft]
|
|
related:
|
|
- testing-telos.md
|
|
abstract: >
|
|
The testing pyramid was built for deterministic, functional code, and it breaks on probabilistic systems where "correctness" is a statistical property rather than a per-invocation one. ML components and signal-producing pipelines demand a different shape of test — and the usual telos-shaped diagrams do not quite accommodate them.
|
|
---
|
|
|
|
Testing Probabilistic Systems. LiverMultiScan has ML components; cardiac T1 mapping produces distributions not binaries. The testing pyramid was built for deterministic, functional code — it breaks on probabilistic systems, where "correctness" is a statistical property, not a per-invocation one. This is a natural sequel to Testing Telos: none of your four shapes quite fits ML. Google's "ML Test Score" paper[^1] and Christian Kästner's "Machine Learning in Production"[^2] are good starting points. This is also where your concern about LLMs and your day job most obviously meet.
|
|
|
|
[^1]: https://research.google/pubs/the-ml-test-score-a-rubric-for-ml-production-readiness-and-technical-debt-reduction/
|
|
[^2]: https://ckaestne.github.io/seai/ |