A retryable JUnit 5 extension for flaky tests
As I work a lot with Large Language Models (LLMs), I often have to deal with flaky test cases, because LLMs are not always consistent and deterministic in their responses. Thus, sometimes, a test…
Read the full articleYou might also wanna read
Your Flaky Tests Are a Data Problem, Not a Test Problem
Most flaky test fixes focus on retries and quarantine. The real fix is replacing hand-written test data with recorded traffic that stays fre
The Challenge of Reproducible LLM Inference: Why Even Greedy Sampling Isn't Deterministic
Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language mo
thinkingmachines.ai·10mo agoEliminating Flaky Tests with Traffic Replay
There are few things that can derail developer productivity and undermine your pipeline like a flaky test. Testing is the backbone of a good
Building a Production Control Layer for Reliable LLM Structured Outputs
Most LLM failures in production aren’t random — they’re predictable. I kept hitting broken JSON, silent failures, and outages that froze my
towardsdatascience.com·24d agoWhy LLM Evaluation Methods Fail When Models Enter New Capability Regimes
May 17, 2026
Research Shows LLM Programming Performance Varies by Success Criteria: Test Passing vs. Maintainer Approval
Article URL: Comments URL: Points: 40 # Comments: 19
entropicthoughts.com·3mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.