Letting AI write the tests humans never would.

Here is the boring truth about regression suites. Nobody wants to write them. They are tedious, they age badly, and the payoff on any individual test is close to zero. So they do not get written, and six months later you have a flaky suite that everyone ignores.

AI changes that math for me. Generating a thousand plausible edge case tests against a stable interface now takes minutes instead of weeks. The hard part is not generation. It is selection. Which generated tests actually encode a real invariant, and which ones are just noise that will break on the next refactor?

The pattern that has worked for me is to generate broadly and then promote narrowly. AI proposes. I (or a second AI pass with stricter prompts) decide what graduates into the protected suite. Everything else lives in a quarantine tier that runs nightly and produces signal instead of blocking builds.

Selection alone is not enough though. The promoted tests also have to stay trustworthy over time, which is exactly the problem I tackle in predicting flaky tests with Random Forest. The two pieces compose: AI writes broadly, ML quietly evicts the flakiest tests before they erode trust.

Doing any of this credibly as a leader assumes you are still close enough to the tools to use them, which is the whole point of the unpopular opinion that might save your career. Testing AI and LLMs is a different beast and nobody is going to learn it for you.

The full case study, including the prompts and the CI wiring, lives on my main site.