Claims regression suite for a policy admin replatform

The HMO was replatforming its policy administration system, and every release candidate had to clear a regression pack of roughly two thousand manual test cases covering claims intake, adjudication paths, and member eligibility. Six testers took about three weeks per cycle, releases were slipping to protect the regression window, and the replatform vendor was shipping monthly. The math simply did not work, and everyone knew it — the question was whether automation would be built properly or panic-bought.

Automation lead — my first lead role — for twelve months, with a team of three testers who had deep domain knowledge and no programming background. My job was as much teaching as building: the suite would outlive my involvement only if the people who understood claims could maintain it without me.

I owned the framework: the choice of tooling, the page object layer, the test data strategy, the CI wiring, and the conventions that decided what a readable test looks like. I also owned the training curriculum — weekly pairing sessions where the testers converted their own manual cases — and the suite's flake budget, which I tracked and defended like a financial figure because in practice it is one.

The contested decision was building fresh on Playwright rather than extending the Selenium suite a previous contractor had left — the old suite's flake rate had already taught the organization to ignore red builds, which is worse than having no suite, and Playwright's trace files meant a failure handed developers a recording instead of an argument. Test data gets created through the application's own APIs at the start of each test rather than assuming database state, which cost us speed and bought us truth. And I capped the smoke suite at fifteen minutes by decree: a suite developers actually run before merging is worth three suites that run overnight.

The vendor refreshed the shared test environment every weekend, wiping whatever state existed — which forced the tests-create-their-own-world discipline that I would now choose anyway. The three testers wrote their first tests by pattern-matching mine, so every convention had to be teachable in an afternoon; anything clever enough to need explaining twice got rewritten plainer. And the manual pack was the political baseline: coverage had to be argued case by case against a spreadsheet the QA manager had maintained for nine years and trusted more than any dashboard.

Release regression went from three weeks of six people to a nightly run plus two days of targeted exploratory testing. Flake has held under one percent, which is the number I'm proudest of because it's the one that keeps the builds credible. Two of the three testers now write and review tests without me, and one has since been promoted into an SDET title that didn't exist at the company before this project — the suite was the deliverable, but that career path might be the actual impact.

The three testers are the co-authors of the suite in every meaningful sense — they knew which of the two thousand cases were load-bearing and which were archaeology. The QA manager challenged the coverage mapping case by case, which was exhausting and correct. On the vendor side, one sympathetic engineer got us API access for data setup that wasn't in the contract, and the suite would be half as good without it.