Regression Automation for a Banking Back Office

Every release required a manual regression pass that took two testers about two weeks, and because it was expensive the team released rarely, which made each release larger and riskier. The suite was a spreadsheet of around six hundred steps, and by the end of a cycle testers were pattern-matching rather than reading. Defects were still reaching production at a steady rate, mostly in areas the manual pass covered but late, when attention had gone. The ask was automation, but the actual problem was a feedback loop measured in weeks.

I was the QA lead on a team of about twelve, with two other testers reporting to me in practice if not on paper. I owned the test strategy, the automation architecture and the reporting. I also spent a lot of time in release meetings arguing about what a failing test should mean, which was the part that determined whether any of it worked.

I designed the suite structure and wrote the framework layer — page objects, the fixtures that build test data through the API rather than the interface, and the parallel execution setup. I wrote roughly the first hundred and fifty tests myself, then handed patterns to the other testers and reviewed theirs. I owned the pipeline integration and the reporting, including the flake tracking that records every intermittent failure with its history so we could argue about specific tests instead of vague distrust. I did not own unit or integration testing, which sat with the developers, though I pushed hard on where the boundary was.

The decision I'd defend first is that I refused to automate the existing six hundred manual steps. Most of them were variations covering the same paths, and a direct translation would have produced a slow, brittle suite that no one trusted. I analysed two years of production defects instead and built coverage around where things actually broke, which came to about a hundred and eighty tests covering more real risk. I also insisted test data be built through the API before each test rather than relying on a shared seeded database, which cost a lot of setup work and eliminated the cross-test interference that makes suites flaky. And I made a failing test block the pipeline from day one — the team wanted a warning period, and I've seen warning periods become permanent.

The platform integrates with a core banking system that has no test environment we controlled; it was shared with two other projects and reset on a schedule that wasn't ours. That meant designing tests to tolerate a moving baseline and, for a handful of flows, stubbing an integration we'd rather have exercised. There were strict rules about test data — no production data, ever, even masked — so everything is synthetic. We also had no capacity to hire, so the suite had to be maintainable by two testers alongside their other work.

The regression cycle went from two testers for two weeks to about fifty minutes in the pipeline. Release frequency moved from roughly every six weeks to weekly within a quarter, and the first genuinely uneventful weekly release was the point the team stopped treating releases as events. Production defects in covered areas fell by about seventy percent over the following two quarters. The measure I care about most is that flake rate stayed under two percent, because a suite people trust is the difference between automation and theatre.

The two testers on my team wrote more than half the tests and one of them built the flake tracking dashboard, which I'd sketched and she made considerably better. The platform's tech lead moved a set of validations from the interface into the API at my request, which made a whole class of tests unnecessary. Our release manager backed the blocking pipeline when it was unpopular in month two, which is the only reason it survived.