Load Testing an Appointment Platform

The provider's booking platform fell over during their seasonal campaign two years running — not gradually, but a hard failure about twenty minutes into the morning the campaign opened, with patients unable to book at all. Each time it was diagnosed after the fact and fixed with more servers, and each time it happened again. Nobody knew where the actual limit was or what broke first, because the failures were only ever observed in production under real load. They wanted to know the ceiling before the next campaign rather than during it.

I was brought in as a performance testing specialist for a fixed engagement of about three months, working with their platform team of five. I owned the testing approach, the scenarios and the analysis. I was explicit at the start that I would not be fixing what we found — their team would — because the last contractor had done both and left them unable to maintain either.

I built the load testing suite, including the scenario modelling based on their actual traffic logs rather than a guess at user behaviour. I owned the test environment setup, the data generation for realistic booking loads, and the analysis that traced each failure to a specific cause. I wrote the capacity model that maps expected concurrent users to required resources, and the runbook for their team to rerun everything before each campaign. I did not implement any of the fixes, though I reviewed two of them.

I insisted on modelling scenarios from real traffic logs instead of the even, well-behaved load pattern that had been used before. Real campaign traffic is a spike inside the first fifteen minutes with a heavy bias toward one endpoint, and testing an even ramp is what had made the platform look fine. That single change reproduced the production failure on the second test run. I also tested against a scaled-down environment with a documented scaling factor rather than pushing for a full production clone, which the client wanted — a clone would have taken most of the engagement to build and I'd rather have four weeks of testing at two-thirds fidelity than one week at full. I ran the database under separate isolated tests too, because in a full-stack test the application layer saturates first and hides everything behind it.

Three months, fixed, with the campaign date immovable. The test environment was smaller than production and shared with a development team, so heavy runs had to happen overnight. Patient data rules meant fully synthetic test data and a generator that produced realistic distributions, since a uniform dataset made the database look far healthier than it is. I also couldn't test the third-party payment and identity integrations under load at all — their providers wouldn't allow it — so those were stubbed with measured latency profiles and flagged as an open risk.

We found the actual ceiling and it was around forty percent of what the team believed. The first failure was not capacity at all — it was a connection pool limit interacting with a slow query on the availability lookup, which is why adding servers had never helped. Their team fixed that plus two other issues we identified, and the retest showed roughly a four-fold improvement in sustainable concurrent bookings. The campaign that year ran without an outage, which is the only result anyone remembers. Their team has rerun the suite before each campaign since, twice without me.

Their platform lead sat with me for most of the analysis and found the connection pool issue himself once we had the evidence in front of us. A database administrator on their side did the query work and taught me more about execution plans than I knew going in. The service manager provided two years of real traffic data, without which the whole engagement would have been guesswork.