ENVIRONMENTS: Team discovers the staging environment had been used by customers since February
The discovery happened when an automated test changed the price of a real product. Nobody complained — which, according to the team, was the most worrying part.
- environments
- deploy
- processes
notice: This is satire. Names, companies and telemetry are invented. Any resemblance to your environment is your problem.
An automated test that was supposed to run in the staging environment changed, on Monday morning, the price of a product that was on sale. The change stayed live for six hours. No customer complained.
The investigation revealed that the staging environment had been receiving real traffic since February, when an address was published by mistake in campaign material. Since the environment responded correctly and nobody was monitoring it, the situation went unnoticed for seven months.
"The technical part was easy to fix," said the person in charge. "The hard part was explaining that the environment we call a test environment was getting real people, and that we only found out because a test touched the wrong price."
The team said it will rename the environments, restrict access, and add a visual warning to the interface. Also under discussion is the creation of a fourth environment, to test the changes made in the third.
Satire. Names, dates, and prices are invented.