Just before midnight Pacific time on October 19, 2025, a DNS record for Amazon DynamoDB in AWS's Northern Virginia region, us-east-1, ended up empty. A latent race condition in DynamoDB's DNS automation had deleted every IP address for the service's regional endpoint, and the automation could not repair it. In AWS's words, it "ultimately required manual operator intervention to correct" (AWS post-event summary).
DynamoDB was working again within about three hours. The region took until mid-afternoon the next day. EC2 launches failed, network load balancer health checks flapped and pulled capacity out of service, and Lambda, ECS, EKS and Fargate were affected along the way. AWS's status updates at the time said more than 75 services were impacted at the peak, with full recovery about 15 hours after the start (Converge Digest).
It was one fault in one service in one region. For a lot of businesses it meant one bad day with everything down.
A year later, the more useful question for most businesses is whether running their systems in one region was ever a decision anyone made on purpose.
What actually broke
The most useful detail is in AWS's own summary: "Existing EC2 instances that had been launched prior to the start of the event remained healthy." The servers already running kept running. What failed was anything that needed to change: launching new instances, opening new connections to DynamoDB, getting credentials from STS, passing load balancer health checks.
That left systems built to respond to trouble by scaling, replacing instances and re-authenticating more exposed than ones that could sit still. Parametrix, which sells insurance against cloud outages, found that impact after the first two and a half hours depended on which availability zones a company used, its peak operating hours, and its "reliance on autoscaling" (Insurance Edge).
The second detail is that "single region" doesn't only mean where your workload runs. Several AWS global services keep their control plane in one region, and in the standard aws partition that region is us-east-1. This covers IAM, AWS Organizations, Route 53 public DNS and CloudFront (AWS Fault Isolation Boundaries). During the outage, Redshift customers "in all AWS Regions" couldn't run queries with IAM user credentials, because of a call to an IAM API hosted in us-east-1. Some console sign-ins failed in other regions too. And until July 2025, several AWS SDKs, including Python and .NET, sent STS credential requests to a global endpoint that resolves to us-east-1 by default. AWS has since switched those defaults to regional, but older SDK and CLI versions that were never upgraded can still use the global endpoint (AWS Developer Tools Blog). A team that deployed to Oregon can still depend on Virginia without ever having chosen to.
It wasn't a one-off either. AWS publishes a public post-event summary only for events with broad customer impact. Of the six it has published since November 2020, five were in us-east-1 (AWS post-event summaries). A raw count partly reflects how much runs in us-east-1. But Uptime Institute's conclusion after the outage is hard to argue with: "a regional failure is not a rare, unpredictable event" (Uptime Institute).
Who pays for the bad day
The comfortable assumption is that someone else is holding this risk. Mostly, nobody is.
The SLA. AWS's compute SLA promises 99.99% at the region level. If it's missed, you can claim a service credit of 10%, 30% or 100% of that month's EC2 bill in the affected region. You have to request it through support, and the SLA describes that credit as your "sole and exclusive" remedy (AWS Compute SLA). Even that assumes the event meets the SLA's definition of unavailability, which is about running instances losing external connectivity. AWS says running instances stayed healthy, so whether a given claim qualifies is not obvious.
Insurance. Cyber business-interruption policies typically have a waiting period of 8 to 12 hours before cover starts. Moody's modeled a 10-hour average and put its mean estimate of US insured losses from the outage at $22 million, with 95% confidence it would not exceed $76 million. Its own summary: "These policy waiting periods dramatically limit potential insured gross losses" (Moody's). CyberCube's range ran from $38 million to $581 million and called it a "moderate" event for insurers (Risk & Insurance). Parametrix, which has a product to sell, put the financial loss to US companies at $500 million to $650 million.
These estimates measure different things and don't line up neatly. They point the same way, though. Most of the cost of a 15-hour outage stays with the business that had it.
Pricing it
Take a hypothetical. A mid-market company runs its customer ordering platform in us-east-1, spread across two availability zones. During business hours the platform handles orders worth around $20,000 an hour. These figures are illustrative, not from any client.
A repeat of last October costs roughly $300,000 in delayed or lost orders, before staff overtime, missed delivery commitments and the customers who don't come back. Against that:
- The SLA credit, if the event qualifies, is a percentage of that region's monthly EC2 bill. Fifteen hours down in a 31-day month lands in the 30% tier, so on a $15,000 monthly bill that's $4,500.
- A cyber policy with a 10-hour waiting period covers, at best, the last few hours, after the deductible.
That doesn't automatically justify a second region. Equal Experts argued after the outage that multi-region "probably isn't the answer." Replication costs add up, and a multi-region system can still lose its ability to fail over when the provider's core services are struggling (Equal Experts). Uptime puts the other side plainly: "if an outage is going to have financial repercussions, paying for greater resiliency is worthwhile."
Both can be true for the same company with different systems. The ordering platform might justify a warm standby. The internal reporting tool almost certainly doesn't. Choosing one region is fine. Never making the choice is the problem.
Earlier this month we wrote about the cost of leaving a cloud provider. This is the other side of it: the cost of staying through a bad day.
What to do before the next one
Write the decision down. For each important system, record which region it runs in, what 15 hours offline would cost, and who accepted that risk. If nobody signed off, nobody owns it.
Find your hidden us-east-1 dependencies. Make sure every SDK and CLI you run uses regional STS endpoints, including older versions nobody has upgraded (AWS re:Post). Set up break-glass access that doesn't depend on your identity provider. List everything in your recovery plan that would need IAM, Route 53 or CloudFront changes to work.
Turn on the fix AWS already shipped. In November 2025, AWS launched Route 53 accelerated recovery, which targets a 60-minute recovery time for public DNS changes during a us-east-1 disruption. It costs nothing extra, but it's disabled by default and has to be enabled for each hosted zone (AWS News Blog).
Don't rely on scaling to get you through the incident. AWS's own advice is to design so that recovery doesn't need control-plane changes. Pre-provision the capacity and resources a failover needs, and cache what you'd otherwise look up.
Rehearse it. A failover plan that has never run is a hope, not a plan. As AWS puts it, "You need to practice the failover to know that it will work when you need it."
Then decide on a second region, system by system. Spend it where the hourly cost of being down is larger than the cost of standing by.
We design cloud architecture on AWS and Azure for the bad day as well as the normal ones. If you want to know what a region failure would cost your business, let's talk.