Running a global observability platform means one thing above all: your infrastructure must never go down. When you're responsible for monitoring thousands of customers' applications 24/7, network failures aren't just inconvenient, they're existential threats. At New Relic, hundreds of clusters run on multiple clouds, and regions. These clusters depend on a complex web of network connections: regional transit gateways, inter-regional hubs, and cross-cloud links.
Signal, like many other internet services, failed briefly during the sizable AWS outage that occurred on October 19 and 20. The cause, as AWS explained in its paragraph-starved post-mortem last week, was an error in AWS' automated DNS management system. And the loss of availability and productivity across the many AWS-dependent businesses has been estimated to have cost businesses more than a hundred billion dollars.