AT&T’s Failed Network Update Disrupts Millions of Calls and Emergency Services

A recent government investigation has uncovered more details about the widespread AT&T outage on February 22, 2024. The nationwide disruption, which occurred immediately following an erroneous network update, resulted in over 92 million phone calls being blocked, including over 25,000 emergency calls to 911.

The Federal Communications Commission (FCC) criticized AT&T for not adhering to best practices, which recommend that network changes be thoroughly tested, reviewed, and approved before implementation. Service restoration took more than 12 hours, affecting over 125 million devices and customers, including those using the First Responder Network Authority (FirstNet). The full impact of the outage is detailed in the FCC report.

AT&T acknowledged the issue, stating that it was due to the application of an incorrect process related to network expansion—a claim further supported by the FCC’s findings. Despite rolling back the network change within two hours, it took longer to fully restore service due to an overload of re-registration requests from disconnected devices.

The FCC investigation identified several systemic problems within AT&T’s processes, which heightened the likelihood of an outage and complicated recovery efforts. The absence of adequate peer review and insufficient oversight and controls allowed a misconfigured network element to be introduced, exacerbating the situation.

AT&T has since pledged to implement changes to prevent future occurrences, emphasizing their commitment to higher standards and expressing regret for failing to meet customer and public safety expectations. The FCC Enforcement Bureau is currently evaluating potential violations of FCC rules, with possible penalties on the horizon. Notably, Verizon Wireless recently agreed to a $1,050,000 fine for a similar outage caused by a botched update.

For a detailed account of the FCC’s findings, visit the official report.