Hex outage, 13 May 2025
Degraded kernel acquisition
- Started
- Resolved
- 13 May 2025, 19:10 UTC (after 4 h 32 min)
- Impact
- Critical
- Affected
- Kernels
Last update from Hex
After fully analyzing the timeline of events, **we’ve confirmed that the root cause of the incident was an issue in the AWS EBS storage system backing our primary database replica**. The issue began on May 12 11:00 PM PDT according to AWS, and the effect was gradual degradation of performance on the replica, eventually leading to a cascade where even our primary database got backpressured on critical write operations, which triggered the instability causing the incident. The AWS issue was resolved May 13 9:32 AM PDT, after we pushed a database configuration change that caused the EBS system software to reset. We then stabilized and cleaned up systems on our end before resolving the incident....
Is Hex down now?
This page is the record of one past incident. For the live status, see Is Hex down right now? - it reads the official status page every 2 minutes.