Tonal outage, 25 May 2022

Cpu Utilization spiked to near saturation potentially leading to incident with higher db response times

Started
Resolved
25 May 2022, 23:39 UTC
Impact
Not stated

Last update from Tonal

Our care admin tool has a leak where in if someone puts in a 0 it leads to a massive table scan on the backend database. Immediate resolution was switching read endpoints from write to read db. Long term: backend team would consider adding an Elastic Search instance to our stack to support search through our user_profile, devices, subscriber and subscription models.

25 May 2022, 23:39 UTC

Read the full incident on the official Tonal status page

Is Tonal down now?

This page is the record of one past incident. For the live status, see Is Tonal down right now? - it reads the official status page every 2 minutes.

Other Tonal incidents

  1. Elevated API Errors due to a networking issue

  2. This is an example incident

All recorded Tonal incidents