Regarding the connection error for speech recognition (Conversation_General-purpose, 8kHz) (August 29, 2026) [Resolved]
Thank you for your continued use of the AmiVoice API.
We would like to report on the cause and preventative measures regarding the connection failure of "Conversation_General-purpose (8kHz)" that occurred on August 29, 2026.
Regarding this incident, we announced it on Statuspage at 6:22 AM on August 29, 2026, during the occurrence of the failure, and marked it as Resolved at 7:57 AM when recovery was confirmed. In this announcement, we will report on the cause and measures to prevent recurrence based on the subsequent investigation results.
Date and Time of Occurrence
August 29, 2026 (Saturday), approximately 6:16 AM to 7:56 AM (about 1 hour and 40 minutes)
Impact Range
During the aforementioned outage, all requests using the "Conversation_General-purpose" speech recognition engine with 8kHz speech were affected.
- The connection to the speech recognition server failed, preventing speech recognition from starting.
- If you are using "Conversation_General-purpose" with 16kHz audio, or if you are using a different engine, this issue has not had any effect.
Incident Detection and Recovery
At 6:22 AM, the monitoring system detected a series of failed speech recognition tests and automatically notified users of the failure via Statuspage.
Subsequently, automatic scaling activated a new speech recognition server, and the system was restored to a normal connection state around 7:56 AM. The monitoring system confirmed the success of the speech recognition test, and the Status page was updated to Resolved at 7:57 AM.
Cause
The issue stemmed from a large number of sessions where, after a session was established via a speech recognition request, no voice data was sent, causing these sessions to accumulate on the speech recognition server. While these sessions are designed to be sequentially disconnected after a 60-second timeout, the rate at which new connections were increasing exceeded the rate at which disconnections were occurring.
We were aware of this issue in advance and had taken temporary measures by reviewing the server's memory settings and the number of connections it could accept. We also planned to implement a more fundamental solution between September 1st and 3rd. However, before we could apply the solution to the production environment, a large number of sessions occurred when the number of active speech recognition servers was reduced due to a scale-in caused by decreased demand at night. This resulted in insufficient memory on all speech recognition servers providing the engine, making it impossible to accept connections, and ultimately leading to this outage.
Recovery
The service was restored around 7:56 AM after a new speech recognition server was started up.
Recurrence Prevention
In response to this issue, we are taking the following steps:
- Originally scheduled for September 1st-3rd, we will be modifying the number of connections accepted per server and the memory settings to prevent the speech recognition server from running out of memory even if this issue occurs.
- We will ensure that our servers have sufficient processing power to handle sudden increases in connections, even during off-peak hours such as at night. This measure was implemented on August 31st.
- We will strengthen the mechanism that, when a process anomaly is detected in the speech recognition server, promptly disconnects the abnormal server and replaces it with a healthy server.
We will continue to strengthen our monitoring and recovery systems in the event of a failure, and strive to provide stable services.
We sincerely apologize for the great inconvenience caused to our customers.