MyPage is unavailable [Restored]
Between 17:56(JST) and 19:06(JST) on Friday, March 6, 2026, it was not possible to log in to MyPage or perform any operations on MyPage. We have now confirmed that normal operation has been restored.
We would like to report the details of this matter as follows:
1. Overview of the issue
- Date and time of occurrence: Friday, March 6, 2026, 17:56(JST) - 19:06(JST)
- Affected: All users
- Target service: AmiVoice API MyPage
- Symptom: Unable to log in to MyPage and all functions within MyPage are unavailable
2. Reason
In order to resolve inconsistencies (drift) between the infrastructure configuration and the actual environment, we were updating our load balancers and deleting unnecessary ones. During this process, we deleted a stack (a unit of configuration) that contained an unnecessary load balancer, which inadvertently deleted another internal load balancer that was included in the stack. This meant that we were no longer able to call the functions required for session management on MyPage, and as a result, MyPage became unavailable.
This time, prior to execution, the operation was reviewed by multiple people and by AI, and it was determined that there were no problems before it was carried out. However, there was insufficient checking of the resources included in the stack of unnecessary load balancers, such as checking stack dependencies.
In addition, the load balancer providing the actual service did not have deletion protection configured, and a human error resulted in the deletion of resources used in the production environment, leading to an outage. If deletion protection had been configured, the problem could have been avoided.
3. Countermeasures
We deployed it again and the issue was resolved.
4. Measures to prevent recurrence
We will take drastic measures to prevent recurrence.
Configuring deletion protection for an ALB
The ALB in question did not have deletion protection enabled. We will enable deletion protection for all ALBs, not just the affected ALB, and will modify the CDK code and update the development guidelines.
Improving reviews
This was because the review was unable to determine whether the changes and dependent resources were necessary. The AI review also did not define the items to be checked at the time of deployment. Going forward, we will improve the AI review by checking changes, actual resource dependencies, and usage status.
Once again, we would like to apologize for the inconvenience caused over such a long period of time. We will continue to strive to ensure stable operation and quality improvement of our services, so we appreciate your continued support.