Service Issue in Vonage API Platform

Incident Report for Vonage API Status

Postmortem

Voice & SMS Call Failures

 

Date/Time Impacted

Incident Started:  2022/12/13 21:38 UTC

Incident Recovered: 2022/12/13  22:12 UTC

‌

Services Impacted

Voice & SMS

‌

Summary

Voice & SMS-related calls started to fail in the Asia-Pacific Southeast Region, starting at 21:38 UTC on December 13, 2022. The issue was resolved at 22:12 UTC on December 13, 2022. 

‌

Root Cause

Our Cloud Provider in the Asia-Pacific Southeast Region was experiencing connectivity issues starting at 21:38 UTC on December 13, 2022. Several services of the Vonage API Platform rely on the connectivity to that Cloud Provider, and the disruption in connectivity affected those managed services. This resulted in Voice & SMS-managed services being unable to function correctly.

‌

Restoration Actions

  • Failed over the calls to the US East Region
  • Cloud Provider acknowledged and fixed the connectivity issues.

‌

Next Steps

  • To enable the manual failover to another data center in the paired region
  • To load balance the traffic between different regions

We value your feedback and want to do the best we can for you. Please let us know how we did or your ideas for how we can improve our incident communication processes through the following short survey: https://forms.gle/Eizg5CKM42ZVPk5v9

Posted Dec 15, 2022 - 22:27 UTC

Resolved

All services have been restored.
Please let support@api.vonage.com know if you continue to experience problems with this.
A post-mortem will be published in the next few days.
Posted Dec 14, 2022 - 09:22 UTC

Monitoring

Our Cloud Provider had a connectivity issue in the Asia Pacific South-east region which caused the issue. The issue is resolved now, and Engineering is monitoring the issue.
Posted Dec 13, 2022 - 22:39 UTC

Investigating

Our monitoring has alerted us to a potential service issue with our platform. Voice and SMS API Requests in the Asia Pacific region are impacted, with API requests getting rejected with 4XX and 5XX errors. The issue started at 21:38 UTC on December 13, 2022. Engineering is currently investigating the issue.

If there is a customer impact, it is likely to be affecting the following products:
Voice API
SMS API

We aim to provide an update on our investigation within 1 hour.

Please see this article - https://api.support.vonage.com/hc/en-us/articles/6749696044828-Vonage-API-Support-Incident-Handling - for an explanation of our approach to publishing incidents.
Posted Dec 13, 2022 - 22:17 UTC
This incident affected: SMS & Messages (SMS) and Voice (Voice API).