Service Issue in Platform affecting EU datacentre.

Incident Report for Vonage API Status

Postmortem

Date/Time Impacted

2025-07-29 08:01 UTC until 2025-07-29 9:26 UTC

‌

Services Impacted

SMS API

Voice API

Messages API

‌

Summary

An unintended change was made to Web Application Firewall (WAF) rules in the API infrastructure in the EU West region, which resulted in service disruptions to API requests, which were rejected with a 403 response code.

At 8.36 UTC, our monitoring alerted us to an issue impacting SMS API & Voice API.

Some customers experienced failures to send messages, delayed read receipts from WhatsApp messages, and a small number of Voice API request failures. Voice call transcriptions were also failing or delayed.

Once customer impact was identified, the affected infrastructure was removed from service at 9.26 UTC, marking the end of customer impact. The problematic firewall rule was then identified and deleted.

Root Cause

A rule was added to the Web Application Firewall (WAF) used by the API infrastructure in the EU West region. This rule inadvertently implemented a rate limit of 1500 requests, per 5 minutes, per source IP address. This affected customers and suppliers who send large volumes of HTTP requests.

‌

Restoration Actions

The affected infrastructure was initially removed from service at 09.26 UTC. We subsequently identified and deleted the problematic rule, and the infrastructure was returned to service.

‌

Next Steps

Improve our monitoring and alerting mechanisms to capture an enhanced level of key metrics, enabling faster issue identification.
Review our access control guidelines

‌

We value your feedback and want to do the best we can for you. Please let us know how we did or your ideas for how we can improve our incident communication processes through the following short survey: https://forms.gle/oHG3KinmGarhpvU78

Posted Jul 31, 2025 - 12:15 UTC

Resolved

We can confirm this problem has been resolved, and all services have been restored.

The following services were impacted from [29-07-2025 08:01] UTC until [29-07-2025 09:26] UTC:

API requests handled by our EU Region were being rejected with HTTP 403.

Impacted Service-
SMS API
Voice API
Messages API

Please let support@api.vonage.com know if you continue to experience problems with this.

A post-mortem will be published in the next few days.

Posted Jul 29, 2025 - 11:27 UTC

Update

We have implemented a fix for this issue.

Customers in the EU region using our APIs may have experienced 403 errors from [29-07-2025 08:01] UTC until [29-07-2025 09:26] UTC

The list of impacted services has been confirmed as follows:

SMS API
Voice API
Messages API

We will continue to monitor services during the next few hours and post updates should anything change.

Posted Jul 29, 2025 - 09:55 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Jul 29, 2025 - 09:46 UTC

Update

We are continuing to investigate this issue.

Posted Jul 29, 2025 - 09:05 UTC

Investigating

Our monitoring system has alerted us to a potential service issue with our platform. Some API requests handled by our EU datacentre are being rejected with HTTP 403.

We are alerting customers immediately while our engineering teams investigate.

If there is a customer impact, it is likely to affect the following products:

- Voice
- SMS
- Messaging
- Other API services which are processed in the EU datacentre

We aim to provide an update on our investigation within 1 hour.

Please see this article - https://api.support.vonage.com/hc/en-us/articles/6749696044828-Vonage-API-Support-Incident-Handling - for an explanation of our approach to publishing incidents.

Posted Jul 29, 2025 - 09:00 UTC

This incident affected: SMS & Messages (SMS, Messages API), Multiple Services Affected (Multiple Services Affected), Voice (Voice API), and Global Connectivity (SMS Europe, Voice Europe).