← Incidents

DEGRADEDAPI Service Issues

Feb 27, 10:30 - 10:45 AMResolved after 15 minutesAffected: API

Updates:

Last week, the Postmark APIs became sporadically unavailable. The root cause of this was due to saturation of some of our underlying physical infrastructure. We have been in the process of migrating our core systems to a cloud provider in order to better respond to these types of spikes. This process is on-going, and the API has not yet been migrated.

To respond to the increased load to the Postmark API last week, we have done the following:

  • We doubled our API capacity.
  • We have shifted lower-priority workloads to our cloud provider to reduce saturation on our physical infrastructure.
  • We implemented additional traffic-shaping rules in our load balancers to isolate traffic spikes so that they cannot cause our API cluster to cascade into failure.

In addition, we have been planning to add to new public endpoints for our API. This work was already planned, but we are prioritizing. If your application restricts access to public IPs, please ensure that the following IPs are whitelisted:

  • 3.137.63.180
  • 18.216.134.80

We intend to start moving API traffic to these new IPs on March 16, 2020, so please make sure these IPs are allowed by any firewall or routing rules you may be using.

Incident’s History

We have identified the issue and the API should be returning to normal. We'll be monitoring closely and will update again shortly.

UP

API is back to normal and we are taking measure to help mitigate similar issues.

DEGRADED

We are investigating issues with our API and will be posting more details shortly. Thank you for your patience.