DEGRADEDAPI Service Issues
Last week, the Postmark APIs became sporadically unavailable. The root cause of this was due to saturation of some of our underlying physical infrastructure. We have been in the process of migrating our core systems to a cloud provider in order to better respond to these types of spikes. This process is on-going, and the API has not yet been migrated.
To respond to the increased load to the Postmark API last week, we have done the following:
- We doubled our API capacity.
- We have shifted lower-priority workloads to our cloud provider to reduce saturation on our physical infrastructure.
- We implemented additional traffic-shaping rules in our load balancers to isolate traffic spikes so that they cannot cause our API cluster to cascade into failure.
In addition, we have been planning to add to new public endpoints for our API. This work was already planned, but we are prioritizing. If your application restricts access to public IPs, please ensure that the following IPs are whitelisted:
We intend to start moving API traffic to these new IPs on March 16, 2020, so please make sure these IPs are allowed by any firewall or routing rules you may be using.