Below is a message from our co-founder and CEO, Natalie Nagele, about recent service issues.
Hi there,
I’d like to apologize for the issues on Postmark, beginning at the end of August. As you probably noticed, we’ve been having sporadic issues with various parts of the service, including intermittent API timeouts, Inbound delays, and Outbound delays. Some Outbound delays were quite short, but there were several that lasted way too long where emails sat in queue. It’s been a really bumpy road for you, and I’m sorry.
Earlier this month we noticed some degraded performance across our APIs and services, mostly coming from database contention. We began to investigate and repair, which made some improvements temporarily. While we repaired, we worked on identifying the overarching issue(s) that could be causing this degradation. We’ve been all hands on deck, working day and night getting ourselves back to the reliable service you expect from us.
As part of our investigation we worked with Percona to audit our databases. They found some ways to improve, and we found some solutions in the application ourselves. So last night we performed a last-minute maintenance that required us to intentionally queue emails for under 30 minutes. We waited until sending was at its lowest all day (10pm EST) and performed the maintenance successfully. All night and this morning we monitored performance and feel confident that we’re back to a reliable state.
Our reliability team has been working on some major improvements to our infrastructure the last four months to handle the planned growth we’ve seen. With these planned updates, and the updates we’ve made in the last week, we’ll be able to continue growing with you and maintain the top performance you expect from us.
We know this has been rough for you, especially when it comes to explaining issues to your own customers. I want you to know how much we care and to promise that we take this very seriously.
If you have any questions, please don’t hesitate to email us.
With tremendous respect,
Natalie
Co-founder and CEO
Incident’s History
All queues are clear and sending is back to normal. Thank you for your patience during the maintenance window.
We have successfully completed our server maintenance, and we have started processing queued messages.
We have started our planned server maintenance, and all messages are now being accepted and queued. We will update this incident page as we go.
Due to recent service issues we will be performing server maintenance tonight, September 11, 2018, at 10pm Eastern Time.