Saturday October 13, 2018

Incident

Issues with notifications, unfurls, Slackbot custom responses, and Events API for all customers

Technical Summary:

On October 13th between 5:43pm PT and 6:51pm PT a group of servers were failing to process some tasks. This impacted a number of features for all customers, including notifications, unfurls, custom responses and our Events API (which some apps and integrations rely on).

We identified issues with the health of some servers involved which were causing these processes to fail. At 6:40pm PT we restarted one of the servers and things started recovering. By 6:51pm PT, tasks that had queued up during the incident were cleared and service was back to normal.

We’re investigating the root cause, as well as making plans to ensure this doesn’t happen in the future.

5:39 PM PDT - Oct 14th・See in your timezone

Thanks for your patience while we worked through this one. All functionality should be back to normal now. We're sorry for the disruptions and are continuing to investigate what went wrong to ensure this doesn't happen again.

7:17 PM PDT - Oct 13th・See in your timezone

Things are improving for notifications and unfurls, but we're not out of the woods yet. Customers may have experienced issues with some apps and integrations. We're seeing signs of things improving.

7:08 PM PDT - Oct 13th・See in your timezone

We're still working to fix the problem with notifications and unfurls. At the moment, notifications are not being received, and unfurls are not working. We are continuing to investigate the issue.

6:36 PM PDT - Oct 13th・See in your timezone

There’s an issue with notifications, and its not working properly. Our team is on the case, and we'll have an update for you soon.

5:43 PM PDT - Oct 13th・See in your timezone

Services affected

Link Previews

Notifications

Apps/Integrations/APIs

Status

Resolved