Starting around 6:00 a.m. PST on January 4, 2021, some customers started experiencing occasional errors and increased latency while using Slack. Around 7:00 a.m. PST there was a rapid increase in errors and Slack wasn’t usable for all customers.
Around 8:13 a.m. PST, we addressed an issue with our provisioning service and began provisioning healthy servers once again to address traffic requests. From there, at 8:45 a.m. PST, some customers began to see improvements, but others who were trying to launch their Slack clients were unable to do so. By around 9:15 a.m. PST most customers were able to use Slack again. We continued to experience elevated errors until 10:40 a.m. PST, after which all customers were able to use Slack again.
We also discovered some customers were stuck on a webpage in the Slack desktop app. This is a separate bug that’s being investigated, but was heightened during the outage. Troubleshooting steps, such as restarting, forcing quitting Slack from Activity Monitor or Task Manager, or clearing cache, allowed affected customers to access the app once again.
We know our customers rely on Slack. Our top priorities are working alongside our cloud vendors to fully understand the underlying causes of the outage and putting safeguards in place to prevent similar problems from happening again. If you’d like to receive a full Root Cause Analysis (RCA) report, please reach out to firstname.lastname@example.org to request one.
5:04 PM PST
For any customers unable to access Slack, please try refreshing by using CMD/CTRL + SHIFT + R. It may also help to try closing Slack via the Task Manager or Activity Monitor. If you encounter any trouble, please let us know at email@example.com.
We'll have a separate post on this site for the outstanding calendar apps and email notifications trouble.
We're truly sorry for the disruption today. We'll be back with a summary of this issue.
Note: We've edited this message to ask users to refresh their clients entirely, as opposed to the reload instructions we shared before (CMD/CTRL + R). It's also been edited to ask users to try force closing their app.
12:10 PM PST
We're seeing improvements with error rates on our side, and we believe affected customers should be able to access Slack.
We're continuing to work to resolve the trouble with calendar apps and email notifications.
11:28 AM PST
For those customers who are unable to access Slack (e.g. seeing a blank screen or error), please try reloading (CTRL/CMD + R). If you can access Slack, please hold off on reloading at this time.
In addition, Google Calendar and Outlook Calendar aren't working as expected, and some email notifications (e.g. a notification of a new direct message in Slack) are failing to send.
Note: We've updated this message to specify holding off on reloading if you can access Slack.
10:51 AM PST
Some customers may be able to connect, but may also experience degraded performance. We're continuing to work to resolve the issue.
10:00 AM PST
While the issue is largely still ongoing, we believe some customers may see improvement in connecting to Slack after a refresh (CTRL/CMD + R).
9:29 AM PST
There are no changes to report as of yet. We're still all hands on deck and continuing to dig in on our side. We'll continue to share updates every 30 minutes until the incident has been downgraded.
8:52 AM PST
We're continuing to investigate connection issues for customers, and have upgraded the incident on our side to reflect an outage in service. All hands are on deck on our end to further investigate. We'll be back in a half hour to keep you posted.
8:20 AM PST
We’re still investigating the ongoing connectivity issues with Slack. There's no additional information to share just yet, but we’ll follow up in 30 minutes. Thanks for bearing with us.
7:44 AM PST
Customers may have trouble loading channels or connecting to Slack at this time. Our team is investigating and we'll follow up with more information as soon as we have it. We apologize for any disruption caused.
7:14 AM PST