Thursday August 16, 2018

Incident

Problems with search and Posts for some workspaces

Technical Summary:
A configuration change at 9:19 am PT caused teams created between 9:36 am PT and 10:27 am PT to be misconfigured. For these workspaces, Slack was unusable, including degraded search and post functionality.

This misconfiguration was discovered at 4:06 pm PT and updated all the necessary configurations for these teams. By 6:01 pm PT, everything had been updated and the all clear was sounded.

We are now working on improvements to ensure this doesn’t happen in the future.

7:21 PM PDT - Aug 16th・See in your timezone

We've been given the all-clear from the team! The issues with search, Posts, and workspace management have now been fixed. To those affected, we are truly sorry for taking time out of your day. Thank you for your patience and understanding.

6:01 PM PDT - Aug 16th・See in your timezone

We're happy to report that we are seeing positive signs to the changes we've made so far. We hope to bring more positive news to you soon.

5:53 PM PDT - Aug 16th・See in your timezone

Thank you for sticking with us on this. We don't yet have further information to share but this is the highest priority for the team right now.

5:23 PM PDT - Aug 16th・See in your timezone

We don't have any news to share just yet, but our team is continuing to work on sorting out these issues. We hope to have a positive update for you soon.

4:52 PM PDT - Aug 16th・See in your timezone

Some workspaces may be having trouble with search and other features, such as Posts at this time. We are very sorry for this issue, and our team is actively working to resolve this.

4:06 PM PDT - Aug 16th・See in your timezone

Services affected

Posts/Files

Search

Workspace/Org Administration

Status

Resolved

Incident

Channel membership issues on Enterprise Grid

Technical Summary:
On August 16 at 8:01 AM (PT), we received reports that users were unable to access member lists for channels that had been converted from public to private. We were able to identify the issue as being related to a server-side cache of this data, and were subsequently able to address this at 10:53 AM (PT). We are taking steps to prevent this in the future.

6:11 PM PDT - Aug 16th・See in your timezone

We have good news! We've deployed a fix for the issue with the channel member list on Enterprise Grid workspaces after a reload (CMD/CTRL + R). We appreciate your patience while we got this resolved.

10:53 AM PDT - Aug 16th・See in your timezone

We don't have any news to share at this time about the membership list. We'll provide you with updates as we have them. Thanks for bearing with us.

9:36 AM PDT - Aug 16th・See in your timezone

Our team is looking into the issues preventing private channel membership from displaying in Enterprise Grid workspaces.

9:06 AM PDT - Aug 16th・See in your timezone

We're still narrowing down the cause of problems with displaying channel membership in private channels on Enterprise Grid. We're sorry for the inconvenience.

8:36 AM PDT - Aug 16th・See in your timezone

We've received reports that on some Enterprise Grid workspaces, member lists are not loading in private channels. We're on the case. Sorry for the trouble.

8:01 AM PDT - Aug 16th・See in your timezone

Services affected

Workspace/Org Administration

Status

Resolved

Outage

Connectivity issues

Technical Summary:
On August 16th, 2018 between 9:19 am PDT and 9:37 am PDT, Slack experienced an 18 minute outage, where all users could not use the Slack product.

A configuration change at 9:19 am PDT prevented our web infrastructure from communicating with a subset of our database infrastructure. This resulted in a complete loss of Slack functionality until the change was reverted at 9:37 am PDT. We called the all clear at 11:07 am PDT.

We are currently hard at work on scoping preventive measures to ensure this doesn't happen in the future.

4:48 PM PDT - Aug 16th・See in your timezone

Thanks for your patience and understanding while we worked to resolve the connection problems. Everyone should now be able to connect once again, but please let us know if you run into any further problems at feedback@slack.com.

11:07 AM PDT - Aug 16th・See in your timezone

While some folks are able to connect successfully once again, we're not out of the woods just yet. We hope to have some good news for you soon.

10:20 AM PDT - Aug 16th・See in your timezone

We're still working to ensure this is fully resolved, however folks may be seeing improvements at this time. Thanks for bearing with us.

9:37 AM PDT - Aug 16th・See in your timezone

We're investigating connectivity issues impacting all workspaces. Our team is on the case, and we are very sorry for the disruption.

9:19 AM PDT - Aug 16th・See in your timezone

Services affected

Connections

Apps/Integrations/APIs

Status

Resolved

Incident

User group mentions aren't generating notifications

Technical Summary:
On August 14th at 07:19 AM (PT), We received reports that user group notifications were not working for some users. This was identified as part of an known bug, and we have closed this event.

4:37 PM PDT - Aug 16th・See in your timezone

User group mentions are now successfully generating notifications again. Thanks for sticking with us.

8:35 AM PDT - Aug 16th・See in your timezone

We've received reports of user group notifications failing for some workspaces. We're on the case to find a solution — apologies about this disruption.

7:19 AM PDT - Aug 16th・See in your timezone

Services affected

Notifications

Status

Resolved