Thursday April 4, 2019

Incident

Workspace Admins and Owners may be unable to search for some members on the 'Manage members' page

Issue Summary:

On April 4th, we received reports from our users that they were unable to find members of their workspace on the Manage Members page. During our investigation, we realized these issues stemmed from a misconfiguration in our database in early March.

Our engineering team began working on a fix and on April 8th at 10:16am PDT, a new configuration was implemented. This resolved the issue and brought full functionality back to the Manage Members page.

We’re currently investigating preventive measures to ensure that this doesn’t happen again in the future. If you’re still running into any issues, please reach out to us at feedback@slack.com.

4:37 PM PST

Our engineers have narrowed down the issues with the 'Manage members' page. If you're experiencing any troubles with this particular page, please reach out to us at feedback@slack.com.

11:26 PM PST

Workspace Admins and Owners may be unable to search for some members of their workspace on the Manage Members page. We're investigating, and we're very sorry for the disruption.

6:12 PM PST

Services affected

Workspace/Org Administration

Status

Resolved

Outage

Customers are experiencing degraded functionality

Issue Summary:

At 7:25pm PT on April 4th, we changed the configuration for one of our core services which caused it to break. As the majority of our functionalities depend on this service, we faced problems across the board.

Our engineers worked on a fix, and an all clear was called at 2:39am PT.

4:08 AM PST

We've got good news: we're in the clear and the issue has been fixed. If you don’t automatically reconnect, refresh your workspace using Ctrl/Cmd + R. We're sorry for any trouble this caused. Thanks for sticking with us.

1:51 AM PST

Sorry for any disruption to your day. Our team is still working to fully resolve the issue. Once we have more information, we'll be sure to update you!

1:08 AM PST

We do not have new information to share, but we are still working to make sure our systems are running smoothly.

12:37 AM PST

Our engineers are working to make sure systems are in a healthy state. If you're still seeing significant issues with Slack functionality, please reach out to us at feedback@slack.com.

11:54 PM PST

We're continuing our work in the background to ensure things are running as smoothly as possible across all fronts.

11:23 PM PST

We have no new discoveries to share, but we're keeping a watchful eye on impacts to your experience with these features.

10:45 PM PST

We’re seeing positive signs that functionality is returning to normal, but we still have work to do. Please continue to let us know if you're experiencing specific issues with these features through feedback@slack.com.

10:11 PM PST

All hands are working to fix this. We're seeing improvements in certain areas, and we're continuing our efforts.

9:37 PM PST

We believe we're on the right track for a solution, but we're not out of the woods. Thanks for sticking with us.

9:05 PM PST

We don’t have additional information to share, but we're laser focused on fixing these features. We'll continue to update you on our progress.

8:32 PM PST

We've discovered an issue with the checkout page for workspace upgrades. Our engineers are still looking into this and the other issues. Thanks for bearing with us.

8:02 PM PST

We're still exploring the cause of these issues and we've seen that API calls may be failing as well. We'll be back in 30 minutes with another update.

7:32 PM PST

We’re still investigating the issues with these key features, especially with messages failing to send. We’re so sorry for the interruption to your day.

7:01 PM PST

All customers may be facing issues with file uploads, unfurls, custom responses and other features. We are actively investigating. Thank you for your patience, and sorry for the disruption.

6:31 PM PST

Services affected

Apps/Integrations/APIs

Workspace/Org Administration

Messaging

Link Previews

Posts/Files

Status

Resolved