Wednesday November 23, 2022

Issue summary:

From November 23, 2022 at 7:30pm PST to November 23, 2022 at 8:22pm PST, some users were unable to send messages, load content or use workflows within Slack. We discovered the issue was related to a recent change which accidentally severed access to a number of endpoints. A network change was made and most services started to auto-recover and work again.

During the restore process some users experienced search errors or slow responses within the search bar. This was resolved on November 24, 2022 at 5:58pm PST, restoring functionality for all affected users.

6:10 AM PST

We’ve resolved the issue with search for all impacted users. We’re sorry for the disruption caused, and thank you for your patience.

6:53 PM PST

All users should now be able to search for messages that were created prior to 12 hours ago. We anticipate continued improvement over the next few hours for searching messages created within the last 12 hours. Some users may continue to experience issues when searching for files and we're continuing our investigation in this area. We will provide an update once we have additional details to share. We want to sincerely thank everyone for their patience.

1:40 PM PST

We want to thank you for your patience and apologize for any continued disruptions. We have no further information at this time and will be back in 30 minutes with more updates.

11:56 AM PST

Thank you for sticking with us throughout this and our apologies for any continued trouble. We are seeing gradual improvements to the message search and will be back in another 30 minutes with further updates.

11:25 AM PST

While we are seeing further signs of improvement, we have no new information to share — please stand by for another update in 30 minutes.

10:55 AM PST

We're continuing to make progress however no new updates to share at this time. We appreciate your patience and apologize for the ongoing disruption.

10:25 AM PST

Progress is looking good, but we’re not quite at the finish line. The work continues as we get closer to resolving this issue for you. Stay tuned, we’ll post another update in 30 minutes. Thanks for your patience.

9:55 AM PST

No new updates to share at this point, but we are still working away to get things back in order for you. We apologize for the continued trouble.

9:30 AM PST

The work is continuing towards a fix and we are slowly seeing improvements. We aren’t out of the woods just yet, so we will keep you updated in another 30 minutes on how we’re doing with our investigation. Thanks for sticking with us.

8:59 AM PST

While we don't have additional news to share, we are continuing to work through this issue and making progress. Our apologies for the ongoing disruption. We are working hard to fix this as soon as we can.

8:28 AM PST

As part of the ongoing issue users are experiencing with search, we've found that users may also experience difficulty when trying to view their sent messages in the "Drafts & sent" tab. We're still actively working towards a solution, and we'll have another update for you in 30 minutes. Thank you for your patience.

7:59 AM PST

We've identified some possible solutions for the ongoing search issues users may be experiencing and are continuing to work towards a fix. We'll be back with another update in 30 minutes, and we apologize for any inconvenience this is causing.

7:25 AM PST

While we don't have any new information to share about the ongoing search issues since our last update, we are still actively working on addressing this issue and will continue to provide updates every 30 minutes as we investigate further. Thank you for your patience.

6:54 AM PST

We've found that searching for messages and files may be slow, and searching for members/groups may fail. Additionally, any recent changes (~7 hours) to channels/files will not appear in searches or the Admin Dashboard. We'll continue to keep you posted every 30 minutes as we continue our investigation.

6:25 AM PST

Our investigation is still ongoing but we're focused on getting message, file and recently added member searches back to normal. We'll continue to share updates every 30 minutes. We apologise for the continued trouble.

5:51 AM PST

No additional news to share just yet regarding searching messages, files and recently added members, but we're focused on getting things back to normal as quickly as we can. We will share an update in the next 30 minutes. Thank you again for your patience as we work towards a fix.

5:21 AM PST

We're still actively investigating this issue regarding message search, files search, and searching for members and groups recently added to the workspace. We don't have any new information to share at this time. Thanks for sticking with us as we continue to work towards a fix.

4:52 AM PST

Message search is slow for all customers, and returning incomplete/partial data for EKM customers. Files search is not working for most customers. Searching for members or user groups added to the workspace over the last 3-5 hours is also impacted. We're still working to fix this as fast as we can. Apologies for the disruption to your work.

4:19 AM PST

Message search is slow for all customers, and returning incomplete/partial data for EKM customers. Files search is not working for most customers. Searching for members added to the workspace over the last 3-5 hours is also impacted. We're still working to fix this as fast as we can. Apologies for the disruption to your work.

3:48 AM PST

Message search is slow for all customers, and returning incomplete/partial data for EKM customers. Files search is not working for most customers. We're still working to fix this as fast as we can. Apologies for the disruption to your work.

3:01 AM PST

Message search is slow for all customers, and returning incomplete/partial data for EKM customers. Files search is not working for most customers. We're still working to fix this as fast as we can. Apologies for the disruption to your work.

2:29 AM PST

Message search is slow for all customers, and returning partial data for EKM customers. Files search is not working for most customers. We're working to fix this as fast as we can. Apologies for the disruption to your work.

1:48 AM PST

Some customers may be experiencing issues with the search in Slack. We're working to fix this as fast as we can. Apologies for the disruption to your work.

12:18 AM PST

Some customers may be experiencing search issues in Slack. We're very sorry for the disruption, and we're working to resolve this as fast as we can.

11:46 PM PST

We're still working to resolve the search issues. We'll share another update as soon as we have more information. Thank you for bearing with us.

11:09 PM PST

Our work on the search issue is still ongoing at this time. We’ll be back with another update in 30 minutes.

10:26 PM PST

We're still working to resolve the issues with search. We don't have any new information to share at this stage, but we'll provide another update in 30 minutes.

9:56 PM PST

We have restored functionality to many of the affected features, including sending and receiving messages, huddles, and workflows.

We are still investigating some issues with search in Slack, and we apologise for the continued disruption. We'll be back with an update in 30 minutes.

9:27 PM PST

We are seeing further signs of improvement, but we are still monitoring the situation and working to completely restore all affected functionality.

We'll share another update in 30 minutes or as soon as we can confirm the issue has been fully resolved, whichever is soonest.

8:51 PM PST

Customers may also be experiencing trouble with huddles and workflows. However, we are beginning to see steady improvements as we work to resolve this issue.

We’ll continue to provide updates every 30 minutes.

8:28 PM PST

Some customers may be experiencing issues loading content or sending new messages. We are investigating and we're very sorry for the disruption.

We'll be back with an update in 30 minutes or as soon as we have more news to share.

7:58 PM PST