Thursday July 29, 2021

Incident

Trouble with search

Issue summary:
On July 29, 2021 starting around 12:09 a.m. PDT, some users encountered a "Something went wrong" error when searching in Slack.

We discovered a recent enhancement to Japanese language search requests led to certain queries interacting poorly with relevance rankings. This resulted in errors in our server infrastructure, causing other search queries to fail.

We restarted servers to resolve the issue for all users by 1:53 a.m. PDT, and adjusted configurations in our search infrastructure to prevent recurrence.

While search functionality had been restored, a backlog of messages were not indexed during the incident. This resulted in some messages not being returned in search results until all messages were fully processed by 12:10 p.m. PDT.

2:31 PM PST

We've rolled out a fix and search is back to normal. No additional action is required for this change to be reflected on your end. Thank you for your patience while we sorted out this issue.

12:41 PM PST

Our search success rate has recovered, but some users may still not see the most recent results in their searches. No additional information to share just yet, but we'll keep you posted as soon we have an update.

7:28 AM PST

We have identified the root cause of this issue with search which may cause some users to not see the most recent results in their searches. We're focused on getting things back to normal as quickly as we can and we'll provide another update as soon as we have news.

4:04 AM PST

We’ve identified the cause of the search issue and we’re working on getting a fix out as soon as possible. Thank you for your patience.

1:44 AM PST

Some customers may be experiencing issues search. Our team is currently investigating and we'll provide another update as soon as we have more information to share.

1:07 AM PST

Services affected

Search

Status

Incident