On February 8, 2021 at 3:15 p.m. to 4:19 p.m. PST, some customers may have experienced intermittent search failures.
We determined that a recent change involving a new set of hosts for one of our services was the source of the issue. Our search infrastructure failed to redirect searches away from the now outdated hosts, resulting in the intermittent failures. After restarting the affected services and updating the host list, search began working as expected at about 4:19 p.m. PST.
We continued investigating to be sure we had a full understanding of the cause, and we called the all clear at 6:56 p.m. PST.
8:36 AM PST
We have resolved the issue and restored full search functionality for all affected customers. Thanks again for your patience and understanding.
7:43 PM PST
We have remediated the immediate issue and search should be running smoothly once again. We're still digging into things, however, and we'll post an update as soon as we have more news to share.
If you are still experiencing search issues, please let us know at firstname.lastname@example.org.
5:13 PM PST
We're still investigating the search issues, but we don’t have additional information just yet. We'll be back in 30 minutes with another update. Thank you for bearing with us.
4:32 PM PST
Some customers may be experiencing intermittent search issues in Slack. We apologise for the disruption. We're investigating and working to resolve this as fast as we can.
4:00 PM PST