Tuesday May 23, 2023

Issue summary:

From May 19, 2023 at 12:00 AM PDT until May 25, 2023 at 8:00 AM PDT, some users may have experienced issues triggering shortcut workflows.

We carried out our investigation and determined that cached data on the back end temporarily broke legacy workflows. In the process of determining our path to a resolution the issue resolved itself. We are taking the necessary steps to prevent similar issues in the future.

11:00 AM PST

We've resolved the issue with triggering shortcut workflows. Thank you very much for your patience while we worked on getting this fix completed. We're sorry for the disruption caused.

8:36 AM PST

We're continuing to work to fully resolve the issue triggering shortcut Workflows and we will provide an update as soon we have more information to share. Sorry for the trouble here and thank you for your patience.

1:42 AM PST

We’re still looking into the cause of the executing workflows issue, and we’ll provide new information as soon as it’s available. We apologize for the continued trouble.

5:23 AM PST

Some users are reporting issues triggering shortcut Workflows at the moment. The team are currently working hard to fix this. We're very sorry for any inconvenience caused in the meantime.

3:55 AM PST

On May 23, 2023 between 8:38 AM PDT and 9:44 AM PDT some users were unable to send and edit messages, load threads, join channels and create group DMs.

Our investigation uncovered that a recent code change we made had caused an overwhelming impact on our databases. We rolled this change back and temporarily redirected user traffic as our databases recovered. Once we rolled back the change and our databases were in a healthier state, the issue became resolved for impacted users.

We will continue to implement new ways to detect issues like this early-on.

Thank you for your patience while we investigated and resolved the issue.

3:29 PM PST

We’ve released a fix and all impacted features are up and running once again. We
appreciate your patience while we sorted this out.

9:55 AM PST

Some users may still be experiencing trouble in Slack, specifically with sending and editing messages, loading threads, and joining channels. These features are working slowly for some users, and not working at all for others. We’ll continue to provide updates as we know more.

9:32 AM PST

We're receiving reports that a small percentage of users are experiencing issues with loading and sending messages. We're actively looking into this, and we'll report back as more information becomes available.

9:02 AM PST

Issue summary:
On May 23, 2023 from 5:30 PM PDT to around 8:15 PM PDT, some customers may have experienced blank profile pictures, display names, channel names, and emoji in Slack.

A code change inadvertently altered the firewall rules for some of our regional servers, preventing the Slack clients connected to these regions from loading profile and channel elements as expected.

As an immediate mitigation step, we stopped the code change from rolling out any further, and pushed all traffic from the affected servers to other regions. This resolved the issue for impacted customers.

We then reverted the code change and manually restored the correct firewall rules on the affected servers. Once the correct rules were back in place, we reintroduced traffic to these regions, fully resolving the issue.

11:34 PM PST

We’ve released a fix and all impacted features are up and running once again. If you’re still encountering any trouble, please reload Slack using Command + Shift + R (Mac) or Ctrl + Shift + R (Windows/Linux). Thank you for your patience while we sorted out this issue.

8:31 PM PST

We are currently investigating an issue where profile information is failing to load for some users, resulting in blank profile pictures, display names and emojis.

We apologise for the disruption and please stay tuned for updates as we work to resolve this.

7:57 PM PST