Tuesday July 26, 2022

Incident

Issues with some third-party apps

Issue summary:
On July 26, 2022 between 5:45 AM PDT and 5:23 PM PDT some users may have experienced connectivity issues with third-party Slack apps using Socket Mode.

We found that this was caused by an internal configuration failing to exclude stale hosts when routing incoming requests.

Clearing out the stale hosts resolved the timeout issues, restoring normal app behaviour for affected users.

Note: We've edited this summary to better reflect our understanding of the issue.

7:15 PM PST

We’ve rolled out a fix to resolve the issue, and impacted customers should now be able to connect to their third party apps. Thanks for bearing with us, and we apologize for the disruption.

6:10 PM PST

Some third-party apps may be experiencing connection issues. We're investigating and will let you know as soon as we have further information to share. We appreciate your patience in the meantime.

5:22 PM PST

Services affected

Apps/Integrations/APIs

Status

Incident

Incident

Messages, threads, file uploads/downloads and API calls failures.

Issue Summary:

On July 26, 2022 between 05:45 AM PDT and 07:45 AM PDT, International Data Residency customers were experiencing trouble sending messages and loading channels, while other customers also experienced issues with Slack Connect, Slack's API and uploading/downloading files and images.
We found this issue was caused by some backend system nodes failing. Provisioning new nodes for these systems restored functionality.
We're continuing to investigate the cause of the node failures and possible mitigations. We appreciate your patience while this work continues.

10:02 AM PST

Customers should no longer be experiencing issues with sending messages, loading threads and uploading or downloading files. Thank you for your patience while we worked to resolve this!

8:03 AM PST

We are continuing to work to resolve the current issues surrounding sending messages, loading threads and accessing file uploads. Customers may still be experiencing issues. We'll continue to post updates as we have them. Thank you for your continued patience.

7:46 AM PST

Some customers may continue to experience trouble sending messages, loading threads and uploading or downloading files. We'll continue to provide further updates as soon as we know more. We thank you for your patience in the meantime.

7:19 AM PST

Customers may still be experiencing trouble in Slack, specifically with sending messages and loading threads. These features may be working slowly for some users, and not working at all for others. We’ll continue to provide updates as we know more.

6:45 AM PST

We are receiving multiple reports of threads not loading and some users are unable to send messages. We’re investigating and will let you know as soon as we have further information to share. We appreciate your patience in the meantime.

6:12 AM PST

Services affected

Messaging

Posts/Files

Apps/Integrations/APIs

Status

Incident