Thursday September 30, 2021

Incident

Less than 1% of users may be experiencing trouble connecting to Slack

Issue summary:
From September 30, 2021 at 8:36 a.m. PDT until October 1, 2021 around 8:49 a.m. PDT, some users were unable to load Slack. This may have prevented access to the Slack desktop app, mobile app, or any websites under the slack.com domain.

At 8:21 a.m. PDT on September 30, we introduced a DNS (Domain Name System) configuration change to our slack.com domain, designed to improve the security of domain name lookups. A small number of third-party DNS resolvers did not properly interpret the change, causing some users to be unable to connect to Slack.

Out of caution, we rolled back this configuration change at 10:02 a.m. on September 30, but some third-party DNS resolvers still expected the configuration to exist in our slack.com domain. As a result, some third-party DNS resolvers failed to return records for the slack.com domain. Because some internet service providers and public resolvers persist records for up to 24 hours as a standard practice, the issue continued for a percentage of users.

In the meantime, we reached out to third-party DNS resolvers to request they refresh the record for the slack.com domain, which would resolve the issue for Slack users on that DNS resolver. We’re grateful to the internet service providers and DNS community members who worked quickly to refresh records around the globe, improving conditions for many impacted users.

By around 8:49 a.m. PDT on October 1, the majority of DNS records for the slack.com domain had refreshed, and connectivity to Slack returned to baseline levels.

We know our customers rely on Slack. If you’d like to receive a full Root Cause Analysis (RCA) report, please reach out to feedback@slack.com to request one.

4:53 PM PST

We've confirmed this issue is now resolved. If you're still encountering any issues connecting, please reload Slack using Ctrl + Shift + R (Windows/Linux) / Command + Shift + R (Mac) on desktop, or fully quit and reopen Slack on mobile. If that doesn't help, let us know at feedback@slack.com.

If you tried switching to Google DNS during the course of this issue, you can choose to keep this enabled, or switch back using these steps: https://developers.google.com/speed/public-dns/docs/using#switch_back_to_your_old_dns_settings

Our sincere apologies for the interruption to your work, and we appreciate you bearing with us in the meantime. We'll be back with a full summary of the issue.

Note: We've adjusted the timestamp of this update to accurately reflect the issue's time of resolution.

8:49 AM PST

Apologies for the continued trouble. It will take some time for the DNS changes to resolve this issue for everyone. If organization policy permits, and if you are comfortable making this change, switching to Google DNS may improve things in the meantime. Please find guides below:

Windows: https://developers.google.com/speed/public-dns/docs/using#windows
Mac: https://developers.google.com/speed/public-dns/docs/using#macos
iOS: https://developers.google.com/speed/public-dns/docs/using#ios
Android: https://developers.google.com/speed/public-dns/docs/using#android

If this does not improve things for you, please switch back to your original DNS settings by following these steps: https://developers.google.com/speed/public-dns/docs/using#switch_back_to_your_old_dns_settings


We'll be back with an update as soon as we have more information to share.

11:57 PM PST

We're seeing some signs of improvement, and reloading Slack (Ctrl/Command + R) may allow Slack to load again. The issue is not resolved for everyone, however, and we'll be back again once we have another update to share.

Note: We've edited the title of this incident to accurately reflect the scope of this issue.

2:30 PM PST

Thank you all for your patience today as we work on resolving this connection issue. We will be back with a final update once the issues have been resolved.

1:34 PM PST

We are aware of connectivity issues related to DNS that are impacting a small subset of users. This issue was caused by our own change and not related to any third-party DNS software and services. In order to resolve this faster, your ISP (Internet Service Provider) will need to flush their DNS record for slack.com. Please reach out to your networking team to provide them with this information.
We expect all customers’ connectivity issues to be resolved within the next 24 hours. We know how important it is for people to stay connected and we apologize for this disruption.

12:50 PM PST

Our investigation of this connection issue is still ongoing at this time. We’ll be back with another update and apologize for this disruption. Thank you for your continued patience here!

12:19 PM PST

We are continuing to investigate the connection issue and will keep you posted as soon as we have more information available. We apologize for any disruption this has caused and thank you for your continued patience.

11:17 AM PST

We are still digging in to this connection issue and we will keep you up to date as more information becomes available. Thank you for your patience today.

10:46 AM PST

We continue to investigate this connection issue and we’ll continue to keep you posted as soon as we have more information. Thanks for your patience and we apologize for any disruption this may have caused.

10:05 AM PST

We’re currently investigating this connection issue and we’ll keep you posted as soon as we have more information. Thanks for your patience and we apologize for the disruption.

9:30 AM PST

Services affected

Connectivity

Status

Incident