Friday September 7, 2018

Notice

Slack’s forwarding email feature is failing for some customers

Technical Summary:
On September 7th at 2:35pm PT, we received reports that emails were failing to deliver to Slack forwarding addresses. We identified that this was the result of an expired certificate used to verify requests sent from our email provider. At 4:55pm PT, we deployed an update that corrected this and fixed the problem. Unfortunately any email sent to a forwarding address during this time is not retrievable and will need to be resent.

6:34 PM PDT - Sep 7th・See in your timezone

The issue with Slack's forwarding email feature has been fixed. Thank you for your patience while we resolved this!

4:55 PM PDT - Sep 7th・See in your timezone

A problem with an upstream provider has caused the feature that allows you to send emails to Slack to fail. We’re working with our partners to resolve this problem.

2:35 PM PDT - Sep 7th・See in your timezone

Services affected

Apps/Integrations/APIs

Status

Resolved

Incident

Connectivity issues affecting some workspaces in South America

Technical Summary:
On September 7th between 7:43am PT and 8:05am PT, Slack could not connect to our servers in the Sao Paulo region due to network issues. As a result, some people could not access their workspaces.

At 8:05am PT we deployed a fix that would route users out of the troubled region and to a healthier server, which allowed people to successfully connect to Slack again.

We are currently hard at work on scoping preventive measures to ensure this doesn’t happen in the future.

6:23 PM PDT - Sep 7th・See in your timezone

We've patched up the connectivity issues for the South America folks, and things should be working better now with a refresh (CMD/CTRL + R). If this isn't the case, don't hesitate to drop us a line and let us know.

8:23 AM PDT - Sep 7th・See in your timezone

We've confirmed that folks are running into issues when trying to connect from South America specifically. We're looking into this further, and will keep updating with new information.

7:58 AM PDT - Sep 7th・See in your timezone

Services affected

Connections

Status

Resolved

Outage

Some people are having trouble connecting to Slack, we're working on it.

Technical Summary:
From 11:15am to 11:28am PT, 100% of customers were not able to use Slack.

A deploy to our webapp to enable a feature for how we handle incoming connection requests referenced a function which wasn’t present in a specific file. This missing function caused all incoming requests to fail, returning connection errors to our users.

We rolled back that change for all users at 11:26am. Users were able to receive responses from our app again by 11:28am.

6:20 PM PDT - Sep 7th・See in your timezone

Everyone should be able to connect to their workspace again now. We're very sorry for the disruption to your day, and appreciate your patience while we got things fixed up on our end.

11:28 AM PDT - Sep 7th・See in your timezone

We're investigating problems with connectivity at this time. We're sorry for the interruption and will keep you posted as soon as we have an update.

11:15 AM PDT - Sep 7th・See in your timezone

Status

Resolved

Incident

Issues with opening new direct messages for Guests on Enterprise Grid

Technical Summary:
On September 4th at 10:39am PT, we deployed a code change intended to fix an existing bug with guests on Enterprise Grid. On September 7th at 4:49am PT, we first received reports that Enterprise Grid users were unable to send direct messages with users on other workspaces whom they had not messaged before. At 6:38am PT, we reverted that code change and this fixed the problem.

5:59 PM PDT - Sep 7th・See in your timezone

We confirmed the fix. Direct messages are working again for Guests on Enterprise Grid. Apologies for the trouble and thanks for your patience.

7:38 AM PDT - Sep 7th・See in your timezone

We're almost out of the woods. Guests on Enterprise Grid should be able to start new direct messages again. Please reload (Ctrl + R or Cmd + R) to make sure the fix is applied.

7:06 AM PDT - Sep 7th・See in your timezone

We found the cause of the problem with new direct messages for Guests on Enterprise Grid. The team is patching it up and the functionality should be back soon.

6:34 AM PDT - Sep 7th・See in your timezone

We've received reports that some Guests on Enterprise Grid workspaces cannot start new direct messages. Apologies for the trouble - our team is on the case.

4:49 AM PDT - Sep 7th・See in your timezone

Services affected

Messaging

Status

Resolved