Home

Salesforce engineers roll back change after breaking its own cloud for hours today

If you noticed something funky going on with Salesforce and its software-as-a-service empire today, it's not you: it's recovering from a mild outage.

As of writing, Salesforce said at 1829 UTC (1130 PT) in a status update that all of its clouds are recovering after suffering about a four-hour downtime.

Well, all except Tableau and Mulesoft, which remain down to some extent.

The IT breakdown started at 1448 UTC, and we're told by the enterprise software giant that it hit "customers across multiple clouds including, Commerce Cloud, Mulesoft, Tableau, Core, Marketing Cloud Account Engagement, Marketing Cloud Intelligence, and Omni Channel."

Two hours later, the team said its ClickSoftware, Trailblazer, and Data Cloud products were also affected. At first a third-party cloud provider was blamed for intermittent networking issues causing Salesforce services to fail.

Then it said, actually, that cloud provider wasn't at fault, and Salesforce engineers "successfully executed a rollback to mitigate the issue." By 1752 UTC, the biz said its cloud systems were getting back on track.

Us vultures, as Slack users, noticed a blip in connectivity with the Salesforce-owned chat app, for what it's worth, though Slack is not reporting any substantial outage.

If you're relying on Tableau, though Tableau Public has been apparently fully restored, Tableau Cloud is still screwed. A status update two hours ago stated:

Tableau Engineers have identified the issue causing the Tableau Cloud outage for all pods. Engineers continue to work on the solution to mitigate user impact. We thank you for your patience; our next update will be when new information becomes available.

Tableau Engineers have identified the issue causing the Tableau Cloud outage for all pods. A solution to mitigate user impact is currently being worked on.

Meanwhile, 20 minutes ago, Mulesoft said it is services are recovering after a fix was applied.

We think it's safe to say Salesforce broke part of its back-end systems with an update, causing services and products relying on that infrastructure to fail, and customers unable to use the stuff as expected. We'll keep you updated. ®

Source: The register

Previous

Next