Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Support

Issue with Google Cloud Support

Incident began at 2019-05-17 10:08 and ended at 2019-05-17 14:07 (all times are US/Pacific).

Date Time Description
31 May 2019 14:55 PDT

ISSUE SUMMARY

On Friday 17 May, 2019, Google Cloud Support experienced case creation and read issues for a duration of 4 hours. We sincerely apologize to our customers for the difficulties and delays in communicating with Google Cloud Support during this time.

DETAILED DESCRIPTION OF IMPACT

On Friday 17 May, 2019 from 10:08 to 14:07 US/Pacific, both Google Cloud Support and our customers were unable to view, create, and modify Support cases, utilize Chat Support, or view known issues via the Google Cloud Support Center (GCSC) and the Cloud Console. Phone support was also unavailable from 10:08 to 14:26.

ROOT CAUSE

A critical external dependency for Google Cloud Support encountered an issue resulting in service unavailability, which impacted both Google’s and Google Cloud Platform customers’ ability to use the service. This impacted case creation for chat, phone, and email cases, which all rely on the external dependency to reference customer data and route to the correct channel. Unfortunately, our tooling also did not allow for an automated mitigation.

REMEDIATION AND PREVENTION

Google’s engineers were alerted to the problem at 10:26 and escalated immediately. Once the nature and scope of the situation became clear, Cloud Support identified customer communication options through secondary channels. Customers were able to file cases through the backup “Contact Support” mechanism that was automatically available in the Google Cloud Support Center (GCSC). All cases reported through the backup mechanism were routed internally to the appropriate Support Engineers.

We will refactor Cloud Support’s Business Continuity Plan to account for issues of this nature, to ensure that:

  • Existing cases can continue to be viewed and updated
  • Chat Support will be available
  • Phone support will be available
17 May 2019 14:16 PDT

The issue with users being unable to create new cases has been resolved for all users. We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence
17 May 2019 10:37 PDT

We are experiencing an issue with Google Cloud Support in which users are unable to create new support cases.