Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Console

Users unable to create new projects

Incident began at 2015-05-08 04:53 and ended at 2015-05-08 12:31 (all times are US/Pacific).

Date Time Description
11 May 2015 18:54 PDT

SUMMARY:

On Friday 8 May 2015, users were unable to reliably create projects in the Google Cloud Developers Console for 6 hours 58 minutes in aggregate. We treat the reliability and availability of the Developers Console with the highest priority and we apologize to any users who were unable to create new projects during that time.

DETAILED DESCRIPTION OF IMPACT:

On Friday 8 May 2015, from 04:53 to 07:25 and from 09:35 to 12:31 PDT, users were unable to reliably create new projects using the Developers Console. Specifically, up to 90% of project creation requests failed and returned a "Backend Error" message to the user. Existing projects were not affected by this incident.

ROOT CAUSE AND REMEDIATION:

A configuration change in the access management system of Developers Console triggered an existing software bug in a previously unused code path. As the change was rolled out, the access management system could no longer correctly verify user’s permissions, resulting in an error response.

Google engineers were automatically alerted to the issue at 05:04. At 05:22 a preliminary source of the issue was identified. At 09:32 the root cause was confirmed and a solution was developed. An expedited roll-out of the fix commenced at 11:41 and completed at 12:31.
8 May 2015 13:28 PDT

The Developer Console issue that prevented users from creating projects was resolved as of 2015-05-08 13:15 US/Pacific. We will conduct an internal investigation of this issue and make appropriate improvements to our systems to prevent or minimize future recurrence. We will provide a more detailed analysis of this incident once we have completed our internal investigation.
8 May 2015 12:26 PDT

We are continuing to work toward a resolution of the Developers Console issue that is preventing users from creating projects. We will provide another status update no later than 13:25 pacific time.
8 May 2015 11:24 PDT

We are currently experiencing an issue with the Developers Console where users are unable to create new projects. We are currently working toward a resolution. For everyone who is affected, we apologize for any inconvenience you may be experiencing.