Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Console

We are investigating an issue with Developers Console

Incident began at 2015-02-19 13:45 and ended at 2015-02-21 22:33 (all times are US/Pacific).

Date Time Description
9 Jun 2016 21:26 PDT

We are investigating an issue with Developers Console. We will provide more information by 21:40 US/Pacific.
21 Feb 2015 22:33 PST

SUMMARY:

On Thursday 19 February 2015, the Google Developers Console (https://console.developers.google.com) served a high rate of errors like “Failed to load” on many pages for a duration of 3 hours and 3 minutes. This affected users in all regions to varying degrees. If your service or application was affected, we apologize. We have taken and are taking immediate steps to improve our preparedness to mitigate and resolve similar issues in much shorter time in the future.

DETAILED DESCRIPTION OF IMPACT:

On Thursday 19 February 2015 from 12:42 to 15:45 PST the vast majority of attempts to create new projects and interact with existing projects via the Google Developers Console failed.

ROOT CAUSE:

The Google Developers Console relies upon a component of Google Cloud Platform's billing system to provide project details. This component was under-provisioned for certain observed traffic patterns and suffered from contention under load, resulting in timeouts when serving user requests.

REMEDIATION AND PREVENTION:

Google engineers were automatically alerted to the issue at 12:51, and immediately began investigating. To mitigate the issue, at 13:33 Google engineers directed traffic away from the datacenter that was most heavily affected. Following further investigation, Google engineers directed traffic away from a second affected datacenter at 15:41.

In order to prevent this issue from recurring, Google engineers have made immediate increases to the available capacity in the billing system. Additionally, Google engineers are implementing changes to reduce the Developers Console’s dependence on the affected systems.
19 Feb 2015 16:32 PST

The problem with the Developers Console should be resolved as of 16:09 PST. We apologize for any issues this may have caused to you or your users and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are constantly working to improve the reliability of our systems.
19 Feb 2015 15:56 PST

We are still investigating the issue with the Developers Console. Some users are seeing errors when browsing the Developers Console. Impacted users may use gcloud command line tools (https://cloud.google.com/sdk) for most Cloud Platform interactions. We will provide another status update by 17:00 PST.
19 Feb 2015 14:29 PST

We are still investigating the issue with the Developers Console. Some users are seeing errors when browsing the Developers Console. Impacted users may use gcloud command line tools (https://cloud.google.com/sdk) for most Cloud Platform interactions. We will provide another status update by 15:00 PST.
19 Feb 2015 14:01 PST

We are still investigating the issue with the Developers Console. Some users are seeing errors when browsing the Developers Console. We will provide another status update by 14:30 PST.
19 Feb 2015 13:46 PST

We are currently experiencing an issue with the Developers Console and some users are seeing errors when accessing pages on the site. For everyone who is affected, we apologize for any inconvenience you may be experiencing.