Google Cloud Status Dashboard
This page provides status information on the services that are part of Google Cloud Platform. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit cloud.google.com.
Google Cloud Infrastructure Components Incident #19002
High latency and elevated error rate
Incident began at 2019-12-09 00:26 and ended at 2019-12-09 03:44 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| Dec 09, 2019 | 03:44 | The issue with Google Cloud infrastructure components is believed to be affecting a very small number of projects and our Engineering Team is working on it. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. No further updates will be provided here. |
|
The issue with Google Cloud infrastructure components is believed to be affecting a very small number of projects and our Engineering Team is working on it. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. No further updates will be provided here. |
|||
| Dec 09, 2019 | 03:08 | Description: We have restored the affected capacity. We are still seeing a backlog in Cloud Logging that we expect to clear shortly. Full resolution is expected to complete by Monday, 2019-12-09 03:40 US/Pacific. We will provide an update by Monday, 2019-12-09 03:40 US/Pacific with current details. Workaround: None at this time. |
|
Description: We have restored the affected capacity. We are still seeing a backlog in Cloud Logging that we expect to clear shortly. Full resolution is expected to complete by Monday, 2019-12-09 03:40 US/Pacific. We will provide an update by Monday, 2019-12-09 03:40 US/Pacific with current details. Workaround: None at this time. |
|||
| Dec 09, 2019 | 02:50 | Description: We believe the issue with Google Cloud infrastructure components is partially resolved. Full resolution is expected to complete by Monday, 2019-12-09 03:40 US/Pacific. We will provide an update by Monday, 2019-12-09 03:45 US/Pacific with current details. Workaround: None at this time. |
|
Description: We believe the issue with Google Cloud infrastructure components is partially resolved. Full resolution is expected to complete by Monday, 2019-12-09 03:40 US/Pacific. We will provide an update by Monday, 2019-12-09 03:45 US/Pacific with current details. Workaround: None at this time. |
|||
| Dec 09, 2019 | 02:15 | Description: We believe the issue with Google Cloud infrastructure components is partially resolved as we brought additional capacity online. We do not have an ETA for full resolution at this point. We were seeing impact in Asia on GCS, Stackdriver Logging, Stackdriver Monitoring, Cloud Spanner, Cloud Function, Container Registry, Firebase Cloud Messaging, Firebase AB Testing, Firebase Remote Config, Firebase ML Kit, BigQuery and App Engine. We will provide an update by Monday, 2019-12-09 03:00 US/Pacific with current details. Workaround: None at this time. |
|
Description: We believe the issue with Google Cloud infrastructure components is partially resolved as we brought additional capacity online. We do not have an ETA for full resolution at this point. We were seeing impact in Asia on GCS, Stackdriver Logging, Stackdriver Monitoring, Cloud Spanner, Cloud Function, Container Registry, Firebase Cloud Messaging, Firebase AB Testing, Firebase Remote Config, Firebase ML Kit, BigQuery and App Engine. We will provide an update by Monday, 2019-12-09 03:00 US/Pacific with current details. Workaround: None at this time. |
|||
| Dec 09, 2019 | 01:49 | Description: Mitigation work is currently underway by our engineering team. This issue is due to multiple fiber cuts in Asia and our engineering team is working on restoring capacity. We do not see any indications of this networking issue affecting VM to VM communication. We will provide more information by Monday, 2019-12-09 02:45 US/Pacific. Workaround: None at this time. |
|
Description: Mitigation work is currently underway by our engineering team. This issue is due to multiple fiber cuts in Asia and our engineering team is working on restoring capacity. We do not see any indications of this networking issue affecting VM to VM communication. We will provide more information by Monday, 2019-12-09 02:45 US/Pacific. Workaround: None at this time. |
|||
| Dec 09, 2019 | 01:30 | Description: Mitigation work is currently underway by our engineering team. We will provide more information by Monday, 2019-12-09 02:30 US/Pacific. Workaround: None at this time. |
|
Description: Mitigation work is currently underway by our engineering team. We will provide more information by Monday, 2019-12-09 02:30 US/Pacific. Workaround: None at this time. |
|||
| Dec 09, 2019 | 00:33 | Description: We are experiencing an issue with Google Cloud infrastructure components beginning at Sunday, 2019-12-08 20:59 US/Pacific. Symptoms: High latency and failures on queries and deployments. Our engineering team continues to investigate the issue. We will provide an update by Monday, 2019-12-09 01:30 US/Pacific with current details. Workaround: None at this time. |
|
Description: We are experiencing an issue with Google Cloud infrastructure components beginning at Sunday, 2019-12-08 20:59 US/Pacific. Symptoms: High latency and failures on queries and deployments. Our engineering team continues to investigate the issue. We will provide an update by Monday, 2019-12-09 01:30 US/Pacific with current details. Workaround: None at this time. |
|||