Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Composer

We are mitigating an issue with creating and deleting Cloud Composer environments.

Incident began at 2019-07-25 05:13 and ended at 2019-07-25 19:59 (all times are US/Pacific).

Date Time Description
1 Aug 2019 11:44 PDT

After further investigation, we have determined that the impact was minimal and affected a subset of users. We have conducted an internal investigation of this issue and made appropriate improvements to our systems to help prevent or minimize a future recurrence.

25 Jul 2019 18:31 PDT

The issue with Cloud Composer environment creation/deletion errors has been resolved for all affected projects as of Thursday, 2019-07-25 18:31 US/Pacific. We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a more detailed analysis of this incident once we have completed our internal investigation.

25 Jul 2019 16:51 PDT

The issue with Cloud Composer environment creation/deletion errors should be resolved for the majority of projects. Currently, approximately 5% of the requests are still failing. Our engineers are still working to mitigate the issue and we expect a full resolution in the near future. We will provide another status update by Thursday, 2019-07-25 18:00 US/Pacific with current details.

25 Jul 2019 16:42 PDT

The issue with Cloud Composer environment creation/deletion errors should be resolved for the majority of projects. Currently, approximately 5% of the requests are still failing. Our engineers are still working to mitigate the issue and we expect a full resolution in the near future. We will provide another status update by Thursday, 2019-07-25 18:00 US/Pacific with current details.

25 Jul 2019 15:39 PDT

The rate of errors is decreasing. We will provide another status update by Thursday, 2019-07-25 16:45 US/Pacific with current details.

25 Jul 2019 14:42 PDT

We are still seeing errors. Our Engineers have implemented a mitigation and are now monitoring the situation before gradually rolling it out to a wider audience. We will provide another status update by Thursday, 2019-07-25 15:45 US/Pacific with current details.

25 Jul 2019 13:43 PDT

We are still seeing errors. Our Engineers are attempting several mitigation strategies in parallel and are beginning to see positive results. We will provide another status update by Thursday, 2019-07-25 14:45 US/Pacific with current details.

25 Jul 2019 12:29 PDT

Mitigation work is currently still underway by our Engineering Team. We will provide another status update by Thursday, 2019-07-25 13:45 US/Pacific with current details.

25 Jul 2019 10:34 PDT

We are investigating errors with creating and deleting Cloud Composer environments. Our engineering team has identified a root cause, and mitigation work is currently underway. Efforts to accelerate recovery using the previous mitigation were not effective and the issue has been escalated internally, we are continuing to work on a new mitigation. We will provide another status update by Thursday, 2019-07-25 12:30 US/Pacific with current details.

25 Jul 2019 09:14 PDT

We are investigating errors with creating and deleting Cloud Composer environments. Our engineering team has identified a root cause, and mitigation work is currently underway. Mitigation appears effective, and we are working to accelerate recovery. We will provide another status update by Thursday, 2019-07-25 10:30 US/Pacific with current details.

25 Jul 2019 08:36 PDT

We are investigating errors with creating and deleting Cloud Composer environments. Our Engineering Team believes they have identified the root cause and mitigation work is underway. We will provide another status update by Thursday, 2019-07-25 09:30 US/Pacific with current details.