Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Dataflow

We are experiencing an issue with increased system lag in some Google Cloud Dataflow jobs.

Incident began at 2019-03-11 10:33 and ended at 2019-03-12 06:13 (all times are US/Pacific).

Date Time Description
12 Mar 2019 06:13 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag has been resolved for all affected projects as of Tuesday, 2019-03-12 04:49 US/Pacific. We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence.
12 Mar 2019 05:00 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects. We are still monitoring the system to confirm that the issue has been completely mitigated. We will provide another status update by Tuesday, 2019-03-12 06:30 US/Pacific with current details.
12 Mar 2019 02:57 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the vast majority of projects and we expect it to be resolved in near future. We will provide another status update by Tuesday, 2019-03-12 05:00 US/Pacific with current details.
11 Mar 2019 23:29 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects. Our mitigation efforts are effective and we expect a full resolution in the near future. We will provide another status update by Tuesday, 2019-03-12 03:00 US/Pacific with current details.
11 Mar 2019 21:56 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects. Our mitigation efforts are effective and we expect a full resolution in the near future. We will provide another status update by Monday, 2019-03-11 23:00 US/Pacific with current details.
11 Mar 2019 19:59 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects and we expect a full resolution in the near future. We will provide another status update by Monday, 2019-03-11 22:00 US/Pacific with current details.
11 Mar 2019 18:00 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects and we expect a full resolution in the near future. We will provide another status update by Monday, 2019-03-11 20:00 US/Pacific with current details.
11 Mar 2019 16:44 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects and we expect a full resolution in the near future. We will provide another status update by Monday, 2019-03-11 18:00 US/Pacific with current details.
11 Mar 2019 15:59 PDT

The issue with Google Cloud Dataflow jobs experiencing system lag should be resolved for the majority of projects and we expect a full resolution in the near future. A very small number of jobs are still impacted and Google Engineers are very close to a full resolution. We will provide another status update by Monday, 2019-03-11 16:45 US/Pacific with current details.
11 Mar 2019 14:02 PDT

The issue with stuck Dataflow jobs has been resolved for some users and we expect a full resolution in the near future. We will provide another status update by Monday, 2019-03-11 16:00 US/Pacific with current details.
11 Mar 2019 12:45 PDT

Our Engineering Team continues to mitigate the issue. We will provide another status update by Monday, 2019-03-11 14:00 US/Pacific with current details.
11 Mar 2019 11:43 PDT

Our Engineering Team believes they have identified the root cause of the stuck Dataflow jobs and is working to mitigate. We will provide another status update by Monday, 2019-03-11 12:45 US/Pacific with current details.

As a reminder, Dataflow jobs using newly-created Google Cloud Pub-Sub topics are unaffected.
11 Mar 2019 11:40 PDT

Our Engineering Team believes they have identified the root cause of the stuck Dataflow jobs and is working to mitigate. We will provide another status update by Monday, 2019-03-11 12:45 US/Pacific with current details.
11 Mar 2019 10:33 PDT

Mitigation work is currently underway by our Engineering Team. We will provide another status update by Monday, 2019-03-11 11:45 US/Pacific with current details.
11 Mar 2019 10:33 PDT

We've received a report of an issue with increased system lag in some Google Cloud Dataflow jobs.