Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google BigQuery

Multi-region/US: BigQuery queries may show latencies during execution.

Incident began at 2021-10-27 02:00 and ended at 2021-10-30 12:00 (all times are US/Pacific).

Date Time Description
1 Nov 2021 12:27 PDT

We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: 27 Oct 2021 02:00

Incident End: 30 Oct 2021 12:00

Duration: 82 hours

Affected Services and Features:

Google BigQuery

Regions/Zones: US Multi-region

Description:

Google BigQuery service performance experienced an increase in latency for queries in multiple regions in the US for 82 hours. From preliminary analysis, the root cause of the issue was capacity increase that caused rebalancing of workload and increased the scheduler processing time. This caused delays in query execution.

Customer Impact:

  • Queries exhibited higher than normal latency.

  • Customers were unable to fully utilize their reserved slots.

28 Oct 2021 17:58 PDT

The issue with BigQuery queries showing latencies, has been resolved for all affected projects as of Thursday, 2021-10-28 17:45 US/Pacific.

If you have any further questions or are still impacted, please open a case with the Support Team and we will work with you until the issue is resolved.

We thank you for your patience while we worked towards resolving the issue.

28 Oct 2021 16:01 PDT

Summary: Multi-region/US: BigQuery queries may show latencies during execution.

Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance.

We do not have an ETA for mitigation completion at this moment.

We will provide more information by Thursday, 2021-10-28 18:00 US/Pacific.

Diagnosis: Some queries may may exhibit higher than normal latencies during execution.

Workaround: None at this time.

28 Oct 2021 14:49 PDT

Summary: Multi-region/US: BigQuery queries may show latencies during execution.

Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance.

We do not have an ETA for mitigation completion at this moment.

We will provide more information by Thursday, 2021-10-28 16:00 US/Pacific.

Diagnosis: Some queries may may exhibit higher than normal latencies during execution.

Workaround: None at this time.

28 Oct 2021 13:18 PDT

Summary: Multi-region/US: BigQuery queries may show latencies during execution.

Description: Engineering has confirmed that the mitigation work is progressing and we are continuing to monitor the system performance.

We do not have an ETA for the mitigation of the impact.

We will provide more information by Thursday, 2021-10-28 15:00 US/Pacific.

Diagnosis: Some queries may may exhibit higher than normal latencies during execution.

Workaround: None at this time.

28 Oct 2021 11:33 PDT

Summary: Multi-region/US: BigQuery queries may show latencies during execution.

Description: Mitigation work is still underway by our engineering team. The issue is affecting the US region and is currently under investigation.

We do not have an ETA for the mitigation of the impact.

We will provide more information by Thursday, 2021-10-28 13:30 US/Pacific.

Diagnosis: Some queries may may exhibit higher than normal latencies during execution.

Workaround: None at this time.