Service Health
-
-
-
-
Incident affecting Google BigQuery
Multi-region/US: BigQuery queries may show latencies during execution.
Incident began at 2021-10-27 02:00 and ended at 2021-10-30 12:00 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| | 1 Nov 2021 | 12:27 PDT | We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support. (All Times US/Pacific) Incident Start: 27 Oct 2021 02:00 Incident End: 30 Oct 2021 12:00 Duration: 82 hours Affected Services and Features: Google BigQuery Regions/Zones: US Multi-region Description: Google BigQuery service performance experienced an increase in latency for queries in multiple regions in the US for 82 hours. From preliminary analysis, the root cause of the issue was capacity increase that caused rebalancing of workload and increased the scheduler processing time. This caused delays in query execution. Customer Impact:
|
| | 28 Oct 2021 | 17:58 PDT | The issue with BigQuery queries showing latencies, has been resolved for all affected projects as of Thursday, 2021-10-28 17:45 US/Pacific. If you have any further questions or are still impacted, please open a case with the Support Team and we will work with you until the issue is resolved. We thank you for your patience while we worked towards resolving the issue. |
| | 28 Oct 2021 | 16:01 PDT | Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance. We do not have an ETA for mitigation completion at this moment. We will provide more information by Thursday, 2021-10-28 18:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time. |
| | 28 Oct 2021 | 14:49 PDT | Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is progressing and our Engineering teams continue to monitor the system performance. We do not have an ETA for mitigation completion at this moment. We will provide more information by Thursday, 2021-10-28 16:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time. |
| | 28 Oct 2021 | 13:18 PDT | Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Engineering has confirmed that the mitigation work is progressing and we are continuing to monitor the system performance. We do not have an ETA for the mitigation of the impact. We will provide more information by Thursday, 2021-10-28 15:00 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time. |
| | 28 Oct 2021 | 11:33 PDT | Summary: Multi-region/US: BigQuery queries may show latencies during execution. Description: Mitigation work is still underway by our engineering team. The issue is affecting the US region and is currently under investigation. We do not have an ETA for the mitigation of the impact. We will provide more information by Thursday, 2021-10-28 13:30 US/Pacific. Diagnosis: Some queries may may exhibit higher than normal latencies during execution. Workaround: None at this time. |
- All times are US/Pacific