Google Cloud Status Dashboard
This page provides status information on the services that are part of Google Cloud Platform. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit cloud.google.com.
Google BigQuery Incident #20005
Google BigQuery is currently experiencing an elevated rate of job failures.
Incident began at 2020-07-23 16:02 and ended at 2020-07-23 16:08 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| Jul 24, 2020 | 01:02 | The issue with Google BigQuery experiencing an elevated rate of job failures is believed to be affecting a very small number of projects and our Engineering Team is working on it. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. We thank you for your patience while we're working on resolving the issue. |
|
The issue with Google BigQuery experiencing an elevated rate of job failures is believed to be affecting a very small number of projects and our Engineering Team is working on it. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. We thank you for your patience while we're working on resolving the issue. |
|||
| Jul 23, 2020 | 23:28 | Description: Mitigation work is making progress, and we are seeing error rates down from the peak at 1% to under 0.01%. We are mitigating the affected projects in batch and individually. If you already have a support case open with an affected project, we will let you know individually once your project is mitigated. We will provide more information by Friday, 2020-07-24 01:00 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: Mitigation work is making progress, and we are seeing error rates down from the peak at 1% to under 0.01%. We are mitigating the affected projects in batch and individually. If you already have a support case open with an affected project, we will let you know individually once your project is mitigated. We will provide more information by Friday, 2020-07-24 01:00 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 22:20 | Description: As the full mitigation is taking longer than we have expected, we are trying to resolve individual projects manually at the same time. If you already have a support case open with an affected project, we will let you know individually once your project is mitigated. We will provide more information by Thursday, 2020-07-23 23:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: As the full mitigation is taking longer than we have expected, we are trying to resolve individual projects manually at the same time. If you already have a support case open with an affected project, we will let you know individually once your project is mitigated. We will provide more information by Thursday, 2020-07-23 23:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 21:15 | Description: Mitigation work is still underway by our engineering team. We will provide more information by Thursday, 2020-07-23 22:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: Mitigation work is still underway by our engineering team. We will provide more information by Thursday, 2020-07-23 22:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 20:21 | Description: Mitigation work is still underway by our engineering team. We are making progress in the mitigation, but it is taking longer than our initially expected timeline. We will provide more information by Thursday, 2020-07-23 21:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: Mitigation work is still underway by our engineering team. We are making progress in the mitigation, but it is taking longer than our initially expected timeline. We will provide more information by Thursday, 2020-07-23 21:30 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 19:39 | Description: Mitigation work is still underway by our engineering team. The mitigation is expected to complete by Thursday, 2020-07-23 20:30 US/Pacific. We will provide more information by Thursday, 2020-07-23 20:45 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: Mitigation work is still underway by our engineering team. The mitigation is expected to complete by Thursday, 2020-07-23 20:30 US/Pacific. We will provide more information by Thursday, 2020-07-23 20:45 US/Pacific. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 17:52 | Description: Mitigation work is still underway by our engineering team. Affected projects are being mitigated in batches, so retrying the failed job periodically may succeed. We will provide an update by Thursday, 2020-07-23 20:00 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|
Description: Mitigation work is still underway by our engineering team. Affected projects are being mitigated in batches, so retrying the failed job periodically may succeed. We will provide an update by Thursday, 2020-07-23 20:00 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: Retry the failed jobs periodically, as they will start succeeding once your project is mitigated. |
|||
| Jul 23, 2020 | 16:39 | Description: Google BigQuery is currently experiencing an elevated rate of job failures in the US multi-region. Symptoms: Google BigQuery job failures returning an "Internal Error" message. Mitigation is under way and our engineering team is working on it. We will provide an update by Thursday, 2020-07-23 18:05 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|
Description: Google BigQuery is currently experiencing an elevated rate of job failures in the US multi-region. Symptoms: Google BigQuery job failures returning an "Internal Error" message. Mitigation is under way and our engineering team is working on it. We will provide an update by Thursday, 2020-07-23 18:05 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|||
| Jul 23, 2020 | 16:34 | Description: Google BigQuery is currently experiencing an elevated rate of job failures in the US multi-region. Symptoms: Google BigQuery job failures returning an "Internal Error" message. Mitigation is under way and our engineering team is working on it. We will provide an update by Thursday, 2020-07-23 18:00 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|
Description: Google BigQuery is currently experiencing an elevated rate of job failures in the US multi-region. Symptoms: Google BigQuery job failures returning an "Internal Error" message. Mitigation is under way and our engineering team is working on it. We will provide an update by Thursday, 2020-07-23 18:00 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|||
| Jul 23, 2020 | 16:29 | Description: We are experiencing an intermittent issue with some Google BigQuery jobs failing with "Internal Error" messages. Symptoms: Intermittent BigQuery job failure with corresponding "Internal Error" messages. Our engineering team continues to mitigate the issue. We will provide an update by Thursday, 2020-07-23 17:30 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|
Description: We are experiencing an intermittent issue with some Google BigQuery jobs failing with "Internal Error" messages. Symptoms: Intermittent BigQuery job failure with corresponding "Internal Error" messages. Our engineering team continues to mitigate the issue. We will provide an update by Thursday, 2020-07-23 17:30 US/Pacific with current details. Diagnosis: You may be affected if you notice Google BigQuery jobs failing with an "Internal Error" message. Workaround: None at this time. |
|||