Service Health
-
-
-
-
Incident affecting Google Compute Engine
Elevated frequency of Host Maintenance events on GCE instances with an attached GPU(s) and SSD(s)
Incident began at 2020-11-10 13:41 and ended at 2020-11-11 11:01 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| | 11 Nov 2020 | 11:01 PST | The issue with Google Compute Engine instances with an attached GPU(s) and SSD(s) is believed to be affecting a very small number of projects and our Engineering Team continues to work on it. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. No further updates will be provided here. We thank you for your patience while we're working on resolving the issue. |
| | 10 Nov 2020 | 16:04 PST | Description: Mitigation work is still underway by our engineering team. Further investigation of current impact and mitigation timeline is ongoing. We will provide more information by Wednesday, 2020-11-11 13:00 US/Pacific. Diagnosis: Affected customers will experience elevated frequency of Host Maintenance events on GCE instances with an attached GPU(s) and SSD(s). Workaround: Temporarily switch to use V100 GPU's which are unaffected by this issue. https://cloud.google.com/compute/docs/gpus |
| | 10 Nov 2020 | 14:33 PST | Description: We are experiencing an issue with Google Compute Engine beginning in 2020-08. A firmware rollout is being created that should address the issue. The rollout is currently expected to complete next week, but mitigation efforts are still ongoing. We will provide more information by Tuesday, 2020-11-10 16:30 US/Pacific. Diagnosis: Affected customers will experience elevated frequency of Host Maintenance events on GCE instances with an attached GPU(s) and SSD(s). Workaround: Temporarily switch to use V100 GPU's which are unaffected by this issue. https://cloud.google.com/compute/docs/gpus |
- All times are US/Pacific