Service Health
Incident affecting Google Compute Engine
VMs using Local SSD are experiencing intermittent terminations.
Incident began at 2022-10-26 22:32 and ended at 2022-11-02 11:47 (all times are US/Pacific).
Previously affected location(s)
Taiwan (asia-east1)Hong Kong (asia-east2)Tokyo (asia-northeast1)Osaka (asia-northeast2)Seoul (asia-northeast3)Mumbai (asia-south1)Delhi (asia-south2)Singapore (asia-southeast1)Jakarta (asia-southeast2)Sydney (australia-southeast1)Melbourne (australia-southeast2)Warsaw (europe-central2)Finland (europe-north1)Madrid (europe-southwest1)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Netherlands (europe-west4)Zurich (europe-west6)Milan (europe-west8)Paris (europe-west9)Montréal (northamerica-northeast1)Toronto (northamerica-northeast2)São Paulo (southamerica-east1)Santiago (southamerica-west1)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)Columbus (us-east5)Dallas (us-south1)Oregon (us-west1)Los Angeles (us-west2)Salt Lake City (us-west3)Las Vegas (us-west4)
Date | Time | Description | |
---|---|---|---|
| 2 Nov 2022 | 11:47 PDT | The issue with Google Compute Engine has been resolved for all affected users as of Wednesday, 2022-11-02 08:50 US/Pacific. We thank you for your patience while we worked on resolving the issue. |
| 28 Oct 2022 | 15:34 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: We have completed all short term mitigation work to prevent impact to customers. VMs with Local SSD may still see a >0.008% chance of being terminated. We have identified a fix for the remaining failures, and are working on a plan to roll out the fix, with our next update by Friday, 2022-11-04 16:00 US/Pacific. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Customers can reduce the possibility of being impacted by setting automaticRestart to false when scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false. For VMs that are terminated, restarting the VM will restore the VM back to a working state. |
| 27 Oct 2022 | 15:44 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Friday, 2022-10-28 16:30 US/Pacific. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false. |
| 27 Oct 2022 | 10:12 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: Our engineering team continues to investigate the issue. The impact of this issue was experienced beginning at Thursday, 2022-09-22 03:14 US/Pacific. We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false. |
| 27 Oct 2022 | 01:20 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false. |
| 26 Oct 2022 | 22:40 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Set the automaticRestart=false for scheduling the VM under host maintenance policy of a VM. |
| 26 Oct 2022 | 22:32 PDT | Summary: VMs using Local SSD are experiencing intermittent terminations. Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Wednesday, 2022-10-26 23:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine. Workaround: Set the automaticRestart=false for scheduling the VM under host maintenance policy of a VM. |
- All times are US/Pacific