Service Health
For incidents related to Google Security Products, visit https://status.cloud.google.com/security. For incidents related to Looker (original), visit https://status.cloud.google.com/looker.
Incident affecting VMWare engine
Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions.
Incident began at 2026-07-14 10:00 and ended at 2026-07-14 21:46 (all times are US/Pacific).
Previously affected location(s)
Sydney (australia-southeast1)Melbourne (australia-southeast2)Frankfurt (europe-west3)
| Date | Time | Description | |
|---|---|---|---|
| | 14 Jul 2026 | 22:34 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers may have experienced zonal outages impacting network connectivity across multiple regions. Description We experienced an inter-site communication issue with Google Cloud VMware Engine (GCVE) Stretched Cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We identified the affected customers, and have worked with them to fully mitigate the issue by Tuesday, 2026-07-14 at 21:46 PDT. Preliminary analysis indicates that a network configuration update was the cause of the inter-zone network disruption. Our engineering team mitigated the issue by rolling back the faulty configuration to its last-known good value. If customers are still experiencing impact from this issue, please contact Support, and we will work with you to resolve any residual impact. We thank you for your patience while we worked to resolve this issue. Customer Symptoms Some GCVE customers using Stretched Cluster may have experienced inter-site communication failures to their GCVE environments within the affected zones. Workaround The issue is now mitigated. |
| | 14 Jul 2026 | 20:27 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. A remediation rollout is currently in progress to address the underlying network issue. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 22:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following to restore access to your workloads:
|
| | 14 Jul 2026 | 18:55 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. A remediation rollout is currently in progress to address the underlying network issue. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 20:30 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 18:15 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. A remediation rollout is currently in progress to address the underlying network issue. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 19:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 17:13 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. A remediation rollout is currently in progress to address the underlying network issue. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 18:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Upon further investigation we identified that the northamerica-northeast2 region was not impacted. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 16:05 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. Our investigation has identified a recent configuration update that is the likely cause of the inter-zone network disruption. Teams are working on remediation. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 17:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 15:16 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. Our investigation has identified underlying inter-zone communication failures and Border Gateway Protocol (BGP) session flapping between cluster zones. Specifically, network connectivity has been lost between the affected zones and the witness appliance. Because the witness appliance is currently unreachable, the cluster zones are unable to safely synchronize state. As a result, VMs on the affected sites are becoming isolated and may be left without writable data. Failover / VM Migration: Migrating affected VMs to the healthy, secondary zone of your stretch cluster remains the primary mitigation strategy. Because of the complexities surrounding failover risks and secondary zone health, we highly encourage you to open a ticket with Google Cloud Support if you are severely impacted. We do not currently have an ETA for resolution. We will provide another update by Tuesday, 2026-07-14 16:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 14:31 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an inter-site communication issue with Google Cloud VMware Engine (GCVE) stretched cluster customers beginning Tuesday, 2026-07-14 at 10:00 PDT. We have identified the customers that are affected by this issue and we are working with them on mitigation. Our preliminary investigation indicates this is stemming from an underlying network connectivity issue affecting the infrastructure that links the zones within a stretch cluster. This disruption is causing synchronization issues between the affected zones. We believe Storage and Compute services remain unaffected, and VMs are running as expected, though connectivity to them may be degraded. We will provide another update by Tuesday, 2026-07-14 15:00 PDT with current details. Customer Symptoms Some GCVE customers using Stretched Cluster may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround While we work on restoring full connectivity, we recommend the following workarounds to restore access to your workloads:
|
| | 14 Jul 2026 | 13:24 PDT | Summary Google Cloud VMware Engine (GCVE) Stretched Cluster customers are experiencing zonal outages impacting network connectivity across multiple regions. Description We are experiencing an issue with Google Cloud VMware Engine (GCVE) beginning Tuesday, 2026-07-14 at 10:00 PDT. We are continuing to investigate and mitigate the issue. We believe this disruption is isolated to network connectivity issues for stretch clusters, while Storage and Compute services appear to be unaffected. GCVE VMs are running as expected but customers may experience connectivity issues to the VMs. We will provide another update by Tuesday, 2026-07-14 14:00 PDT with current details. Customer Symptoms Some GCVE customers may experience inter-site communication failures to their GCVE environments within the affected zones. Workaround None at this time. |
- All times are US/Pacific