Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting VMWare engine, Google Compute Engine

vSphere vCenter and NSX may not have been available for a short period of time in us-central1

Incident began at 2022-06-23 00:13 and ended at 2022-06-23 06:36 (all times are US/Pacific).

Previously affected location(s)

Iowa (us-central1)

Date Time Description
23 Jun 2022 14:57 PDT

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case https://cloud.google.com/support or help article https://support.google.com/a/answer/1047213.

(All Times US/Pacific)

Incident Start: 23 June 2022 00:13 PT

Incident End: 23 June 2022 06:36 PT

Duration: 6 hours, 23 minutes

Affected Services and Features:

VMWare Engine

Regions/Zones: us-central1a

Description:

Google Cloud customers using VMWare Engine experienced connectivity issues while using vSphere, vCenter and NSX in us-central1a zone for a duration of 6 hours, 23 minutes.

From preliminary analysis, the issue was caused due to a network switch-over not working as expected during a planned maintenance window of ToR switches. The issue was mitigated by performing a file system recovery and restart of related services on the resources hosted by Google Cloud VMware Engine.

Customer Impact:

  • Affected Customers may have observed issues with network partitions and accessing workload VMs.
  • Affected Customers may have experienced connectivity issues using vCenter, HCX, NSX Manager, and NSX-T Edge virtual machines.

Additional details:

Google engineers have paused the maintenance upgrade and also all redundant devices have been brought back to operational state. We are also working with the impacted customers to check the state of remaining workload VMs and shall assist in recovery if required.
23 Jun 2022 06:36 PDT

The issue with VMWare engine has been resolved for all affected projects as of Thursday, 2022-06-23 06:36 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
23 Jun 2022 04:02 PDT

Summary: vSphere vCenter and NSX may not have been available for a short period of time in us-central1

Description: We are experiencing an issue with VMWare engine beginning at Thursday, 2022-06-23 07:13 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Thursday, 2022-06-23 09:30 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: vSphere vCenter and NSX may not have been available for a short period of time in us-central1, vCenter VMs has been recovered. Some of the Workload VMs might be still impacted.

Workaround: None at this time.