Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Virtual Private Cloud (VPC), Google Cloud Networking, Cloud NAT

Virtual Private Cloud public IP packet loss affecting GCE instances in multiple regions

Incident began at 2023-06-06 07:35 and ended at 2023-06-06 08:35 (all times are US/Pacific).

Previously affected location(s)

Taiwan (asia-east1)Singapore (asia-southeast1)Belgium (europe-west1)

Date Time Description
6 Jun 2023 14:02 PDT

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support .

(All Times US/Pacific)

Impact Window #1:
Region: asia-southeast1
Incident Start: 6 June 2023 at 07:35
Incident End: 6 June 2023 at 07:58
Duration: 23 minutes

Impact Window #2:
Region: asia-east1
Incident Start: 6 June 2023 at 08:02
Incident End: 6 June 2023 at 08:18
Duration: 16 minutes

Impact Window #3:
Region: europe-west1
Incident Start: 6 June 2023 at 08:08
Incident End: 6 June 2023 at 08:35
Duration: 27 minutes

Affected Services and Features:

Google Cloud Networking - Cloud VPN, Cloud NAT, Virtual Private Cloud (VPC)

Regions/Zones: asia-southeast1, asia-east1, europe-west1

Description:

Google Cloud Networking experienced elevated packet loss and latency which impacted multiple Cloud Networking services in asia-southeast1, asia-east1, europe-west1 for a maximum duration of 27 minutes. From preliminary analysis, the root cause was a configuration change for one of the load balancers, which directed traffic for VM public IPs to an incorrect data plane component.

Customer Impact:

Google Cloud Networking

  • VM to VM public IP and VM to External TCP/UDP Network Load Balancer traffic experienced up to ~50% packet loss across above three regions spanning the same durations.

Cloud VPN

  • Affected customers experienced up to ~55% packet loss across above three regions spanning the same durations.

Cloud NAT

  • Affected customers experienced up to ~60% packet loss across above three regions spanning the same durations.
6 Jun 2023 09:02 PDT

The issue with Virtual Private Cloud (VPC) has been resolved for all affected projects as of Tuesday, 2023-06-06 08:47 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
6 Jun 2023 09:00 PDT

Summary: Virtual Private Cloud public IP packet loss affecting GCE instances in multiple regions

Description: We are experiencing an issue with Virtual Private Cloud (VPC) beginning at Tuesday, 2023-06-06 07:35 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Tuesday, 2023-06-06 10:06 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Packet loss when connecting to GCE instances or Network Load Balancers using public IP addresses

Workaround: None at this time.