Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Networking

US-WEST1: Multiple cloud products experiencing network issues

Incident began at 2022-01-08 15:14 and ended at 2022-01-08 18:36 (all times are US/Pacific).

Date Time Description
10 Jan 2022 18:34 PST

Mini Incident Report (Full Incident Report To Follow)

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: 08 January 2022 15:14

Incident End: 08 January 2022 18:36

Duration: 3 hours, 22 minutes

Affected Services and Features:

  • Cloud Router
  • Cloud VPN
  • Cloud DNS
  • Cloud AI
  • Cloud Run
  • Cloud Spanner
  • Persistent DIsk

Regions/Zones: us-west1-b

Description: Multiple GCP products experienced network issues in the us-west1 region for 3 hours and 22 minutes. From preliminary analysis, the root cause of the issue is related to an unexpected port/switch configuration and we are continuing investigations.

Customer Impact:

  • Cloud Router - Customers may experience elevated packet loss.
  • Cloud DNS - Increased latency and/or errors reaching out to Cloud DNS
  • Cloud AI - Elevated prediction latencies
  • Cloud Run - Network connectivity issues
  • Cloud Spanner - Elevated network latencies
  • Persistent Disk - Short period of slow I/O operations on disks, which may have manifested as unresponsive -Compute Engine instances.
8 Jan 2022 18:42 PST

The issue with Cloud Networking has been resolved for all affected projects as of Saturday, 2022-01-08 18:36 US/Pacific.

We will publish an analysis of this incident once we have completed our internal investigation.

We thank you for your patience while we worked on resolving the issue.

8 Jan 2022 18:02 PST

Summary: US-WEST1: Multiple cloud products experiencing network issues

Description: We are experiencing network issues in us-west1.

Below is the list of affected products:

  • Cloud Router - Customers may experience elevated packet loss.
  • Cloud DNS - Increased latency and/or errors reaching out to Cloud DNS
  • Cloud AI - Elevated prediction latencies
  • Cloud Run - Network connectivity issues
  • Cloud Spanner - Elevated network latencies

Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Saturday, 2022-01-08 18:53 US/Pacific.

Diagnosis: Affected customer will see packet loss.

Workaround: None at this time.

8 Jan 2022 17:49 PST

Summary: US-WEST1: Multiple cloud products experiencing network issues

Description: We are experiencing network issues in us-west1.

Below is the list of affected products:

  • Cloud Router - Customers may experience elevated packet loss.
  • Cloud DNS - Increased latency and/or errors reaching out to Cloud DNS
  • Cloud AI - Elevated prediction latencies
  • Cloud Run - Network connectivity issues

Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Saturday, 2022-01-08 18:53 US/Pacific.

Diagnosis: Affected customer will see packet loss.

Workaround: None at this time.

8 Jan 2022 16:54 PST

Summary: Multiple cloud connectivity products experiencing packet loss in US-WEST1

Description: Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Saturday, 2022-01-08 17:53 US/Pacific.

Diagnosis: Affected customer will see packet loss.

Workaround: None at this time.