Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Networking

Cloud Networking issues in us-east1

Incident began at 2019-07-02 10:25 and ended at 2019-07-03 12:00 (all times are US/Pacific).

Date Time Description
3 Jul 2019 12:00 PDT

The disruption with Google Cloud Networking and Load Balancing has been resolved as of Wednesday, 2019-07-03 07:35 US/Pacific.

The physical damage to the fiber bundles serving some network paths in us-east1 have been repaired, and we have removed the elective routing mitigations previously in place and returned to normal routing in the region.

We will perform an internal review of the incident and make appropriate improvements so that our customers continue to be able to operate reliably during incidents such as this.
2 Jul 2019 16:05 PDT

The disruptions with Google Cloud Networking and Load Balancing have been root caused to physical damage to multiple concurrent fiber bundles serving network paths in us-east1, and we expect a full resolution within the next 24 hours.

In the meantime, we are electively rerouting traffic to ensure that customers' services will continue to operate reliably until the affected fiber paths are repaired. Some customers may observe elevated latency during this period.

We will provide another status update either as the situation warrants or by Wednesday, 2019-07-03 12:00 US/Pacific tomorrow.
2 Jul 2019 14:31 PDT

The disruptions with Google Cloud Networking and Load Balancing have been root caused to physical damage to multiple concurrent fiber bundles serving network paths in us-east1. We are electively rerouting some traffic to ensure that customers’ services will continue to operate reliably until the affected fiber paths are repaired. Some customers will observe elevated latency during this period.

We will provide another status update by Tuesday, 2019-07-02 16:30 US/Pacific with current details.
2 Jul 2019 12:39 PDT

Mitigation work is currently underway to address the issue with Google Cloud Networking and Load Balancing in us-east1. The rate of errors has decreased, however some users may still notice elevated latency, and our Engineering Team is continuing to work on it. We will provide another status update by Tuesday, 2019-07-02 14:30 US/Pacific with current details.
2 Jul 2019 11:35 PDT

Mitigation work is currently underway by our Engineering Team to address the issue with Google Cloud Networking and Load Balancing in us-east1. The rate of errors is decreasing, however some users may still notice elevated latency. We will provide another status update by Tuesday, 2019-07-02 12:30 US/Pacific with current details.
2 Jul 2019 10:56 PDT

We continue to experience an issue with Cloud Networking and Load balancing within us-east1. Issue is currently mitigated and we are working towards a resolution. Customers may still observe traffic through Global Load balancers being directed away from backends in us-east1 at this time. We will provide an update by Tuesday, 2019-07-02 11:29 US/Pacific with current details.
2 Jul 2019 10:25 PDT

We continue to experience an issue with Cloud Networking within us-east1. Issue is currently mitigated and we are working towards a resolution. Customers may still observe traffic through Global Load-balancers being directed away from back-ends in us-east1 at this time. We will provide an update by Tuesday, 2019-07-02 11:29 US/Pacific with current details.