Service Health
-
-
-
-
Incident affecting Google Cloud Networking, Cloud Load Balancing
Issue resolved: HTTP load balancing returning error for some users in Europe.
Incident began at 2021-07-27 04:15 and ended at 2021-07-27 21:05 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| | 28 Jul 2021 | 08:12 PDT | We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support (All Times US/Pacific) Incident Timeline: 27 June 2021 04:15 - 05:33 27 June 2021 06:56 - 07:41 Total Duration: 2 hours 3 minutes Affected Services and Features: Google Cloud Load Balancing Regions/Zones: Western Europe Description: Google Cloud Load Balancing (GCLB) experienced increased TLS handshake errors or connection terminations for HTTPS traffic served from load balancers in Western Europe for a duration of 2 hours, 3 minutes. From preliminary analysis, the root cause of the issue is a configuration change that resulted in an overload of an internal component responsible for handling part of the traffic requests. Customer Impact: Increased TLS handshake errors or connection terminations for GCLB traffic served from HTTPS load balancers in Western Europe. Additional details: The issue was fully resolved on 27 June 2021 21:05 US/Pacific, and we are confident that there will not be a recurrence. |
| | 27 Jul 2021 | 21:05 PDT | The issue with HTTP load balancing has been resolved for all affected users in Europe as of Tuesday, 2021-07-27 19:02 US/Pacific. We thank you for your patience while we worked on resolving the issue. |
| | 27 Jul 2021 | 09:27 PDT | Summary: HTTP load balancing returning error for some user in Europe. Description: We believe the issue with Cloud Networking is resolved and there is no further impact. Engineering team has mitigated the current impact and expect the full resolution to be rolled out within next 12 hours. We will provide an update by Tuesday, 2021-07-27 21:30 US/Pacific with current details. Diagnosis: None at this time Workaround: None at this time |
| | 27 Jul 2021 | 08:24 PDT | Summary: HTTP load balancing returning error for some user in Europe. Description: We believe the issue with Cloud Networking is resolved. We do not have an ETA for full resolution at this point. We will provide an update by Tuesday, 2021-07-27 15:30 US/Pacific with current details. Diagnosis: none at this time Workaround: none at this time |
| | 27 Jul 2021 | 08:22 PDT | Summary: HTTP load balancing returning error for some user in Europe. Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Tuesday, 2021-07-27 09:57 US/Pacific. Diagnosis: none at this time Workaround: none at this time |
| | 27 Jul 2021 | 07:51 PDT | Summary: HTTP load balancing returning error for some user in Europe. Description: We are experiencing an intermittent issue with Cloud Networking beginning at Tuesday, 2021-07-27 06:56:46 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2021-07-27 08:20 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: none at this time Workaround: none at this time |
- All times are US/Pacific