Google Cloud Status Dashboard
This page provides status information on the services that are part of Google Cloud Platform. Check back here to view the current status of the services listed below. For additional information on these services, please visit cloud.google.com.
Google Compute Engine Incident #16013
HTTP(S) Load Balancing 502 Errors
Incident began at 2016-07-29 10:37 (all times are US/Pacific).
Date | Time | Description | |
---|---|---|---|
Jul 29, 2016 | 10:37 | We are investigating an issue with 502 errors from HTTP(S) Load Balancing. We will provide more information by 11:05 US/Pacific. |
|
We are investigating an issue with 502 errors from HTTP(S) Load Balancing. We will provide more information by 11:05 US/Pacific. |
|||
Jul 19, 2014 | 10:34 | SUMMARY: On Wednesday 16 July 2014, for a period of 75 minutes newly created Google Compute Engine instances were unable to accept inbound network connections for up to 15 minutes after their creation. If you were impacted, we apologize — this is not the reliability we strive to offer, and we have taken immediate steps to improve the platform. DETAILED DESCRIPTION OF IMPACT: The incident affected Compute Engine users who created new instances. Beginning at 13:00 US/Pacific there was a period after instance creation during which the instance was unable to accept inbound network connections. By 14:15 this issue was resolved; all existing incidents were able to accept connections and new instances were once again able to accept connections immediately after creation. Existing virtual machine instances continued to operate normally throughout the incident. ROOT CAUSE: The portion of the system responsible for allocating Compute Engine instance IP addresses became overloaded, preventing the system which configures software defined network routes from reading the necessary information to configure the network fabric. REMEDIATION AND PREVENTION: To resolve the immediate issue, Google engineers redirected traffic to datacenters that were performing better, reducing the replication load and resolving the issue. Afterward, Google engineers optimized the tasks which read from the IP address management system to increase future capacity. Finally, Google engineers are improving the monitors around this system in order to alert engineers before high load on the system becomes an issue. |
|
SUMMARY: On Wednesday 16 July 2014, for a period of 75 minutes newly created Google Compute Engine instances were unable to accept inbound network connections for up to 15 minutes after their creation. If you were impacted, we apologize — this is not the reliability we strive to offer, and we have taken immediate steps to improve the platform. DETAILED DESCRIPTION OF IMPACT: The incident affected Compute Engine users who created new instances. Beginning at 13:00 US/Pacific there was a period after instance creation during which the instance was unable to accept inbound network connections. By 14:15 this issue was resolved; all existing incidents were able to accept connections and new instances were once again able to accept connections immediately after creation. Existing virtual machine instances continued to operate normally throughout the incident. ROOT CAUSE: The portion of the system responsible for allocating Compute Engine instance IP addresses became overloaded, preventing the system which configures software defined network routes from reading the necessary information to configure the network fabric. REMEDIATION AND PREVENTION: To resolve the immediate issue, Google engineers redirected traffic to datacenters that were performing better, reducing the replication load and resolving the issue. Afterward, Google engineers optimized the tasks which read from the IP address management system to increase future capacity. Finally, Google engineers are improving the monitors around this system in order to alert engineers before high load on the system becomes an issue. |
|||
Jul 16, 2014 | 15:04 | The problem with network connectivity to newly created Compute Engine instances should be resolved as of 14:15 US/Pacific. We apologize for any issues this may have caused you or your users and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better. We will provide a more detailed analysis of this incident once we have completed our internal investigation. |
|
The problem with network connectivity to newly created Compute Engine instances should be resolved as of 14:15 US/Pacific. We apologize for any issues this may have caused you or your users and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better. We will provide a more detailed analysis of this incident once we have completed our internal investigation. |
|||
Jul 16, 2014 | 14:38 | We're investigating an issue with network connectivity to newly created Compute Engine instances beginning at approximately 13:00 US/Pacific time. We will provide more information shortly. |
|
We're investigating an issue with network connectivity to newly created Compute Engine instances beginning at approximately 13:00 US/Pacific time. We will provide more information shortly. |