Service Health

The issue with Cloud SQL API instance mutations and SQL Proxy operations has been resolved for all affected users as of Friday, 2021-01-29 18:45 US/Pacific.

All operations including instance restart and delete are healthy.

We thank you for your patience while we worked on resolving the issue.

Description: The issue with Cloud SQL API instance mutations and SQL Proxy operations in us-central1 has partially recovered. We believe impact began at 08:50 US/Pacific with the error rate increasing starting at 09:25 US/Pacific. Partial recovery occurred at 14:54 US/Pacific

Our engineering team has rolled out a mitigation and partially resolved the issue. We believe most of the service has recovered. At this time we believe only instances.delete and instances.restart continue to be impacted. Some operations such as instances.patch which require an instance.restart may also be impacted. The product team has identified the root cause of the issue and has begun implementing a fix.

The product team has started rolling out the potential fix. We are monitoring the effects of this rollout to determine if the fix has solved the issue.

We will provide an update by Friday, 2021-01-29 20:00 US/Pacific with current details.

Diagnosis: Customers impacted by this issue may see increased latency or errors when performing Instance mutation requests, such as create, delete and update. Connection to Cloud SQL instances may be disrupted if a new ephemeral cert is required by the Cloud SQL Proxy. Errors returned may be 502 or 503.

Workaround: None at this time. We believe other regions are not impacted at this time.

Description: The issue with Cloud SQL API instance mutations and SQL Proxy operations in us-central1 has partially recovered. We believe impact began at 08:50 US/Pacific with the error rate increasing starting at 09:25 US/Pacific. Partial recovery occurred at 14:54 US/Pacific

Our engineering team has rolled out a mitigation and partially resolved the issue. We believe most of the service has recovered. At this time we believe only instances.delete and instances.restart continue to be impacted. Some operations such as instances.patch which require an instance.restart may also be impacted. The product team has identified the root cause of the issue and has begun implementing a fix. We expect that the Cloud SQL API will remain in the current partially resolved state for several hours until the fix is fully verified and rolled out.

We will provide an update by Friday, 2021-01-29 19:00 US/Pacific with current details.

Diagnosis: Customers impacted by this issue may see increased latency or errors when performing Instance mutation requests, such as create, delete and update. Connection to Cloud SQL instances may be disrupted if a new ephemeral cert is required by the Cloud SQL Proxy. Errors returned may be 502 or 503.

Workaround: None at this time. We believe other regions are not impacted at this time.

Description: The issue with Cloud SQL API instance mutations and SQL Proxy operations in us-central1 has partially recovered. We believe impact began at 08:50 US/Pacific with the error rate increasing starting at 09:25 US/Pacific. Partial recovery occurred at 14:54 US/Pacific

Our engineering team has rolled out a mitigation and partially resolved the issue. We believe most of the service has recovered. At this time we believe only instances.delete and instances.restart continue to be impacted. Some operations such as instances.patch which require an instance.restart may also be impacted. We are working on further mitigation to bring the service back to a nominal operating state.

We will provide an update by Friday, 2021-01-29 16:30 US/Pacific with current details.

Diagnosis: Customers impacted by this issue may see increased latency or errors when performing Instance mutation requests, such as create, delete and update. Connection to Cloud SQL instances may be disrupted if a new ephemeral cert is required by the Cloud SQL Proxy. Errors returned may be 502 or 503.

Workaround: None at this time. We believe other regions are not impacted at this time.

Description: We are experiencing an issue with Cloud SQL API instance mutations and SQL Proxy operations in us-central1. We believe impact began at 08:50 US/Pacific with the error rate increasing starting at 09:25 US/Pacific.

Our engineering team continues to investigate the issue. Instance mutation operations (such as instances.create or instances.delete) as well as requests for certificates (sslCerts.createEphemeral) when using Cloud SQL Proxy are impacted. Existing connections to Cloud SQL instances should be unaffected. New connections without Cloud SQL Proxy are also unaffected.

The product team has identified a potential mitigation and is working to implement it. We believe the change will be finished rolling out by 15:00 US/Pacific, we will be continuously monitoring the service to evaluate the effectiveness of the mitigation.

We will provide an update by Friday, 2021-01-29 15:30 US/Pacific with current details.

Diagnosis: Customers impacted by this issue may see increased latency or errors when performing Instance mutation requests, such as create, delete and update. Connection to Cloud SQL instances may be disrupted if a new ephemeral cert is required by the Cloud SQL Proxy. Errors returned may be 502 or 503.

Workaround: None at this time. We believe other regions are not impacted at this time.

Description: We are experiencing an issue with Cloud SQL API instance mutations in us-central1. We believe impact began at 08:50 US/Pacific with the error rate increasing starting at 09:25 US/Pacific.

Our engineering team continues to investigate the issue. Instance mutation operations (such as instances.create or instances.delete) as well as requests for certificates (sslCerts.createEphemeral) when using Cloud SQL Proxy are impacted. Existing connections to Cloud SQL instances should be unaffected.

The product team has identified a potential mitigation and is working to implement it. As part of the mitigation, all requests to instances delete and instances restart will receive an error.

We will provide an update by Friday, 2021-01-29 15:00 US/Pacific with current details.

Diagnosis: Customers impacted by this issue may see increased latency or errors when performing Instance mutation requests, such as create, delete and update. Connection to Cloud SQL instances may be disrupted if a new ephemeral cert is required by the Cloud SQL Proxy. Errors returned may be 502 or 503.

Workaround: None at this time. We believe other regions are not impacted at this time.

Description: We are experiencing an issue with Cloud SQL.

Our engineering team continues to investigate the issue. New instance creation, other instance operations, and SQL queries to already existing instances may be failing. We are troubleshooting the error at this time.

We will provide an update by Friday, 2021-01-29 16:00 US/Pacific with current details.

Diagnosis: Instance creation, operations, and queries in us-central1 region may result in 502 (backend timeout) or 503 (service unavailable) errors.

Customers may also experience Cloud SQL Proxy connectivity related issues, due in part to this incident.

Workaround: None at this time.

Description: We are experiencing an issue with Cloud SQL.

Our engineering team continues to investigate the issue. New instance creation, other instance operations, and SQL queries to already existing instances may be failing.

We will provide an update by Friday, 2021-01-29 13:00 US/Pacific with current details.

Diagnosis: Instance creation, operations, and queries in us-central1 region may result in 502 (backend timeout) or 503 (service unavailable) errors.

Workaround: None at this time.

Description: We are experiencing an issue with Cloud SQL.

Our engineering team continues to investigate the issue. New instance creation, other instance operations, and SQL queries to already existing instances may be failing. We are troubleshooting the error at this time.

We will provide an update by Friday, 2021-01-29 16:00 US/Pacific with current details.

Diagnosis: Instance creation, operations, and queries in us-central1 region may result in 502 (backend timeout) or 503 (service unavailable) errors.

Workaround: None at this time.