Google Cloud Service Health

Google Cloud Service Health
Incidents
Create Backup operations are not finishing on time while using Cloud Spanner

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Available
Service information
Service disruption
Service outage

Incident affecting Cloud Spanner

Create Backup operations are not finishing on time while using Cloud Spanner

Incident began at 2023-03-21 01:18 and ended at 2023-03-21 08:12 (all times are US/Pacific).

Previously affected location(s)

Taiwan (asia-east1)Hong Kong (asia-east2)Tokyo (asia-northeast1)Osaka (asia-northeast2)Seoul (asia-northeast3)Mumbai (asia-south1)Delhi (asia-south2)Singapore (asia-southeast1)Jakarta (asia-southeast2)Multi-region: asia1Sydney (australia-southeast1)Melbourne (australia-southeast2)Multi-region: eur3Multi-region: eur5Warsaw (europe-central2)Finland (europe-north1)Madrid (europe-southwest1)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Netherlands (europe-west4)Zurich (europe-west6)Milan (europe-west8)Paris (europe-west9)Tel Aviv (me-west1)Multi-region: nam-eur-asia1Multi-region: nam10Multi-region: nam11Multi-region: nam12Multi-region: nam13Multi-region: nam3Multi-region: nam6Multi-region: nam7Multi-region: nam8Multi-region: nam9Montréal (northamerica-northeast1)Toronto (northamerica-northeast2)São Paulo (southamerica-east1)Santiago (southamerica-west1)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)Columbus (us-east5)Dallas (us-south1)Oregon (us-west1)Los Angeles (us-west2)Salt Lake City (us-west3)Las Vegas (us-west4)

Date	Time	Description
21 Mar 2023	08:12 PDT	The issue with Cloud Spanner has been resolved for all affected users as of Tuesday, 2023-03-21 07:40 US/Pacific. We thank you for your patience while we worked on resolving the issue.
21 Mar 2023	06:16 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific, some Cloud Spanner backups were taking longer than expected. This issue potentially affects backups created before approximately 2023-03-21 03:00 US/Pacific globally and is currently ongoing. Mitigation work should be completed by Tuesday, 2023-03-21 09:00 US/Pacific We will provide an update by Tuesday, 2023-03-21 09:00 US/Pacific with current details. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	05:11 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific, some Cloud Spanner backups were taking longer than expected. This issue potentially affects backups created before approximately 2023-03-21 03:00 US/Pacific globally and is currently ongoing. Mitigation work is still underway by our engineering team. We will provide an update by Tuesday, 2023-03-21 06:20 US/Pacific with current details. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	04:51 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific, some Cloud Spanner backups were taking longer than expected. This issue potentially affects backups created before approximately 2023-03-21 03:00 US/Pacific globally and is currently ongoing. Engineers are working on a resolution. We will provide an update by Tuesday, 2023-03-21 05:20 US/Pacific with current details. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	03:55 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific, some Cloud Spanner backups were taking longer than expected. This issue potentially affects backups created before approximately 2023-03-21 03:00 US/Pacific globally and is currently ongoing. Engineers are working on a resolution. We will provide an update by Tuesday, 2023-03-21 05:00 US/Pacific with current details. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	03:25 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific, some Cloud Spanner backup were taking longer than expected to complete while creating a database backup. This issue is occurring globally and is currently ongoing. Engineers are working on a resolution. We will provide an update by Tuesday, 2023-03-21 04:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	03:20 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific Cloud Spanner backup were taking longer than expected to complete while creating a database backup. This issue is occurring globally and is currently ongoing. Engineers are working on a resolution. We will provide an update by Tuesday, 2023-03-21 03:35 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	03:13 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: Starting at approximately 2023-03-20 15:30 US/Pacific Cloud Spanner backup were taking longer than expected to complete while creating a database backup. This issue is occurring globally and is currently ongoing. Engineers are working on a resolution. We will provide an update by Tuesday, 2023-03-21 03:25 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Cloud Spanner backup were taking longer than expected to complete while creating a database backup. Workaround: None at this time.
21 Mar 2023	03:04 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: We are experiencing an issue with Cloud Spanner beginning at Monday, 2023-03-20 15:30 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2023-03-21 03:25 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users may observe indefinite delay while creating a database backup using Cloud Spanner. Workaround: None at this time.
21 Mar 2023	02:54 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: We are experiencing an issue with Cloud Spanner beginning at Monday, 2023-03-20 15:30 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2023-03-21 03:15 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users may observe indefinite delay while creating a database backup using Cloud Spanner. Workaround: None at this time.
21 Mar 2023	01:48 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: We are experiencing an issue with Cloud Spanner beginning at Monday, 2023-03-20 15:30 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2023-03-21 03:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users may observe indefinite delay while creating a database backup using Cloud Spanner. Workaround: None at this time.
21 Mar 2023	01:34 PDT	Summary: Create Backup operations are not finishing on time while using Cloud Spanner Description: We are experiencing an issue with Cloud Spanner beginning at Monday, 2023-03-20 15:30 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2023-03-21 02:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users may observe delay or incomplete operation while creating a backup. Workaround: None at this time.

All times are US/Pacific