Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Cloud Spanner

us-central1, us-west1, us-west1, us-east1: Cloud Spanner missing stackdriver CPU, memory and disk metrics.

Incident began at 2021-11-12 18:45 and ended at 2021-11-13 20:39 (all times are US/Pacific).

Date Time Description
15 Nov 2021 13:26 PST

We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: 12 November 2021 18:45

Incident End: 13 November 2021 20:39

Duration: 1 day, 1 hours, 54 minutes

Affected Services and Features:

  • Cloud Spanner - Missing/Incorrect Cloud Monitoring metrics for CPU, memory and disk

Regions/Zones:

  • us-central1
  • us-east1
  • Global

Description:

Cloud Spanner experienced unavailable and incorrect monitoring metrics for instance resources including cpu, memory and disk for a duration of 1 day, 1 hours and 54 minutes. From preliminary analysis, the root cause of the issue is a rollout flag which incorrectly modified how metrics are exported.

Customer Impact:

  • us-central1: Experienced missing metrics during the impact duration until mitigation at 13 November 2021 18:30
  • us-east1: Experienced missing metrics during the impact duration until mitigation at 13 November 2021 17:45
  • All other regions: Many customers globally experienced metrics that incorrectly under-reported actual values until full resolution at 13 November 2021 20:39
13 Nov 2021 21:17 PST

The issue with Cloud Spanner has been resolved for all affected users as of Saturday, 2021-11-13 21:16 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
13 Nov 2021 16:47 PST

Summary: us-central1, us-west1, us-west1, us-east1: Cloud Spanner missing stackdriver CPU, memory and disk metrics.

Description: Mitigation work is currently underway by our engineering team.

Current data indicates that customers in us-central1, us-west1, us-west1 and us-east1 are affected by this issue.

We will provide more information by Saturday, 2021-11-13 23:00 US/Pacific.

Diagnosis: Missing Stackdriver metrics on the dashboard.

Workaround: None at this time.