Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Operations

The issue with Cloud Logging has been resolved for all affected projects.

Incident began at 2021-08-18 11:15 and ended at 2021-08-18 15:06 (all times are US/Pacific).

Date Time Description
19 Aug 2021 12:03 PDT

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support

(All Times US/Pacific)

Incident Start: 18 August 2021 11:15

Incident End: 18 August 2021 15:06

Duration: 3 hours, 51 minutes

Affected Services and Features: Cloud Logging

Regions/Zones: Global

Description:

Cloud Logging experienced increased read errors globally for 3 hours, 51 minutes. From preliminary analysis, the root cause of the issue was due to a large increase in work from an internal service which overwhelmed the available capacity on a backend database. Log ingestion was not impacted.

Customer Impact:

  • Customers experienced elevated read failures when attempting to query logs.
  • Customers were unable to access logs in the Cloud Console due to query timeouts.

Additional details:

The issue has been mitigated and the problematic service has been identified and canceled. Query operations have returned to normal.

18 Aug 2021 15:48 PDT

The issue with Cloud Logging has been resolved for all affected projects as of Wednesday, 2021-08-18 15:30 US/Pacific.

We thank you for your patience while we worked on resolving the issue.

18 Aug 2021 14:51 PDT

Summary: We are experiencing elevated read error rates with Cloud Logging in multiple regions.

Description: Mitigation work is currently underway by our engineering team.

The mitigation is expected to complete by Wednesday, 2021-08-18 16:00 US/Pacific.

We will provide more information by Wednesday, 2021-08-18 16:00 US/Pacific.

Diagnosis: Some customers may be unable to view logging in the Cloud Console.

Workaround: None at this time.

18 Aug 2021 14:38 PDT

Summary: We are experiencing elevated read error rates with Cloud Logging in multiple regions.

Description: We are experiencing elevated read error rates with Cloud Logging in multiple regions beginning at Wednesday, 2021-08-18 11:15 US/Pacific.

Our engineering team is working on mitigation options.

We will provide an update by Wednesday, 2021-08-18 15:15 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Some customers may be unable to view logging in the Cloud Console.

Workaround: None at this time.

18 Aug 2021 14:14 PDT

Summary: We are experiencing elevated read error rates with Cloud Logging in multiple regions.

Description: We are experiencing elevated read error rates with Cloud Logging in multiple regions beginning at Wednesday, 2021-08-18 11:15 US/Pacific.

Our engineering team is exploring mitigation options.

We will provide an update by Wednesday, 2021-08-18 14:45 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Some customers may be unable to view logging in the Cloud Console.

Workaround: None at this time.

18 Aug 2021 13:42 PDT

Summary: We are experiencing elevated read error rates with Cloud Logging in multiple regions

Description: We are experiencing elevated read errors with Cloud Logging in multiple regions beginning at Wednesday, 2021-08-18 11:15 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Wednesday, 2021-08-18 14:15 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Customers may be unable to view logging in the Cloud Console.

Workaround: None at this time.