Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Operations, Cloud Logging

Cloud Logging customers experiencing issues while querying logs

Incident began at 2023-12-03 18:26 and ended at 2023-12-03 20:28 (all times are US/Pacific).

Previously affected location(s)

GlobalNorthern Virginia (us-east4)

Date Time Description
4 Dec 2023 10:01 PST

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support. (All Times US/Pacific)

Incident Start: 3 Dec 2023 16:00 Incident End: 3 Dec 2023 20:22 Duration: 4 hours, 22 minutes

Reoccurrence Start: 3 Dec 2023 22:25 Recurrence End: 4 Dec 2023 01:18 Recurrence Duration: 2 hours, 53 minutes

Affected Services and Features: Cloud Logging

Regions/Zones: Global

Description:

Impacted users were unable to load the most recent logs in Google Cloud Console UI in Log Explorer or while querying logs using API for the log entries that were ingested during the outage (historical queries still worked). From preliminary investigation, the root cause has been identified as a log processing component being in a non-functional state, which was triggered by a cluster migration stuck in a bad state.

Customer Impact:

Around 12 percent of user queries were failing globally for log entries ingested in the interval from 16:00 to 20:22 and from 22:25 to 01:18 the next day.
3 Dec 2023 20:28 PST

The issue with Cloud Logging has been resolved for all affected users as of Sunday, 2023-12-03 20:22 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
3 Dec 2023 20:23 PST

Summary: Cloud Logging customers experiencing issues while querying logs

Description: Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Sunday, 2023-12-03 21:00 US/Pacific.

Diagnosis: Impacted users may not be able to load most recent logs in Google Cloud Console UI in Log Explorer or while querying logs using API.

Workaround: None at this time.
3 Dec 2023 19:56 PST

Summary: Cloud Logging customers experiencing issues while querying logs

Description: We are experiencing an issue with Cloud Logging beginning at Sunday, 2023-12-03 16:00 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Sunday, 2023-12-03 20:30 US/Pacific with current details.

Diagnosis: Impacted users may not be able to load most recent logs in Google Cloud Console UI in Log Explorer or while querying logs using API.

Workaround: None at this time.
3 Dec 2023 19:36 PST

Summary: Cloud Logging customers with log buckets in us-east4 and global regions experiencing issues while querying logs

Description: We are experiencing an issue with Cloud Logging beginning at Sunday, 2023-12-03 16:00 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Sunday, 2023-12-03 20:30 US/Pacific with current details.

Diagnosis: Impacted users may not be able to load most recent logs in Google Cloud Console UI in Log Explorer or while querying logs using API. All customers who have log buckets in us-east4 and few customers with log buckets in global location are experiencing this issue.

Workaround: None at this time.