Google Cloud Service Health

Google Cloud Service Health
Incidents
Cloud Logging Ingestion delay, causing logs to be delayed.

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Available
Service information
Service disruption
Service outage

Incident affecting Operations

Cloud Logging Ingestion delay, causing logs to be delayed.

Incident began at 2021-03-23 10:09 and ended at 2021-03-23 18:14 (all times are US/Pacific).

Date	Time	Description
23 Mar 2021	18:14 PDT	The issue with Cloud Logging has been resolved for all affected projects as of Tuesday, 2021-03-23 18:14 US/Pacific. We thank you for your patience while we worked on resolving the issue.
23 Mar 2021	16:25 PDT	Description: We are still seeing delays decreasing. We estimate that we are down to 25% of projects using the global region impacted. We are still waiting for the backlog in us-west1 to clear. We will provide more information by Tuesday, 2021-03-23 18:00 US/Pacific. Diagnosis: Logs may be delayed. Current estimated delays in the following region: 2.2h us-west1 Workaround: None at this time.
23 Mar 2021	15:35 PDT	Description: We are still seeing some recovery and latencies decreasing. The backlog in us-east1 has now cleared and we are estimating the backlog for us-west1 to clear in about 2 hours. We will provide more information by Tuesday, 2021-03-23 16:30 US/Pacific. Diagnosis: Logs may be delayed. Current estimated delays in the following region: 2.3h us-west1 Workaround: None at this time.
23 Mar 2021	14:48 PDT	Description: Mitigation work continues by our engineering team. We are starting to see some recovery and latencies decreasing. Current data indicates that approximately 75% of projects in the global region and about 90% of projects in us-east1 and us-west1 are affected by this issue. We will provide more information by Tuesday, 2021-03-23 15:45 US/Pacific. Diagnosis: Logs may be delayed. Current estimated delays in the following regions: 2.5h us-west1, 0.8h us-east1. Workaround: None at this time.
23 Mar 2021	14:00 PDT	Description: Mitigation work continues by our engineering team. Current data indicates that approximately 75% of projects in multiple regions are affected by this issue. The mitigation is now expected to in place by Tuesday, 2021-03-23 14:30 US/Pacific. However it may take some additional time for the backlog to catch up. We will provide more information by Tuesday, 2021-03-23 14:45 US/Pacific. Diagnosis: Logs may be delayed. Current estimated delays in the following regions: 2.3h us-west1, 1.8h us-east1. us-central1 should not be impacted. Workaround: None at this time.
23 Mar 2021	13:14 PDT	Description: Mitigation work is still underway by our engineering team. Current data indicates that approximately 75% of projects in multiple regions are affected by this issue. The mitigation is expected to in place by Tuesday, 2021-03-23 13:30 US/Pacific. However it may take some additional time for the backlog to catch up. We will provide more information by Tuesday, 2021-03-23 14:00 US/Pacific. Diagnosis: Logs appear delayed. Workaround: None at this time.

All times are US/Pacific