Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Chronicle Security

Chronicle Security - WORKSPACE_ACTIVITY data ingestion observed delays in asia-southeast1 & asia-south1 regions

Incident began at 2025-01-30 16:53 and ended at 2025-01-31 06:13 (all times are US/Pacific).

Previously affected location(s)

Mumbai (asia-south1)Singapore (asia-southeast1)

Date Time Description
31 Jan 2025 12:13 PST

# Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: 30 January 2025, 16:53

Incident End: 31 January 2025 06:13

Duration: 13 hours, 20 minutes

Affected Services and Features:

Chronicle Security

Regions/Zones:

asia-south1, asia-southeast1

Description:

Chronicle Security WORKSPACE_ACTIVITY data ingestion observed delays in asia-southeast1 and asia-south1 regions for 13 hours, 20 minutes due to network connectivity issues between the trans-Pacific regions.

While the network connectivity issues are being fixed, the data ingestion delays were mitigated by increasing the Network Quality of Service (QoS) for this traffic to ensure timely processing.

Customer Impact:

  • Chronicle Security customers observed WORKSPACE_ACTIVITY data was delayed in the Secops instance. The impacted regions processed only around one third of the actual traffic.
  • Rule detections which depend on WORKSPACE_ACTIVITY data may have been delayed.
  • Search on WORKSPACE_ACTIVITY data may have shown fewer events due to ingestion delays.
31 Jan 2025 06:14 PST

The issue with Chronicle Security has been resolved for all affected users as of Friday, 2025-01-31 06:13 US/Pacific.

We thank you for your patience while we worked on resolving the issue.

31 Jan 2025 05:50 PST

Summary: Chronicle Security - WORKSPACE_ACTIVITY data ingestion observed delays in asia-southeast1 & asia-south1 regions

Description: We are experiencing an issue with Chronicle Security beginning on Thursday, 2025-01-30 16:53 US/Pacific.

Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Friday, 2025-01-31 07:00 US/Pacific.

Diagnosis: Customers impacted by this issue will see that WORKSPACE_ACTIVITY data is delayed in the Secops instance. Rule detections which depend on this data will be delayed. Search on this data will show fewer events.

Workaround: None at this time.

31 Jan 2025 02:33 PST

Summary: Chronicle Security - WORKSPACE_ACTIVITY data ingestion observed delays in asia-southeast1 & asia-south1 regions

Description: We are experiencing an issue with Chronicle Security beginning on Thursday, 2025-01-30 16:53 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Friday, 2025-01-31 12:00 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Customers impacted by this issue will see that WORKSPACE_ACTIVITY data is delayed in the Secops instance. Rule detections which depend on this data will be delayed. Search on this data will show fewer events.

Workaround: None at this time.

31 Jan 2025 01:05 PST

Summary: Chronicle Security - WORKSPACE_ACTIVITY data ingestion observed delays in asia-southeast1 & asia-south1 regions

Description: We are experiencing an issue with Chronicle Security beginning on Thursday, 2025-01-30 16:53 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Friday, 2025-01-31 03:00 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Customers impacted by this issue will see that WORKSPACE_ACTIVITY data is delayed in the Secops instance. Rule detections which depend on this data will be delayed. Search on this data will show fewer events.

Workaround: None at this time.