Google Security Products Status Dashboard
-
-
-
-
Incident affecting Google SecOps
Rules that were enabled with real-time run frequency were not processed for a period of time.
Incident began at 2025-04-22 10:00 and ended at 2025-05-15 07:30 (all times are US/Pacific).
Previously affected location(s)
Tokyo (asia-northeast1)Mumbai (asia-south1)Singapore (asia-southeast1)Sydney (australia-southeast1)Multi-region: europeTurin (europe-west12)London (europe-west2)Frankfurt (europe-west3)Zurich (europe-west6)Paris (europe-west9)Doha (me-central1)Dammam (me-central2)Tel Aviv (me-west1)Toronto (northamerica-northeast2)São Paulo (southamerica-east1)Multi-region: us
| Date | Time | Description | |
|---|---|---|---|
| | 15 May 2025 | 09:45 PDT | Summary : Reprocessing to create missed detections has completed Description : Our Engineering team has completed the rule reprocessing for all affected rules in all regions as of Thursday, 2025-05-15 07:30 US/Pacific. Any detections that were missed have been created by our engineering teams. We thank you for your patience while working on resolving the issue. |
| | 14 May 2025 | 11:58 PDT | Summary : Reprocessing to create missed detections has been launched - Interim Update Description : Our Engineering team has completed the rule reprocessing for all affected rules, in the following regions : me-central1, europe-west2, europe-west12, asia-south1, southamerica-east1, northamerica-northeast2, australia-southeast1, me-west1, me-central2, europe-west6, europe-west3, asia-southeast1, asia-northeast1 Any detections that were missed have been created by our engineering teams. Reprocessing to create missed detections in the US and Europe regions, is still in progress. The revised ETA for the mitigation on US and Europe regions is, Thursday 2025-05-15 12:00 US/Pacific We will provide more information by Thursday, 2025-05-15 12:30 US/Pacific. Customer Symptoms : If the customer's tenant is in the US or Europe regions, processing for affected rules was skipped for 3 days - between Tuesday 22 April 2025 to Friday 25 April 2025. If the customers’ tenant is any other region (e.g. europe-west2, europe-west3, europe-west6, europe-west9, europe-west12, or any non-europe region like asia-southeast1), processing was skipped for up to 15 days - between Tuesday 22 April 2025 to Wednesday 07 May 2025. The missed detections have now been created as mentioned above. Workaround : Customers can also manually launch retrohunts over the affected time range for rules that meet the criteria mentioned above to regenerate detections now. |
| | 13 May 2025 | 11:38 PDT | Summary : Reprocessing to create missed detections has been launched. Description : Our engineering team has launched the reprocessing for all affected rules in all affected regions, to create missing detections. ETA for reprocessing to complete is as follows:
We will provide more information by Wednesday, 2025-05-14 12:30 US/Pacific. Customer Symptoms : If the customer's tenant is in the US or Europe regions, processing for affected rules was skipped for 3 days - between Tuesday 22 April 2025 to Friday 25 April 2025. If the customers’ tenant is any other region (e.g. europe-west2, europe-west3, europe-west6, europe-west9, europe-west12, or any non-europe region like asia-southeast1), processing was skipped for up to 15 days - between Tuesday 22 April 2025 to Wednesday 07 May 2025. Workaround : Customers can also manually launch retrohunts over the affected time range for rules that meet the criteria mentioned above to regenerate detections now. |
| | 12 May 2025 | 12:15 PDT | Summary : Rules that were enabled with real-time run frequency were not processed for a period of time. Description : Due to a configuration issue, Rules that were enabled with real-time run frequency prior to the start of the impact time range were not processed, beginning on Tuesday, 22 April 2025 at 10:00 US/Pacific. Affected rules have resumed generating detections and the issue was fixed as of Wednesday 07 May 2025 at 15:00 US/Pacific. Our engineering team is working on reprocessing all affected rules to generate the missed detections from within that time range, and will follow up when reprocessing is complete. We will provide more information by Tuesday, 2025-05-13 12:00 US/Pacific. Customer Symptoms : If the customer's tenant is in the US or Europe regions, processing for affected rules was skipped for 3 days - between Tuesday 22 April 2025 to Friday 25 April 2025. If the customers’ tenant is any other region (e.g. europe-west2, europe-west3, europe-west6, europe-west9, europe-west12, or any non-europe region like asia-southeast1), processing was skipped for up to 15 days - between Tuesday 22 April 2025 to Wednesday 07 May 2025. Workaround : Customers can also manually launch retrohunts over the affected time range for rules that meet the criteria mentioned above to regenerate detections now. |
- All times are US/Pacific