Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Cloud Monitoring, Operations

Monarch Tcube Query Availability drop in cloud_prod

Incident began at 2022-12-01 04:10 and ended at 2022-12-01 05:19 (all times are US/Pacific).

Previously affected location(s)

Global

Date Time Description
1 Dec 2022 11:41 PST

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: December 1st, 2022 04:10

Incident End: December 1st, 2022 05:19

Duration: 1 hour, 09 minutes

Affected Services and Features:

Google Cloud Monitoring

Regions/Zones: Global

Description:

Google Cloud Monitoring experienced intermittent responses for failed repository data requests for 1 hour and 09 minutes. From preliminary analysis, the root cause of the issue was a component containing a permission change in database schema in a recent rollout. The issue was mitigated by rolling back to a previous good version in all zones.

Customer Impact:

  • Affected customers may have received partial error notifications for repository data requests between 04:10 and 04:29.
  • Between 04:29 and 05:00, customers received complete failures to query repository data.
  • Between 05:00 and 05:19, while systems were recovering, customers received partial error notifications for repository data requests.
1 Dec 2022 05:21 PST

The issue with Cloud Monarch has been resolved for all affected users as of Thursday, 2022-12-01 05:19 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
1 Dec 2022 05:08 PST

Summary: Monarch Tcube Query Availability drop in cloud_prod

Description: We are experiencing an issue with Cloud Monarch.

Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point.

We will provide an update by Thursday, 2022-12-01 05:43 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Customers are experiencing a returned partial error for repository data requests

Workaround: None.