Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.

(All Times US/Pacific)

Incident Start: 24 May, 2024 03:53

Incident End: 24 May, 2024 09:30

Duration: 5 hours, 36 minutes

Affected Services and Features: Google Distributed Cloud Edge

Regions/Zones: Global

Description:

Google Distributed Cloud Edge users were unable to execute cluster creation and cluster upgrade globally for a duration of 5 hours, 36 minutes.

From preliminary analysis, the root cause of the issue was activation of “safe mode” in a backend system that GDCE depends upon for initiation of certain management operations. These management operations that are blocked by the safe mode activation include certain components of the workflows corresponding to cluster creation and upgrade.

Customer Impact:

During the incident timeframe users in GDC connected regions were unable to

• Execute cluster creation
• Upgrade clusters

The incident only prevented the operations listed above from being executed, it did not affect existing clusters or normal cluster operations.

The issue with Google Distributed Cloud Edge has been resolved for all affected users as of Friday, 2024-05-24 09:36 US/Pacific.

We thank you for your patience while we worked on resolving the issue.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 11:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 09:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 08:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 07:30 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 07:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 06:30 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 06:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 05:30 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.

Summary: EdgeContainer API cannot create and upgrade clusters

Description: We are experiencing an issue with Google Distributed Cloud Edge beginning on Friday, 2024-05-24 03:38 US/Pacific.

Mitigation work is still underway by our engineering team.

We will provide an update by Friday, 2024-05-24 05:00 US/Pacific with current details.

Diagnosis: Inability to exercise cluster creation and cluster upgrade in all regions of GDC connected

Workaround: None at this time.