Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case https://cloud.google.com/support or help article https://support.google.com/a/answer/1047213.

(All Times US/Pacific)

Incident Start: 05 June 2022 12:17

Incident End: 05 June 2022 16:14

Duration: 3 hours, 57 minutes

Affected Services and Features:

Cloud IAM

Regions/Zones: Global

Description:

Cloud IAM experienced elevated delays in propagating changes to Cloud IAM policies due to an unexpected increase in deletion traffic.

Customer Impact:

• Affected customers would experience delays in processing Cloud IAM policy changes and the use of group membership changes in IAM policy evaluation.

The issue with Identity and Access Management has been resolved for all affected projects as of Sunday, 2022-06-05 16:15 US/Pacific.

We thank you for your patience while we worked on resolving the issue.

Summary: Global: Elevated delays in propagating changes to Cloud IAM policies and group memberships

Description: Mitigation work is still underway by our engineering team. We are seeing improvements in several regions globally.

The mitigation is expected to complete by Sunday, 2022-06-05 17:00 US/Pacific.

We will provide more information by Sunday, 2022-06-05 17:00 US/Pacific.

Diagnosis: Affected customers when making IAM policy or group membership changes will experience increased delays before they take effect. Current delays are up to 45 minutes.

Workaround: None at this time.

Summary: Global: Elevated delays in propagating changes to Cloud IAM policies and group memberships

Description: Mitigation work is currently underway by our engineering team.

We do not have an ETA for mitigation at this point.

We will provide more information by Sunday, 2022-06-05 17:00 US/Pacific.

Diagnosis: Affected customers when making IAM policy or group membership changes will experience increased delays before they take effect. Current delays are up to 45 minutes.

Workaround: None at this time.

Summary: Global: Elevated delays in propagating changes to Cloud IAM policies and group memberships

Description: We are experiencing an issue with Identity and Access Management.

Our engineering team continues to investigate the issue.

We will provide an update by Sunday, 2022-06-05 16:45 US/Pacific with current details.

Diagnosis: Affected customers when making IAM policy or group membership changes will experience increased delays before they take effect. Current delays are up to 30 minutes.

Workaround: None at this time.

Summary: Global: Elevated delays in propagating changes to Cloud IAM policies and group memberships

Description: We are experiencing an issue with Identity and Access Management.

Our engineering team continues to investigate the issue.

We will provide an update by Sunday, 2022-06-05 16:30 US/Pacific with current details.

Diagnosis: Affected customers when making IAM policy or group membership changes will experience increased delays before they take effect. Current delays are up to 30 minutes.

Workaround: None at this time.

Summary: Global: Elevated delays in propagating changes to Cloud IAM Policies

Description: We are experiencing an issue with Identity and Access Management beginning at Sunday, 2022-06-05 09:15 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Sunday, 2022-06-05 16:00 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Affected customers when making IAM policies changes will experience increased delays before they take effect.

Workaround: None at this time.