Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support .

(All Times US/Pacific)

Incident Start: 8 July, 2024 23:20

Incident End: 12 July, 2024 06:31

Duration: 3 days, 7 hours, 11 minutes

Affected Services and Features: reCAPTCHA Enterprise

Regions/Zones: Global

Description:

A change in the client code of reCAPTCHA Enterprise caused requests being sent to customer servers instead of reCAPTCHA servers. As a result, customers experienced a significant number of unexpected 4xx errors. A small number of customers reported handling those errors in a way that made reCAPTCHA unusable for them.

A rollout was initiated on Monday, 8 July 2024 23:20 US/Pacific however, impact was limited leading up to Wednesday, 10 July 2024 16:45 US/Pacific. Affected customers experienced extraneous requests on ~50% of reCAPTCHA scripts for 10 hours until the change was rolled back, fully mitigating the issue for all users.

Customer Impact:

Affected customers may have experienced requests sent to non-existent URIs in their servers causing 404 errors. A small number of customers reported handling the errors in a way that made reCAPTCHA unusable for them.

The issue with reCAPTCHA Enterprise has been resolved for all affected users as of Friday, 2024-07-12 06:31 US/Pacific.

We thank you for your patience while we worked on resolving the issue.

Summary: reCAPTCHA Enterprise elevated error rate issue

Description: We believe the issue with reCAPTCHA Enterprise is related to a recent update. Our engineering team is actively working to mitigate the problem by rolling back this change.

The mitigation is expected to be completed by Friday, 2024-07-12 09:00 US/Pacific.

We will provide more information by Friday, 2024-07-12 09:30 US/Pacific.

Diagnosis: Impacted customers may experience requests sent to non-existent URIs in their servers causing 404s and recaptcha not working as a result.

Workaround: None at this time.

Summary: reCAPTCHA Enterprise elevated error rate issue

Description: We believe the issue with reCAPTCHA Enterprise is related to a recent update. Our engineering team is actively working to mitigate the problem by rolling back this change.

The mitigation is expected to be completed by Friday, 2024-07-12 09:00 US/Pacific.

We will provide more information by Friday, 2024-07-12 09:30 US/Pacific.

Diagnosis: Impacted customers may experience requests sent to non-existent URIs in their servers causing 404s and recaptcha not working as a result.

Workaround: None at this time.

Summary: reCAPTCHA Enterprise elevated error rate issue

Description: We believe the issue with reCAPTCHA Enterprise is related to a recent update. Our engineering team is actively working to mitigate the problem by rolling back this change.

The mitigation is expected to completed by Friday, 2024-07-12 09:00 US/Pacific.

We will provide more information by Friday, 2024-07-12 09:30 US/Pacific.

Diagnosis: Impacted customers may experience requests sent to non-existent URIs in their servers causing 404s and recaptcha not working as a result.

Workaround: None at this time.

Summary: reCAPTCHA Enterprise elevated error rate issue

Description: We are experiencing an issue with reCAPTCHA Enterprise.

Our engineering team continues to investigate the issue.

We will provide an update by Friday, 2024-07-12 06:01 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis: Impacted customers may experience 404 error while using reCAPTCHA Enterprise

Workaround: None at this time.