Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Agent Assist, Cloud Machine Learning, Dialogflow CX

Dialogflow CX, and Agent Assist are experiencing elevated error rates and time outs with Generative features in multiple regions

Incident began at 2024-06-07 08:56 and ended at 2024-06-07 10:17 (all times are US/Pacific).

Previously affected location(s)

Tokyo (asia-northeast1)Mumbai (asia-south1)Singapore (asia-southeast1)Sydney (australia-southeast1)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)GlobalMontréal (northamerica-northeast1)Iowa (us-central1)South Carolina (us-east1)Oregon (us-west1)

Date Time Description
7 Jun 2024 14:25 PDT

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support .

(All Times US/Pacific)

Incident Start: 7 June, 2024 08:56

Incident End: 7 June, 2024 10:17

Duration: 1 hour, 21 minutes

Affected Services and Features: Agent Assist Dialogflow CX

Regions/Zones: Multi-Regional

Description: Dialogflow CX, and Agent Assist experienced elevated error rates and timeouts with Generative features in multiple regions for a duration of 1 hour and 20 minutes. From preliminary analysis the root cause has been identified as a specific workload which caused overload to a backend service. All services were mitigated by limiting the specific workload, decreasing utilization which resulted in recovery.

Customer Impact:

During the incident timeframe impacted customers experienced: Dialogflow CX and Agent Assist users observed increased error rates, long latency for queries with generative features enabled and timeouts.
7 Jun 2024 10:34 PDT

The issue with Agent Assist, Dialogflow CX has been resolved for all affected users as of Friday, 2024-06-07 10:34 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
7 Jun 2024 10:15 PDT

Summary: Dialogflow CX, and Agent Assist are experiencing elevated error rates and time outs with Generative features in multiple regions

Description: Our engineering team has identified the cause and performed mitigation actions. Our internal monitoring indicates the error rate subsided.

Our engineers are continuing to monitor our systems closely

We will provide an update by Friday, 2024-06-07 10:50 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis:

  • Dialogflow CX is experiencing elevated error rates and time outs with Generative features enabled.
  • Agent Assist's Knowledge Assist and Knowledge Search features are also impacted

Workaround: None at this time.
7 Jun 2024 09:47 PDT

Summary: Dialogflow CX, and Agent Assist are experiencing elevated error rates and time outs with Generative feature in multiple regions

Description: We are experiencing an issue with Dialogflow CX, Agent Assist beginning at Friday, 2024-06-07 08:56 US/Pacific.

Our engineering team continues to investigate the issue.

We will provide an update by Friday, 2024-06-07 10:30 US/Pacific with current details.

We apologize to all who are affected by the disruption.

Diagnosis:

  • Dialogflow CX is experiencing elevated error rates and time outs with Generate Text requests
  • Agent Assist's Knowledge Assist and Knowledge Search features are also impacted

Workaround: None at this time.