Google Cloud Service Health

Google Cloud Service Health
Incidents
We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Available
Service information
Service disruption
Service outage

Incident affecting Vertex AI AutoML Image, Vertex AI Online Prediction, Vertex AI Batch Prediction, Cloud Machine Learning

We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.

Incident began at 2022-12-27 18:04 and ended at 2022-12-28 04:02 (all times are US/Pacific).

Previously affected location(s)

Iowa (us-central1)

Date	Time	Description
28 Dec 2022	04:02 PST	The issue with Vertex AI AutoML Image, Vertex AI Batch Prediction, Vertex AI Online Prediction has been resolved for all affected projects as of Wednesday, 2022-12-28 04:02 US/Pacific. We thank you for your patience while we worked on resolving the issue.
27 Dec 2022	23:27 PST	Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction. Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Wednesday, 2022-12-28 10:00 US/Pacific with current details. Diagnosis: Users might experience higher latency. Workaround: None.
27 Dec 2022	23:05 PST	Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction. Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2022-12-27 23:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users might experience higher latency. Workaround: None.
27 Dec 2022	19:59 PST	Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction. Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2022-12-27 23:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users might experience higher latency. Workaround: None.
27 Dec 2022	18:04 PST	Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction. Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2022-12-27 20:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Users might experience higher latency. Workaround: None.

All times are US/Pacific