Google Cloud Service Health

Google Cloud Service Health
Incidents
Legacy AutoML Tables training jobs failing

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Available
Service information
Service disruption
Service outage

Incident affecting AutoML Tables, Cloud Machine Learning

Legacy AutoML Tables training jobs failing

Incident began at 2024-02-15 06:27 and ended at 2024-02-15 13:04 (all times are US/Pacific).

Previously affected location(s)

Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Netherlands (europe-west4)Iowa (us-central1)Los Angeles (us-west2)

Date	Time	Description
15 Feb 2024	13:04 PST	We now have confirmation that the issue with AutoML Tables has been resolved for all affected projects as of Thursday, 2024-02-15 11:00 US/Pacific, when the mitigation was applied. We thank you for your patience while we worked on resolving the issue.
15 Feb 2024	12:53 PST	Summary: Legacy AutoML Tables training jobs failing Description: After further investigation, our engineers has identified that more validation is required in order to confirm the effectiveness of the mitigation that was implemented, considering multiple intricacies. Please be assured that we are continuing to work on this with highest priority. We will provide more information by Thursday, 2024-02-15 15:30 US/Pacific. Diagnosis: Legacy AutoML Tables training jobs will fail with "Internal Error" Workaround: None at this time.
15 Feb 2024	11:53 PST	Summary: Legacy AutoML Tables training jobs failing Description: Our Engineers are still performing validation, and we would be requiring additional time to validate. Please be assured that we are working on this with highest priority. We will provide more information by Thursday, 2024-02-15 13:00 US/Pacific. Diagnosis: Legacy AutoML Tables training jobs will fail with "Internal Error" Workaround: None at this time.
15 Feb 2024	10:29 PST	Summary: Legacy AutoML Tables training jobs failing Description: A fix has been rolled out to production by our Engineering team. We are currently in the process of validating the results to ensure the issue has been fully mitigated in production. We will provide more information by Thursday, 2024-02-15 11:30 US/Pacific. Diagnosis: Legacy AutoML Tables training jobs will fail with "Internal Error" Workaround: None at this time.
15 Feb 2024	07:56 PST	Summary: Legacy AutoML Tables training jobs failing Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-02-15 10:30 US/Pacific. Diagnosis: Legacy AutoML Tables training jobs will fail with "Internal Error" Workaround: None at this time.
15 Feb 2024	06:54 PST	Summary: Legacy AutoML Tables training jobs failing Description: We are experiencing an issue with Legacy AutoML Tables. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2024-02-15 08:05 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Legacy AutoML Tables training jobs will fail with "Internal Error" Workaround: None at this time.

All times are US/Pacific