Service Health
Incident affecting AutoML Translation, Cloud Machine Learning
Customers using AutoML translations may experience high latency and 'resource exhausted' failures.
Incident began at 2024-08-02 09:30 and ended at 2024-08-02 17:36 (all times are US/Pacific).
Previously affected location(s)
Netherlands (europe-west4)Iowa (us-central1)
Date | Time | Description | |
---|---|---|---|
| 2 Aug 2024 | 17:36 PDT | Our Engineering team has successfully concluded validations and confirmed that issue has been mitigated. If you have questions or require further assistance, please open a case with the Support Team and we will work with you to ensure that your concerns are addressed. We thank you for your patience while we worked on resolving the issue. |
| 2 Aug 2024 | 14:58 PDT | Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is in the process of validating the mitigation further, to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 19:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time. |
| 2 Aug 2024 | 11:53 PDT | Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is still validating further to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 15:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time. |
| 2 Aug 2024 | 10:18 PDT | Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is currently validating further to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 12:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time. |
| 2 Aug 2024 | 09:56 PDT | Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We are experiencing an issue with AutoML translations Our engineering team continues to investigate the issue. We will provide an update by Friday, 2024-08-02 11:08 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: AutoML translations may experience high latency and resource exhausted failures. Workaround: None at this time. |
| 2 Aug 2024 | 09:49 PDT | Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We are experiencing an issue with Cloud Translation. Our engineering team continues to investigate the issue. We will provide an update by Friday, 2024-08-02 11:08 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: None at this time. Workaround: None at this time. |
- All times are US/Pacific