Google Cloud Service Health

Google Cloud Service Health
Incidents
Workspace creation and release configs failing in certain regions

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Available
Service information
Service disruption
Service outage

Incident affecting Dataform

Workspace creation and release configs failing in certain regions

Incident began at 2023-12-11 01:00 and ended at 2023-12-13 09:56 (all times are US/Pacific).

Previously affected location(s)

Taiwan (asia-east1)Singapore (asia-southeast1)Belgium (europe-west1)Frankfurt (europe-west3)Zurich (europe-west6)South Carolina (us-east1)Oregon (us-west1)

	Date	Time	Description
	13 Dec 2023	09:56 PST	The fix roll out completed to asia-east1, and us-east1 regions as of 2023-12-13 08:45 US/Pacific. The fix roll out completed to us-west1, and europe-west1 regions as of 2023-12-13 09:04 US/Pacific. The fix roll out completed to asia-southeast1 and europe-west3 regions as of 2023-12-13 09:23 US/Pacific. The fix roll out to rest of the regions completed as of Wednesday, 2023-12-13 09:50 US/Pacific, resolving the issue with Dataform for all affected users We thank you for your patience while we worked on resolving the issue.
	13 Dec 2023	09:20 PST	Summary: Workspace creation and release configs failing in certain regions Description: Mitigation work is currently underway by our engineering team. The fix roll out completed to asia-east1, and us-east1 regions as of 2023-12-13 08:45 US/Pacific and to us-west1, and europe-west1 regions as of 2023-12-13 09:04. Users in these four regions should no longer encounter the issue. Full mitigation is expected to complete by Wednesday, 2023-12-13 11:15 US/Pacific. We will provide more information by Wednesday, 2023-12-13 11:20 US/Pacific. Diagnosis: Some customers can't create workspaces and have release configs which fail to compile in the affected regions. Workflow config schedules continue to run against the latest compiled result from the underlying release config. Workaround: None at this time. For the updates on this issue prior to this communication, please refer to our prior post at https://status.cloud.google.com/incidents/YAYwqvoSrqRZZjznUJmC. We apologize for any confusion caused by the two separate links

Date

Time

Description

13 Dec 2023

09:56 PST

The fix roll out completed to asia-east1, and us-east1 regions as of 2023-12-13 08:45 US/Pacific.

The fix roll out completed to us-west1, and europe-west1 regions as of 2023-12-13 09:04 US/Pacific.

The fix roll out completed to asia-southeast1 and europe-west3 regions as of 2023-12-13 09:23 US/Pacific.

The fix roll out to rest of the regions completed as of Wednesday, 2023-12-13 09:50 US/Pacific, resolving the issue with Dataform for all affected users

We thank you for your patience while we worked on resolving the issue.

13 Dec 2023

09:20 PST

Summary: Workspace creation and release configs failing in certain regions

Description: Mitigation work is currently underway by our engineering team.

The fix roll out completed to asia-east1, and us-east1 regions as of 2023-12-13 08:45 US/Pacific and to us-west1, and europe-west1 regions as of 2023-12-13 09:04. Users in these four regions should no longer encounter the issue.

Full mitigation is expected to complete by Wednesday, 2023-12-13 11:15 US/Pacific.

We will provide more information by Wednesday, 2023-12-13 11:20 US/Pacific.

Diagnosis: Some customers can't create workspaces and have release configs which fail to compile in the affected regions. Workflow config schedules continue to run against the latest compiled result from the underlying release config.

Workaround: None at this time.

For the updates on this issue prior to this communication, please refer to our prior post at https://status.cloud.google.com/incidents/YAYwqvoSrqRZZjznUJmC. We apologize for any confusion caused by the two separate links

All times are US/Pacific