Console

Service Health

This page provides status information on the services that are part of Google Cloud. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit https://cloud.google.com/.

Incident affecting Google Cloud Composer

Cloud Composer v2 environments autoscaling might not work.

Incident began at 2022-11-17 04:22 and ended at 2022-11-18 10:06 (all times are US/Pacific).

Previously affected location(s)

Hong Kong (asia-east2)Tokyo (asia-northeast1)Osaka (asia-northeast2)Seoul (asia-northeast3)Mumbai (asia-south1)Singapore (asia-southeast1)Sydney (australia-southeast1)Warsaw (europe-central2)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Zurich (europe-west6)Montréal (northamerica-northeast1)São Paulo (southamerica-east1)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)Oregon (us-west1)Los Angeles (us-west2)Salt Lake City (us-west3)Las Vegas (us-west4)

Date Time Description
18 Nov 2022 10:06 PST

The issue with Google Cloud Composer has been resolved for all affected users as of Friday, 2022-11-18 10:05 US/Pacific.

We thank you for your patience while we worked on resolving the issue.
18 Nov 2022 10:02 PST

Summary: Cloud Composer v2 environments autoscaling might not work.

Description: The issue is partially mitigated. If your environment is impacted by this problem you can upgrade the Composer environment to a newer version to apply the fix.

The Composer engineering team continues to work out a solution to apply a fix for the impacted Composer environments.

We will provide an update by Friday, 2022-11-18 12:00 US/Pacific with current details.

Diagnosis: Number of Airflow workers in the Composer environment won’t change.

Workaround: - Check if an upgrade is available for your Composer environment. If it’s available then perform the update operation to the available version.

  • If an upgrade for your environment is not available or you don’t want to perform the update then you can increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
18 Nov 2022 08:27 PST

Summary: Cloud Composer v2 environments autoscaling might not work.

Description: The issue is partially mitigated. If your environment is impacted by this problem you can upgrade the Composer environment to a newer version to apply the fix.

The Composer engineering team continues to work out a solution to apply a fix for the impacted Composer environments.

We will provide an update by Friday, 2022-11-18 10:00 US/Pacific with current details.

Diagnosis: Number of Airflow workers in the Composer environment won’t change.

Workaround: - Check if an upgrade is available for your Composer environment. If it’s available then perform the update operation to the available version.

  • If an upgrade for your environment is not available or you don’t want to perform the update then you can increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
18 Nov 2022 06:26 PST

Summary: Cloud Composer v2 environments autoscaling might not work.

Description: The issue is partially mitigated. If your environment is impacted by this problem you can upgrade the Composer environment to a newer version to apply the fix.

The Composer engineering team continues to work out a solution to apply a fix for the impacted Composer environments.

We will provide an update by Friday, 2022-11-18 09:00 US/Pacific with current details.

Diagnosis: Number of Airflow workers in the Composer environment won’t change.

Workaround: - Check if an upgrade is available for your Composer environment. If it’s available then perform the update operation to the available version.

  • If an upgrade for your environment is not available or you don’t want to perform the update then you can increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
18 Nov 2022 02:15 PST

Summary: Cloud Composer v2 environments autoscaling might not work.

Description: The issue is partially mitigated. If your environment is impacted by this problem you can upgrade the Composer environment to a newer version to apply the fix.

The Composer engineering team continues to work out a solution to apply a fix for the impacted Composer environments.

We will provide an update by Friday, 2022-11-18 07:00 US/Pacific with current details.

Diagnosis: Number of Airflow workers in the Composer environment won’t change.

Workaround: - Check if an upgrade is available for your Composer environment. If it’s available then perform the update operation to the available version.

  • If an upgrade for your environment is not available or you don’t want to perform the update then you can increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
17 Nov 2022 14:04 PST

Summary: Cloud Composer v2 environments autoscaling might not work.

Description: The issue is partially mitigated. If your environment is impacted by this problem you can upgrade the Composer environment to a newer version to apply the fix.

The Composer engineering team continues to work out a solution to apply a fix for the impacted Composer environments so upgrading without a need for performing the upgrade to fix the issue.

We will provide an update by Friday, 2022-11-18 05:00 US/Pacific with current details.

Diagnosis: Number of Airflow workers in the Composer environment won’t change.

Workaround: - Check if an upgrade is available for your Composer environment. If it’s available then perform the update operation to the available version.

  • If an upgrade for your environment is not available or you don’t want to perform the update then you can increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
17 Nov 2022 13:14 PST

Summary: Cloud Composer v2 environments autoscaling might not work

Description: Composer 2 environments upgraded to newer Composer versions might experience problems with autoscaling. The issue might occur if upgrade operation was performed between October 27th and November 17th.

Users should not upgrade Composer environments to newer versions until further notice.

Our engineering team continues to investigate the issue.

We will provide an update by Thursday, 2022-11-17 14:00 US/Pacific with current details.

Diagnosis: Airflow workers number will not change.

Workaround: Increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
17 Nov 2022 09:51 PST

Summary: Cloud Composer v2 environments autoscaling might not work

Description: Composer 2 environments upgraded to newer Composer versions might experience problems with autoscaling. The issue might occur if upgrade operation was performed between October 27th and November 17th.

Users should not upgrade Composer environments to newer versions until further notice.

Our engineering team continues to investigate the issue.

We will provide an update by Thursday, 2022-11-17 13:00 US/Pacific with current details.

Diagnosis: Airflow workers number will not change.

Workaround: Increase the minimum number of workers to the expected level so autoscaling is not needed to process the DAGs/tasks in your Composer environment.
17 Nov 2022 05:25 PST

Summary: Cloud Composer v2 environments autoscaling might not work

Description: Composer 2 environments upgraded to newer Composer versions might experience problems with autoscaling. The issue might occur if upgrade operation was performed between October 27th and November 17th.

Users should not upgrade Composer environments to newer versions until further notice.

Our engineering team continues to investigate the issue.

We will provide an update by Thursday, 2022-11-17 10:00 US/Pacific with current details.

Diagnosis: Airflow workers number will not change

Workaround: None at this time