Service Health
Incident affecting Cloud Workstations, Google Compute Engine, Google Kubernetes Engine, Persistent Disk, Google Cloud Console
Issue with Google Cloud Console, Cloud Workstation, Cloud Shell, GKE and Persistent Disk.
Incident began at 2024-02-27 10:56 and ended at 2024-02-29 11:19 (all times are US/Pacific).
Previously affected location(s)
Johannesburg (africa-south1)Taiwan (asia-east1)Hong Kong (asia-east2)Tokyo (asia-northeast1)Osaka (asia-northeast2)Seoul (asia-northeast3)Mumbai (asia-south1)Delhi (asia-south2)Singapore (asia-southeast1)Jakarta (asia-southeast2)Sydney (australia-southeast1)Melbourne (australia-southeast2)Warsaw (europe-central2)Finland (europe-north1)Madrid (europe-southwest1)Belgium (europe-west1)Berlin (europe-west10)Turin (europe-west12)London (europe-west2)Frankfurt (europe-west3)Netherlands (europe-west4)Zurich (europe-west6)Milan (europe-west8)Paris (europe-west9)Doha (me-central1)Dammam (me-central2)Tel Aviv (me-west1)Montréal (northamerica-northeast1)Toronto (northamerica-northeast2)São Paulo (southamerica-east1)Santiago (southamerica-west1)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)Columbus (us-east5)Dallas (us-south1)Oregon (us-west1)Los Angeles (us-west2)Salt Lake City (us-west3)Las Vegas (us-west4)
Date | Time | Description | |
---|---|---|---|
| 29 Feb 2024 | 11:19 PST | Our Engineering team has mitigated the issue after we rolled back the recent change. For a few customers who are still observing the impact, we are working closely with them on a workaround. If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved. No further updates will be provided here. We thank you for your patience while we're working on resolving the issue. |
| 29 Feb 2024 | 10:02 PST | Summary: Issue with Google Cloud Console, Cloud Workstation, Cloud Shell, GKE and Persistent Disk. Description: Mitigation work is currently underway by our engineering team. Our engineering team has identified a workaround and are in the process of implementing them for Cloud Workstation. We do not have an ETA for full mitigation at this point. We will provide more information by Thursday, 2024-02-29 12:00 US/Pacific. Diagnosis: Cloud Console: Some customers might be unable to use Cloud Shell. Cloud Workstation: The impacted customers are experiencing intermittent delays or failure with starting Workstation instances. Persistent Disk: Customers may experience issues while re-attaching and mounting Regional Persistent Disk to VMs and issuing I/Os. GKE : Customers may experience issues while attaching Regional Persistent Disk backed Persistent Volume to GKE Nodes Workaround: Cloud Shell: Customers can try to use the ephemeral mode. Cloud Workstation: Users may retry the start workstation operation or create and start a new workstation. Persistent Disk: None at this time. GKE: None at this time. |
| 28 Feb 2024 | 20:25 PST | Summary: Issue with Google Cloud Console, Cloud Workstation, Cloud Shell, GKE and Persistent Disk. Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-02-29 10:00 US/Pacific. Diagnosis: Cloud Console: Some customers might be unable to use Cloud Shell. Cloud Workstation: The impacted customers are experiencing intermittent delays or failure with starting Workstation instances. Persistent Disk: Customers may experience issues while attaching Regional Persistent Disk to VMs and issuing I/Os. GKE: Customers may experience issues while attaching Regional Persistent Disk backed Persistent Volume to GKE Nodes Workaround: Cloud Shell: Customers can try to use the ephemeral mode. Cloud Workstation: Users may retry the start workstation operation or create and start a new workstation. Persistent Disk: None at this time. GKE: None at this time. |
| 28 Feb 2024 | 19:40 PST | Summary: Issue with Google Cloud Console, Cloud Workstation, Cloud Shell and Persistent Disk. Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-02-29 10:00 US/Pacific. Diagnosis: Cloud Console: Some customers might be unable to use Cloud Shell. Cloud Workstation: The impacted customers are experiencing intermittent delays or failure with starting Workstation instances. Persistent Disk: Customers may experience issues while attaching Regional Persistent Disk to VMs and issuing I/Os. Workaround: Cloud Shell: Customers can try to use the ephemeral mode. Cloud Workstation: Users may retry the start workstation operation or create and start a new workstation. Persistent Disk: None at this time. |
| 28 Feb 2024 | 17:38 PST | Summary: Cloud Workstations is experiencing intermittent issues with starting Workstation instances. Description: Our engineering team continues to work on the mitigation steps. We do not have an ETA for mitigation completion at this point. We will provide more information by Wednesday, 2024-02-28 20:00 US/Pacific. We deeply appreciate your patience and understanding while we work towards resolving the issue. Diagnosis: The impacted customers are experiencing intermittent delays or failure with starting Workstation instances. Workaround: Users may retry the start workstation operation. |
| 28 Feb 2024 | 12:52 PST | Summary: Cloud Workstations is experiencing intermittent issues with starting Workstation instances. Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Wednesday, 2024-02-28 18:00 US/Pacific. Diagnosis: Affected customers are experiencing intermittent delays or failure with starting Workstation instances. Workaround: Users may retry the start workstation operation. |
| 28 Feb 2024 | 06:50 PST | Summary: Cloud Workstations is experiencing intermittent issues with starting Workstation instances Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Wednesday, 2024-02-28 13:00 US/Pacific. Diagnosis: Affected customers are experiencing intermittent delays or failure with starting Workstation instances. Workaround: Users may retry the start workstation operation. |
| 28 Feb 2024 | 06:31 PST | Summary: Cloud Workstations is experiencing intermittent issues with starting Workstation instances Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Wednesday, 2024-02-28 11:00 US/Pacific. Diagnosis: Affected customers are experiencing intermittent delays or failure with starting Workstation instances. Workaround: Users may retry the start workstation operation. |
- All times are US/Pacific