Service Health
-
-
-
-
Incident affecting Hybrid Connectivity, Google Cloud Networking
Global: Cloud VPN tunnel creation failures via Terraform
Incident began at 2022-12-08 07:25 and ended at 2022-12-12 03:31 (all times are US/Pacific).
Previously affected location(s)
Taiwan (asia-east1)Hong Kong (asia-east2)Tokyo (asia-northeast1)Osaka (asia-northeast2)Seoul (asia-northeast3)Mumbai (asia-south1)Delhi (asia-south2)Singapore (asia-southeast1)Jakarta (asia-southeast2)Sydney (australia-southeast1)Melbourne (australia-southeast2)Warsaw (europe-central2)Finland (europe-north1)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Netherlands (europe-west4)Zurich (europe-west6)Montréal (northamerica-northeast1)Toronto (northamerica-northeast2)São Paulo (southamerica-east1)Santiago (southamerica-west1)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)Oregon (us-west1)Los Angeles (us-west2)Salt Lake City (us-west3)Las Vegas (us-west4)
| Date | Time | Description | |
|---|---|---|---|
| | 12 Dec 2022 | 03:31 PST | The issue with Hybrid Connectivity has been resolved for all affected users. We thank you for your patience while we worked on resolving the issue. |
| | 9 Dec 2022 | 09:31 PST | Summary: Global: Cloud VPN tunnel creation failures via Terraform Description: Customers attempting to create VPN tunnels using Terraform were experiencing failures and mitigation work to resolve this issue is complete. Google engineers have identified another issue where some customers allow-listed for IoIC private GA may experience errors for ‘compute.interconnectAttachments.get’. Mitigation work is underway with our engineering team. We will provide more information by Monday, 2022-12-12 17:00 US/Pacific. Diagnosis: Customers executing compute.interconnectAttachments.get may see an error. Workaround: None at the moment |
| | 9 Dec 2022 | 04:26 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: We are experiencing an issue with Hybrid Connectivity beginning at Friday, 2022-12-09 00:00 US/Pacific. Mitigation work to resolve issue with Terraform is complete. Google engineers have identified another issue which is being investigated. We will provide more information by Friday, 2022-12-09 09:30 US/Pacific. Diagnosis: Customers executing compute.interconnectAttachments.get may see an error. Workaround: None at the moment |
| | 9 Dec 2022 | 04:08 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is still currently underway by our engineering team. The mitigation is expected to complete by Friday, 2022-12-09 06:00 US/Pacific. We will provide more information by Friday, 2022-12-09 06:00 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 21:47 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Friday, 2022-12-09 04:00 US/Pacific. We will provide more information by Friday, 2022-12-09 04:00 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 16:32 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2022-12-08 23:00 US/Pacific. We will provide more information by Thursday, 2022-12-08 23:00 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 14:40 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2022-12-08 14:30 US/Pacific. We will provide more information by Thursday, 2022-12-08 17:30 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 12:03 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2022-12-08 14:30 US/Pacific. We will provide more information by Thursday, 2022-12-08 15:00 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 08:51 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2022-12-08 14:30 US/Pacific. We will provide more information by Thursday, 2022-12-08 15:00 US/Pacific. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 08:04 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: We are experiencing an issue with Hybrid Connectivity beginning at Thursday, 2022-12-08 00:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2022-12-08 09:15 US/Pacific with current details. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 07:46 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: We are experiencing an issue with Hybrid Connectivity beginning at Thursday, 2022-12-08 00:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2022-12-08 09:15 US/Pacific with current details. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
| | 8 Dec 2022 | 07:44 PST | Summary: Google Engineer are investigating issues with Hybrid connectivity Description: We are experiencing an issue with Hybrid Connectivity beginning at Thursday, 2022-12-08 00:00 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2022-12-08 08:46 US/Pacific with current details. Diagnosis: Customers attempting to create VPN tunnels using Terraform will be experiencing failures. Workaround: Customer can work around Terraform problems by creating VPN tunnels manually (via Cloud Console or gcloud). |
- All times are US/Pacific