Service Health
Incident affecting AlloyDB for PostgreSQL
We are investigating an Issue with AlloyDB for PostgreSQL
Incident began at 2023-03-30 04:04 and ended at 2023-03-30 20:28 (all times are US/Pacific).
Previously affected location(s)
Taiwan (asia-east1)Osaka (asia-northeast2)Seoul (asia-northeast3)Singapore (asia-southeast1)Sydney (australia-southeast1)Warsaw (europe-central2)Finland (europe-north1)Belgium (europe-west1)London (europe-west2)Frankfurt (europe-west3)Zurich (europe-west6)Iowa (us-central1)South Carolina (us-east1)Northern Virginia (us-east4)
Date | Time | Description | |
---|---|---|---|
| 30 Mar 2023 | 20:28 PDT | The issue with AlloyDB for PostgreSQL has been resolved for all affected users as of Thursday, 2023-03-30 18:40 US/Pacific. We thank you for your patience while we worked on resolving the issue. |
| 30 Mar 2023 | 18:12 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Engineers have mitigated the issue for new clusters created from restores as of Thursday, 2023-03-30 07:10 US/Pacific. The issue should be recovered for most customers, however Engineering is taking additional steps to ensure a full recovery for all affected customers, an ETA has not been determined on when this work will be fully completed. We will continue to provide updates on any changes and will provide more information by Thursday, 2023-03-30 21:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: Customers can re-restore from the same backup and potentially delete the old broken one. |
| 30 Mar 2023 | 17:11 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Engineers have mitigated the issue for new clusters created from restores as of Thursday, 2023-03-30 07:10 US/Pacific. The issue should be recovered for most customers, however Engineering is taking additional steps to ensure a full recovery for all affected customers, an ETA has not been determined on when this work will be fully completed. We will continue to provide updates on any changes and will provide more information by Thursday, 2023-03-30 18:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: Customers can re-restore from the same backup and potentially delete the old broken one. |
| 30 Mar 2023 | 15:16 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Engineers have mitigated the issue for new clusters created from restores as of Thursday, 2023-03-30 07:10 US/Pacific. The issue should be recovered for most customers, however Engineering is taking additional steps to ensure a full recovery for all affected customers, an ETA has not been determined on when this work will be fully completed. We will continue to provide updates on any changes and will provide more information by Thursday, 2023-03-30 18:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: None at this time. |
| 30 Mar 2023 | 13:14 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Engineers have mitigated the issue for new clusters created from restores as of Thursday, 2023-03-30 07:10 US/Pacific. Mitigation work is still underway for restored clusters that were previously affected. We do not have an ETA for mitigation of previously affected clusters. We will provide more information by Thursday, 2023-03-30 15:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: None at this time. |
| 30 Mar 2023 | 11:17 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Engineers have mitigated the issue for new clusters created from restores. Mitigation work is still underway for restored clusters that were previously affected. We do not have an ETA for mitigation of previously affected clusters. We will provide more information by Thursday, 2023-03-30 13:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: None at this time. |
| 30 Mar 2023 | 10:24 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Mitigation work is currently underway by our engineering team and is taking longer than initially expected. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2023-03-30 12:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: None at this time. |
| 30 Mar 2023 | 05:30 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2023-03-30 09:30 US/Pacific. We will provide more information by Thursday, 2023-03-30 10:30 US/Pacific. Diagnosis: Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Customers will see an internal error (error code: 13) when they try to create an instance. Workaround: None at this time. |
| 30 Mar 2023 | 05:28 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2023-03-30 09:00 US/Pacific. We will provide more information by Thursday, 2023-03-30 09:30 US/Pacific. Diagnosis: Customers will see an internal error (error code: 13) when they try to create an instance. Customers will experience this issue when they try to create a primary instance in a cluster that was restored. Workaround: None at this time. |
| 30 Mar 2023 | 04:48 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: Mitigation work is currently underway by our engineering team. The mitigation is expected to complete by Thursday, 2023-03-30 09:00 US/Pacific. We will provide more information by Thursday, 2023-03-30 09:30 US/Pacific. Diagnosis: None at this time. Workaround: None at this time. |
| 30 Mar 2023 | 04:24 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: We are experiencing an issue with AlloyDB for PostgreSQL. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-03-30 05:00 US/Pacific with current details. Diagnosis: None at this time. Workaround: None at this time. |
| 30 Mar 2023 | 04:13 PDT | Summary: We are investigating an Issue with AlloyDB for PostgreSQL Description: We are experiencing an issue with AlloyDB for PostgreSQL. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-03-30 04:30 US/Pacific with current details. Diagnosis: None at this time. Workaround: None at this time. |
- All times are US/Pacific