Service Health
Incident affecting Dataproc Metastore, Google Cloud Dataproc
New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures
Incident began at 2023-04-20 17:06 and ended at 2023-04-23 21:18 (all times are US/Pacific).
Previously affected location(s)
Frankfurt (europe-west3)Northern Virginia (us-east4)
Date | Time | Description | |
---|---|---|---|
| 23 Apr 2023 | 21:18 PDT | The issue with Dataproc Metastore has been resolved for all affected users as of Sunday, 2023-04-23 16:13 US/Pacific. We thank you for your patience while we worked on resolving the issue. |
| 21 Apr 2023 | 12:10 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team has validated the fix and have a mitigation rollout in progress. The mitigation is expected to complete by Monday, 2023-04-24 16:00 US/Pacific. We will provide more information by Monday, 2023-04-24 17:00 US/Pacific. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time. |
| 21 Apr 2023 | 08:52 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team has determined that further investigation is required to mitigate the issue. We will provide an update by Saturday, 2023-04-22 12:00 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Some customers may experience issues with GKE clusters autoscaling the nodepool on N1 standard machine types. Workaround: None at this time. |
| 20 Apr 2023 | 23:40 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 09:00 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time. |
| 20 Apr 2023 | 21:34 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 00:30 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time. |
| 20 Apr 2023 | 21:32 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 00:30 US/Pacific with current details. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time. |
| 20 Apr 2023 | 20:08 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Thursday, 2023-04-20 22:00 US/Pacific with current details. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time. |
| 20 Apr 2023 | 18:17 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is currently testing the mitigation steps to reduce the customer impact. We will provide an update by Thursday, 2023-04-20 20:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Workaround: None at this time. |
| 20 Apr 2023 | 17:46 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-04-20 18:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Workaround: None at this time. |
| 20 Apr 2023 | 17:29 PDT | Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: We are experiencing an issue with Dataproc Metastore. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-04-20 18:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing time outs during creation. Workaround: None at this time. |
- All times are US/Pacific