Incident affecting Google App Engine, Google Cloud Functions
us-central1: Google App Engine Standard experiencing increased latency and pending queue aborted error request rate.
Incident began at 2021-09-07 11:19 and ended at 2021-09-08 11:00 (all times are US/Pacific).
| ||9 Sep 2021||14:09 PDT|| |
We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support
(All Times US/Pacific)
Incident Start: 07 September 2021 11:19
Incident End: 08 September 2021 11:00
Duration: 23 hours, 41 minutes
Affected Services and Features:
Google App Engine and Google Cloud Functions experienced elevated latency on requests in us-central1 on a small number of projects for a duration of 23 hours, 41 minutes. From preliminary analysis, the root cause of the issue is due to a sudden increase in load within the region.
| ||8 Sep 2021||12:25 PDT|| |
The issue with Google App Engine and Cloud Functions has been resolved for all affected projects as of Wednesday, 2021-09-08 12:24 US/Pacific.
There is no further impact to our customers.
We thank you for your patience while we worked on resolving the issue.
| ||8 Sep 2021||11:40 PDT|| |
Summary: us-central1: Google App Engine Standard and Cloud Functions experiencing increased latency and pending queue aborted error request rate.
Description: Further investigation into the increased latency issue has confirmed that we are also observing an increase in pushbacks to requests (increase in pending queue aborted error request rate) for some customers.
Our engineering team continues to work on the mitigation to reduce the pushback errors and reduce the latency observed.
We will provide more information by Wednesday, 2021-09-08 12:30 US/Pacific.
Diagnosis: Customer may observe increased latency and heightened pending queue aborted error request rate.
Workaround: None at this time.