Service Health
-
-
-
-
Incident affecting Google Cloud Functions
We've received a report of an issue with Google Cloud Functions.
Incident began at 2019-07-25 05:13 and ended at 2019-07-25 19:59 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| | 1 Aug 2019 | 11:50 PDT | After further investigation, we have determined that the impact was minimal and affected a subset of users. We have conducted an internal investigation of this issue and made appropriate improvements to our systems to help prevent or minimize a future recurrence. |
| | 25 Jul 2019 | 18:23 PDT | The issue with Google Cloud Functions deployments has been resolved for all affected projects as of Thursday, 2019-07-25 18:22 US/Pacific. We will conduct an internal investigation of this issue and make appropriate improvements to our systems to help prevent or minimize future recurrence. We will provide a more detailed analysis of this incident once we have completed our internal investigation. |
| | 25 Jul 2019 | 16:47 PDT | The issue with Google Cloud Functions deployments should be resolved for the majority of projects. Currently, approximately 5% of the requests are still failing. Our engineers are still working to mitigate the issue and we expect a full resolution in the near future. We will provide another status update by Thursday, 2019-07-25 18:00 US/Pacific with current details. |
| | 25 Jul 2019 | 16:45 PDT | The issue with Google Cloud Functions deployments should be resolved for the majority of projects. Currently, approximately 5% of the requests are still failing. Our engineers are still working to mitigate the issue and we expect a full resolution in the near future. We will provide another status update by Thursday, 2019-07-25 18:00 US/Pacific with current details. |
| | 25 Jul 2019 | 15:38 PDT | The rate of errors is decreasing. We will provide another status update by Thursday, 2019-07-25 16:45 US/Pacific with current details. |
| | 25 Jul 2019 | 14:44 PDT | We are still seeing errors. Our Engineers have implemented a mitigation and are now monitoring the situation before gradually rolling it out to a wider audience. We will provide another status update by Thursday, 2019-07-25 15:45 US/Pacific with current details. |
| | 25 Jul 2019 | 13:46 PDT | We are still seeing errors. Our Engineers are attempting several mitigation strategies in parallel and are beginning to see positive results. We will provide another status update by Thursday, 2019-07-25 14:45 US/Pacific with current details. |
| | 25 Jul 2019 | 12:31 PDT | Mitigation work is currently still underway by our Engineering Team. We will provide another status update by Thursday, 2019-07-25 13:45 US/Pacific with current details. |
| | 25 Jul 2019 | 10:35 PDT | Our engineering team has identified a root cause, and mitigation work is currently underway. Efforts to accelerate recovery using the previous mitigation were not effective and the issue has been escalated internally, we are continuing to work on a new mitigation. We will provide another status update by Thursday, 2019-07-25 12:30 US/Pacific with current details. |
| | 25 Jul 2019 | 09:06 PDT | Our engineering team has identified a root cause, and mitigation work is currently underway. Mitigation appears effective, and we are working to accelerate recovery. We will provide another status update by Thursday, 2019-07-25 10:30 US/Pacific with current details. |
| | 25 Jul 2019 | 08:02 PDT | Our engineering team has identified a root cause, and mitigation work is currently underway. We will provide another status update by Thursday, 2019-07-25 09:00 US/Pacific with current details. |
| | 25 Jul 2019 | 07:52 PDT | We are experiencing elevated deployment errors for projects which are deploying to GCF for the first time. Projects with existing deployments are unaffected unless they are being deployed to a new region. We will provide another status update by Thursday, 2019-07-25 08:30 US/Pacific with current details. |
- All times are US/Pacific