Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud Platform. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit cloud.google.com.

Google App Engine Incident #15007

Intermittent errors for Managed VM (Beta) deployments

Incident began at 2015-03-09 18:30 and ended at 2015-03-09 19:45 (all times are US/Pacific).

Date Time Description
Mar 11, 2015 17:28

SUMMARY:

On Monday 9 March 2015, deployments of Google App Engine applications running on Beta Managed VMs experienced intermittent deployment failures for a duration of 75 minutes. Whilst Managed VMs is still in Beta, we apologize for any impact this incident may have had on your service or application.

DETAILED DESCRIPTION OF IMPACT:

Between 18:20 and 19:40 PDT, users of Google App Engine Managed VMs (Beta) experienced intermittent deployment failures. Users impacted by this incident would see gcloud or appcfg deploy commands return with error messages. Over this period, 81% of deployments returned errors.

ROOT CAUSE:

The Managed VMs (Beta) deployment system relies on the Google Cloud Storage JSON API, which experienced a 5% failure rate during this time period. For details, see the Cloud Storage incident post: https://status.cloud.google.com/incident/storage/16021.

REMEDIATION AND PREVENTION:

The primary remediation for this incident is described in the Cloud Storage incident report. In addition to the steps outlined there, Google engineers are tuning the systems that monitor Managed VMs application deployments to alert earlier, which will facilitate speedier resolution.

SUMMARY:

On Monday 9 March 2015, deployments of Google App Engine applications running on Beta Managed VMs experienced intermittent deployment failures for a duration of 75 minutes. Whilst Managed VMs is still in Beta, we apologize for any impact this incident may have had on your service or application.

DETAILED DESCRIPTION OF IMPACT:

Between 18:20 and 19:40 PDT, users of Google App Engine Managed VMs (Beta) experienced intermittent deployment failures. Users impacted by this incident would see gcloud or appcfg deploy commands return with error messages. Over this period, 81% of deployments returned errors.

ROOT CAUSE:

The Managed VMs (Beta) deployment system relies on the Google Cloud Storage JSON API, which experienced a 5% failure rate during this time period. For details, see the Cloud Storage incident post: https://status.cloud.google.com/incident/storage/16021.

REMEDIATION AND PREVENTION:

The primary remediation for this incident is described in the Cloud Storage incident report. In addition to the steps outlined there, Google engineers are tuning the systems that monitor Managed VMs application deployments to alert earlier, which will facilitate speedier resolution.

Mar 10, 2015 11:59

The issue with the Cloud Storage JSON API was resolved as of 19:35 on Monday 2015-03-09. We are sorry for any issues this may have caused to you or your users and thank you for your patience and continued support.

We will provide a detailed analysis of this incident under Cloud Storage incident 16021 once we have completed our internal investigation.

The issue with the Cloud Storage JSON API was resolved as of 19:35 on Monday 2015-03-09. We are sorry for any issues this may have caused to you or your users and thank you for your patience and continued support.

We will provide a detailed analysis of this incident under Cloud Storage incident 16021 once we have completed our internal investigation.

Mar 09, 2015 20:52

Managed VM (Beta) deployments have been functional since 19:45. We will continue monitoring the issue until the Cloud Storage incident is resolved.

Managed VM (Beta) deployments have been functional since 19:45. We will continue monitoring the issue until the Cloud Storage incident is resolved.

Mar 09, 2015 20:03

We are currently experiencing an intermittent issue with Google App Engine Managed VM (Beta) deployments starting Monday 2015-03-09 18:30 (all times are in US/Pacific). For everyone who is affected, we apologize for any inconvenience you may be experiencing. We will provide an update by 20:30 with current details.

This issue is related to Cloud Storage incident: https://status.cloud.google.com/incident/storage/16021 .

We are currently experiencing an intermittent issue with Google App Engine Managed VM (Beta) deployments starting Monday 2015-03-09 18:30 (all times are in US/Pacific). For everyone who is affected, we apologize for any inconvenience you may be experiencing. We will provide an update by 20:30 with current details.

This issue is related to Cloud Storage incident: https://status.cloud.google.com/incident/storage/16021 .