Google Cloud Status Dashboard

This page provides status information on the services that are part of Google Cloud Platform. Check back here to view the current status of the services listed below. If you are experiencing an issue not listed here, please contact Support. Learn more about what's posted on the dashboard in this FAQ. For additional information on these services, please visit cloud.google.com.

Google App Engine Incident #15015

Stale composite indexes on Google App Engine

Incident began at 2015-06-02 10:15 and ended at 2015-06-02 19:52 (all times are US/Pacific).

Date Time Description
Jun 02, 2015 20:05

The remediation of missing index entries has been completed. We apologize for the inconvenience and thank you for your patience and continued support.

The remediation of missing index entries has been completed. We apologize for the inconvenience and thank you for your patience and continued support.

Jun 02, 2015 18:05

Some of the missing index entries have been fixed and the remaining ones will require additional time. We will provide another status update by 20:00 US/Pacific with current details.

Some of the missing index entries have been fixed and the remaining ones will require additional time. We will provide another status update by 20:00 US/Pacific with current details.

Jun 02, 2015 15:58

Google Engineers have identified the issue with Google App Engine Datastore composite indexes missing entries. Remediation has been applied so that all future writes will correctly update indexes. Over the next several hours all remaining missing index entries will be fixed. We will provide another status update by 18:00 US/Pacific with current details.

Google Engineers have identified the issue with Google App Engine Datastore composite indexes missing entries. Remediation has been applied so that all future writes will correctly update indexes. Over the next several hours all remaining missing index entries will be fixed. We will provide another status update by 18:00 US/Pacific with current details.

Jun 02, 2015 15:02

The investigation into Google App Engine stale composite indexes is ongoing. Current data indicates the issue is isolated to approximately 0.01% of applications running in the US. We will provide another status update by 16:00 US/Pacific with current details.

The investigation into Google App Engine stale composite indexes is ongoing. Current data indicates the issue is isolated to approximately 0.01% of applications running in the US. We will provide another status update by 16:00 US/Pacific with current details.

Jun 02, 2015 14:03

We are still investigating the issue with a small number of Google App Engine apps experiencing stale composite indexes for recently updated Datastore entities. Current data indicates the issue only affects apps located in the US. Approximately 0.01% of apps are affected by this issue. We will provide another status update by 15:00 US/Pacific with current details.

We are still investigating the issue with a small number of Google App Engine apps experiencing stale composite indexes for recently updated Datastore entities. Current data indicates the issue only affects apps located in the US. Approximately 0.01% of apps are affected by this issue. We will provide another status update by 15:00 US/Pacific with current details.

Jun 02, 2015 12:52

Approximately 0.01% of Google App Engine apps are experiencing stale composite indexes for recently updated Datastore entities. Partial remediation has been applied but some Datastore queries may still get old results. Google Engineers are working to complete the remediation.

To test if you are affected, use the App Engine Datastore Viewer to query for a suspected stale entity both by it's key and it's indexed value. If you cannot find the entity by index but can see the entity by key you may be affected. Example queries to do this are:

``` SELECT key FROM YourKind WHERE key__=Key('YourKind', 'id_string') ``` and ``` SELECT __key FROM YourKind WHERE indexed_col1='value1' and indexed_col_2='value2' ```

For those affected, we apologize for any inconvenience you may be experiencing. We will provide an update by 14:00 US/Pacific time with current details.

Approximately 0.01% of Google App Engine apps are experiencing stale composite indexes for recently updated Datastore entities. Partial remediation has been applied but some Datastore queries may still get old results. Google Engineers are working to complete the remediation.

To test if you are affected, use the App Engine Datastore Viewer to query for a suspected stale entity both by it's key and it's indexed value. If you cannot find the entity by index but can see the entity by key you may be affected. Example queries to do this are:

``` SELECT key FROM YourKind WHERE key__=Key('YourKind', 'id_string') ``` and ``` SELECT __key FROM YourKind WHERE indexed_col1='value1' and indexed_col_2='value2' ```

For those affected, we apologize for any inconvenience you may be experiencing. We will provide an update by 14:00 US/Pacific time with current details.