Kan du få data tilbake igjen fra dine Elasticsearch snapshots?

Elasticsearch offers the snapshot/restore API for handling backups. How do you actually get data back from a snapshot? Have you ever tried. This lightning talk demonstrates usages of the restore API and strategies for restoring data back into Elasticsearch with and without downtime.

Transcript

1. KAN DU FÅ DATA TILBAKE IGJEN FRA DINE ELASTICSEARCH SNAPSHOTS? JavaZone 2016 Jan Fredrik Wedén
2. WHAT IS SNAPSHOT/ RESTORE?
3. PUT /_snapshot/my_repo { "type": "fs | url | s3 | azure | hdfs", settings… } PUT /_snapshot/my_repo/snap_1
4. PUT /_snapshot/my_repo { "type": "fs | url | s3 | azure | hdfs", settings… } PUT /_snapshot/my_repo/snap_1 GET /_snapshot/my_repo/snap_1 DELETE /_snapshot/my_repo/snap_1
5. Trusted dev Elasticsearch UTV Elasticsearch PROD
6. Evil hacker Trusted dev Internal forgiving gateway Elasticsearch UTV Elasticsearch PROD
13. FIND THE SNAPSHOT TO RESTORE
14. GET _snapshot/my_repo/_all GET _snapshot/_all
15. GET _snapshot/my_repo/_all GET _snapshot/my_repo/snap_1 GET _snapshot/my_repo/snap_1/_status GET _snapshot/_all
16. RESTORE EVERYTHING INTO THE FIERY CLUSTER FROM WHENCE IT CAME
17. POST _all/_close
18. POST _all/_close POST /_snapshot/my_repo/snap_1/_restore { "indices": "config", … }
19. RESTORE TO AN ENTIRELY NEW CLUSTER
20. PUT /_snapshot/my_utv_repo { "type": "url", "settings": { "url": "location/of/prod" } }
21. DELETE _all PUT /_snapshot/my_utv_repo { "type": "url", "settings": { "url": "location/of/prod" } } Setting to allow DELETE <* | _all> should always be disabled in PROD, and preferably in all envs.
22. DELETE _all PUT /_snapshot/my_utv_repo { "type": "url", "settings": { "url": "location/of/prod" } } POST /_snapshot/my_utv_repo/snap_1/_restore { "indices": "config,…", … } Setting to allow DELETE <* | _all> should always be disabled in PROD, and preferably in all envs.
23. RESTORE TO THE SAME CLUSTER WITH RENAMING
24. POST /_snapshot/my_repo/snap_1/_restore { "indices": "config", "include_global_state": false, "rename_pattern": "(.+)", "rename_replacement": "restored_$1” }
25. POST /_snapshot/my_repo/snap_1/_restore { "indices": "config", "include_global_state": false, "rename_pattern": "(.+)", "rename_replacement": "restored_$1” } GET /restored_config/global_type/super_important_doc
26. POST /_snapshot/my_repo/snap_1/_restore { "indices": "config", "include_global_state": false, "rename_pattern": "(.+)", "rename_replacement": "restored_$1” } GET /restored_config/global_type/super_important_doc PUT /config/global_type/super_important_doc { the GET data above }
27. POST /_snapshot/my_repo/snap_1/_restore { "indices": "config", "include_global_state": false, "rename_pattern": "(.+)", "rename_replacement": "restored_$1” } GET /restored_config/global_type/super_important_doc PUT /config/global_type/super_important_doc { the GET data above } DELETE /restored_config
29. CARRY ON… Jan Fredrik Wedén @jfweden