Elasticsearch (R)Evolution — You Know, for Search...

Slide 1

Slide 1 text

(R)Evolution Philipp Krenn̴̴̴̴̴̴̴̴@xeraa

Slide 2

Slide 2 text

Developer

Slide 3

Slide 3 text

Revolution

Slide 4

Slide 4 text

https://db-engines.com/en/ranking

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Who uses Elasticsearch?

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Terms Cluster, Node, Index, Shard, Document, ID

Slide 9

Slide 9 text

$ curl http://localhost:9200 { "name": "elasticsearch1", "cluster_name": "docker-cluster", "cluster_uuid": "NflLphonSp2QPKx2llxkgQ", "version": { "number": "5.6.14", "build_hash": "f310fe9", "build_date": "2018-12-05T21:20:16.416Z", "build_snapshot": false, "lucene_version": "6.6.1" }, "tagline": "You Know, for Search" }

Slide 10

Slide 10 text

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Evolution

Slide 19

Slide 19 text

Rolling Upgrades* 6.0 * Demo

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

PS: Kibana 6.6 elasticsearch.url elasticsearch.hosts

Slide 22

Slide 22 text

Floodstage Watermark* 6.0 * Demo

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Low 85% High 90% Floodstage 95%

Slide 25

Slide 25 text

Sequence Numbers* 6.0 * Demo

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

63 bits ought to be enough for anyone.

Slide 29

Slide 29 text

Tradeoff index.translog.retention.size: 512MB index.translog.retention.age: 12h

Slide 30

Slide 30 text

Cross Datacenter Replication 6.5

Slide 31

Slide 31 text

Types* 5.6 to 8.0 * Demo

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Why Data types Sparsity Scoring

Slide 34

Slide 34 text

How 5.6 opt-in for single type 6.x single type for new indices 7.x types optional ﬁxed API PUT {index}/_doc/{id} include_type_name: true for GET|PUT _mapping 8.x no more types https://www.elastic.co/guide/en/elasticsearch/reference/current/ removal-of-types.html#_schedule_for_removal_of_mapping_types

Slide 35

Slide 35 text

Shrink & Split* ̴5.0̴̴̴6.1 * Demo

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Shrink Combine shards by a factor

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Split Split into a factor of number_of_routing_shards Not required in 7.0+

Slide 40

Slide 40 text

Shards 7.0

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

Default: 1 Shard per Index Oversharding

Slide 43

Slide 43 text

Simon Says Use a single shard until it blows up

Slide 44

Slide 44 text

Conclusion

Slide 45

Slide 45 text

Don't be too Shay to improve

Slide 46

Slide 46 text

Rolling Upgrades Floodstage Watermark Sequence Numbers

Slide 47

Slide 47 text

Types Shrink & Split Default Shard Number

Slide 48

Slide 48 text

Questions? Philipp Krenn̴̴̴̴̴̴̴@xeraa