Slide 1

Slide 1 text

SCALE YOUR CLUSTER PHILIPP KRENN̴̴̴̴@XERAA

Slide 2

Slide 2 text

DEVELOPER

Slide 3

Slide 3 text

AGENDA INDEX LIFECYCLE MANAGEMENT FROZEN INDICES ROLLUPS

Slide 4

Slide 4 text

START DEMO SETUP

Slide 5

Slide 5 text

INDEX LIFECYCLE MANAGEMENT

Slide 6

Slide 6 text

! "

Slide 7

Slide 7 text

FUTURE: SNAPSHOT LIFECYCLE MANAGEMENT FOR NOW ELASTIC CURATOR

Slide 8

Slide 8 text

FEATURES & ORDER https://github.com/elastic/elasticsearch/blob/7.0/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifecycle/TimeseriesLifecycleType.java static final List ORDERED_VALID_HOT_ACTIONS = Arrays.asList( SetPriorityAction.NAME, UnfollowAction.NAME, RolloverAction.NAME ); static final List ORDERED_VALID_WARM_ACTIONS = Arrays.asList( SetPriorityAction.NAME, UnfollowAction.NAME, ReadOnlyAction.NAME, AllocateAction.NAME, ShrinkAction.NAME, ForceMergeAction.NAME ); static final List ORDERED_VALID_COLD_ACTIONS = Arrays.asList( SetPriorityAction.NAME, UnfollowAction.NAME, AllocateAction.NAME, FreezeAction.NAME ); static final List ORDERED_VALID_DELETE_ACTIONS = Arrays.asList( DeleteAction.NAME );

Slide 9

Slide 9 text

DEMO

Slide 10

Slide 10 text

PS: 7.0 IMPROVEMENT index.search.idle.after: 30s IFF DEFAULT index.refresh_interval

Slide 11

Slide 11 text

FROZEN INDICES

Slide 12

Slide 12 text

RATIO HEAP : STORAGE INDEX > FROZEN INDEX > CLOSED INDEX

Slide 13

Slide 13 text

FROZEN INDEX READ-ONLY NO MEMORY

Slide 14

Slide 14 text

THROTTLED THREAD POOL 1 PARALLEL SEARCH / NODE 100 IN QUEUE

Slide 15

Slide 15 text

DEMO

Slide 16

Slide 16 text

ROLLUPS

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

DEMO

Slide 19

Slide 19 text

CONCLUSION

Slide 20

Slide 20 text

CODE https://github.com/xeraa/scale-elasticsearch

Slide 21

Slide 21 text

RECAP INDEX LIFECYCLE MANAGEMENT FROZEN INDICES ROLLUPS

Slide 22

Slide 22 text

Questions? PHILIPP KRENN̴̴̴̴@XERAA