Profiling queries for fun and profit

Slide 1

Slide 1 text

‹#› @ZacharyTong Profiling queries for fun and profit

Slide 2

Slide 2 text

2 Elasticsearch 2.2 received a profiler! • Attempts to time execution of query components • Best-effort profiling • Expensive! Verbose! “SQL Explain for ES” https://www.elastic.co/guide/en/elasticsearch/reference/current/search-profile.html

Slide 3

Slide 3 text

3 Lucene search at 20,000 feet Documents Shard 1

Slide 4

Slide 4 text

4 Search request arrives Search Request Shard 1 Documents

Slide 5

Slide 5 text

5 Shard builds a Collector Shard 1 Search Request Collector Documents

Slide 6

Slide 6 text

6 Collector builds Scorer + Weight Shard 1 Search Request Collector S W Documents

Slide 7

Slide 7 text

7 Collector walks over documents Shard 1 Search Request Collector S W Documents

Slide 8

Slide 8 text

8 Shard 1 Search Request Collector S W Documents Collector walks over documents

Slide 9

Slide 9 text

9 Shard 1 Search Request Collector S W Documents Collector walks over documents

Slide 10

Slide 10 text

10 Shard 1 Search Request Collector S W …. Documents Collector walks over documents

Slide 11

Slide 11 text

11 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 12

Slide 12 text

12 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 13

Slide 13 text

13 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 14

Slide 14 text

14 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W

Slide 15

Slide 15 text

15 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓

Slide 16

Slide 16 text

16 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓

Slide 17

Slide 17 text

17 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓

Slide 18

Slide 18 text

18 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓ ✓

Slide 19

Slide 19 text

19 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓ ✓

Slide 20

Slide 20 text

20 A query often has multiple scorers Shard 1 Search Request Collector S W Documents S W ✓ ✓ ✓ Match!

Slide 21

Slide 21 text

21 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 22

Slide 22 text

22 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 23

Slide 23 text

23 Shard 1 Search Request Collector 2 S W Documents A request can use multiple collectors

Slide 24

Slide 24 text

24 And a request can hit multiple shards Search Request Shard 1 Documents Shard 2 Documents

Slide 25

Slide 25 text

25 Which means the output is verbose

Slide 26

Slide 26 text

26 But maybe some day…

Slide 27

Slide 27 text

Demo Time! 27

Slide 28

Slide 28 text

‹#› Please attribute Elastic with a link to elastic.co Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 28