Slide 1

Slide 1 text

Python DSL Honza Král @honzakral

Slide 2

Slide 2 text

{ } DSL 2

Slide 3

Slide 3 text

{ } DSL ? Don't you mean ORM? 3

Slide 4

Slide 4 text

{ } Current State { "query": { "filtered": { "query": { "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL

Slide 5

Slide 5 text

{ } Now add a filter to it! 5

Slide 6

Slide 6 text

{ } Search Object s = Search(doc_type='question') 6

Slide 7

Slide 7 text

{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php') 7

Slide 8

Slide 8 text

{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python')) 8

Slide 9

Slide 9 text

{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match": {"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9

Slide 10

Slide 10 text

{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)]) Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10

Slide 11

Slide 11 text

{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)}) 11

Slide 12

Slide 12 text

{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12

Slide 13

Slide 13 text

{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'}) 13

Slide 14

Slide 14 text

{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency', 'date_histogram', field='creation_date', interval='month') 14

Slide 15

Slide 15 text

{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15

Slide 16

Slide 16 text

{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term', tag='published') my_glorious_query = s.to_dict() 16 query at a time

Slide 17

Slide 17 text

{ } Response response = s.execute() for hit in response: print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!

Slide 18

Slide 18 text

{ } Persistence From Mapping to Model-like DocTypes 18

Slide 19

Slide 19 text

{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title', String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19

Slide 20

Slide 20 text

{ } DocType class Article(DocType): title = String() created_date = Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog'
 def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20

Slide 21

Slide 21 text

{ } Configuration 21

Slide 22

Slide 22 text

{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts': ['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22

Slide 23

Slide 23 text

{ } Future 23

Slide 24

Slide 24 text

{ } More DSLs! Index Analyzers Settings ... 24

Slide 25

Slide 25 text

{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment] fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???

Slide 26

Slide 26 text

{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '%s: %d' print(mask % (name, count)) 26 Usage ????

Slide 27

Slide 27 text

{ } Django integration ? Model -> DocType signal handlers to update management command to sync FacetedSearch -> Form view + template pattern 27

Slide 28

Slide 28 text

{ } Thank you! @honzakral

Slide 29

Slide 29 text

{ } This work is licensed under the Creative Commons Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29