Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python DSL
Search
Elasticsearch Inc
March 11, 2015
Technology
2
860
Python DSL
Slides for Honza's talk at Elastic{on}
Elasticsearch Inc
March 11, 2015
Tweet
Share
More Decks by Elasticsearch Inc
See All by Elasticsearch Inc
OSCON: Scaling a distributed engineering team from 50-250
elasticsearch
13
1.3k
Stuff a Search Engine Can Do
elasticsearch
17
1.6k
Using Elastic to monitor anything
elasticsearch
3
1.5k
Log all the things!
elasticsearch
4
1.1k
Why Elastic? @ 50th Vinitaly 2016
elasticsearch
5
1.9k
What's New In Elasticland?
elasticsearch
3
810
Kibana, Timelion, Graph Meetup
elasticsearch
3
760
Elastic for Time Series Data and Predictive Analytics
elasticsearch
4
3k
Elastic 2.0
elasticsearch
1
730
Other Decks in Technology
See All in Technology
生産性向上チームの紹介
cybozuinsideout
PRO
1
920
チームでロジカルシンキングに改めて向き合っている話 〜学習環境と実践⽅法〜
sansantech
PRO
3
3.3k
生成AIの変革の時代に、直近1年で直面した課題とその解決策
ktc_wada
0
690
ルーターでプレゼンする
puhitaku
1
3.3k
社内アプリで Cloudflare D1を プロダクト運用してみた体験談(Tokyo)
haochenx
0
120
[新卒向け研修資料] テスト文字列に「うんこ」と入れるな(2024年版)
infiniteloop_inc
5
18k
Cypress or Playwright?
rainerhahnekamp
0
170
Gradle Build Scanを使ってビルドのことを知ろう potatotips #87
tomorrowkey
2
160
リテール金融(キャッシュレス・ネット銀行・ネット証券)の競争環境と経済圏
8maki
0
1.6k
認知症フレンドリーテックとスタックチャン
naokiuc
0
270
ゼロから始めるVue.jsコミュニティ貢献 / first-vuejs-community-contribution-link-and-motivation
lmi
1
150
中年男性がメインフレームから クラウドへキャリアシフトしてみた
uechishingo
0
120
Featured
See All Featured
From Idea to $5000 a Month in 5 Months
shpigford
378
45k
Ruby is Unlike a Banana
tanoku
96
10k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
242
1.2M
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
188
16k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
6.9k
Faster Mobile Websites
deanohume
300
30k
RailsConf 2023
tenderlove
8
550
Building Applications with DynamoDB
mza
88
5.6k
Bootstrapping a Software Product
garrettdimon
PRO
302
110k
Art, The Web, and Tiny UX
lynnandtonic
290
19k
Docker and Python
trallard
35
2.7k
Fireside Chat
paigeccino
22
2.6k
Transcript
Python DSL Honza Král @honzakral
{ } DSL 2
{ } DSL ? Don't you mean ORM? 3
{ } Current State { "query": { "filtered": { "query":
{ "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL
{ } Now add a filter to it! 5
{ } Search Object s = Search(doc_type='question') 6
{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php')
7
{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python'))
8
{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match":
{"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9
{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)])
Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10
{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)})
11
{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12
{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'})
13
{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency',
'date_histogram', field='creation_date', interval='month') 14
{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15
{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term',
tag='published') my_glorious_query = s.to_dict() 16 query at a time
{ } Response response = s.execute() for hit in response:
print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!
{ } Persistence From Mapping to Model-like DocTypes 18
{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title',
String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19
{ } DocType class Article(DocType): title = String() created_date =
Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog' def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20
{ } Configuration 21
{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts':
['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22
{ } Future 23
{ } More DSLs! Index Analyzers Settings ... 24
{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment]
fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???
{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit
in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '<b>%s: %d</b>' print(mask % (name, count)) 26 Usage ????
{ } Django integration ? Model -> DocType signal handlers
to update management command to sync FacetedSearch -> Form view + template pattern 27
{ } Thank you! @honzakral
{ } This work is licensed under the Creative Commons
Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29