Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python DSL
Search
Elasticsearch Inc
March 11, 2015
Technology
2
890
Python DSL
Slides for Honza's talk at Elastic{on}
Elasticsearch Inc
March 11, 2015
Tweet
Share
More Decks by Elasticsearch Inc
See All by Elasticsearch Inc
OSCON: Scaling a distributed engineering team from 50-250
elasticsearch
13
1.4k
Stuff a Search Engine Can Do
elasticsearch
17
1.6k
Using Elastic to monitor anything
elasticsearch
3
1.5k
Log all the things!
elasticsearch
4
1.1k
Why Elastic? @ 50th Vinitaly 2016
elasticsearch
5
1.9k
What's New In Elasticland?
elasticsearch
3
850
Kibana, Timelion, Graph Meetup
elasticsearch
3
760
Elastic for Time Series Data and Predictive Analytics
elasticsearch
4
3k
Elastic 2.0
elasticsearch
1
730
Other Decks in Technology
See All in Technology
20240725 LLMによるDXのビジョンと、今何からやるべきか @Azure OpenAI Service Dev Day
nrryuya
3
1.2k
AIアシスタントの活用で品質の向上と開発ワークフローのスピードアップ
nagix
1
210
Amazon FSx for NetApp ONTAPのパフォーマンスチューニング要素をまとめてみた #cm_odyssey #devio2024
non97
0
220
ゆめみのアクセシビリティの現在地と今後
ryokatsuse
3
290
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
0
710
プレイドにおけるDatadog APMの活用方法
plaidtech
PRO
2
120
Classmethod Odyssey 登壇資料
yamahiro
0
390
運用改善、不都合な真実 / 20240722-ssmjp-kaizen
opelab
17
8.2k
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
160
DDDにおける認可の扱いとKotlinにおける実装パターン / authorization-for-ddd-and-kotlin-implement-pattern
urmot
4
390
開発生産性をむしろ向上させる セキュリティパートナーの作り方 / Dev Productivity Con 2024
flatt_security
0
380
エンジニアリングマネージャーはどう学んでいくのか #devsumi / How Do Engineering Managers Continue to Learn and Grow?
expajp
4
1.3k
Featured
See All Featured
Gamification - CAS2011
davidbonilla
78
4.9k
The Straight Up "How To Draw Better" Workshop
denniskardys
229
130k
Building an army of robots
kneath
301
42k
Fontdeck: Realign not Redesign
paulrobertlloyd
79
5.1k
Web Components: a chance to create the future
zenorocha
307
41k
Web development in the modern age
philhawksworth
203
10k
YesSQL, Process and Tooling at Scale
rocio
166
14k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
36
9.1k
Designing for humans not robots
tammielis
247
25k
Building Your Own Lightsaber
phodgson
101
5.9k
The Power of CSS Pseudo Elements
geoffreycrofte
64
5.2k
Build your cross-platform service in a week with App Engine
jlugia
227
17k
Transcript
Python DSL Honza Král @honzakral
{ } DSL 2
{ } DSL ? Don't you mean ORM? 3
{ } Current State { "query": { "filtered": { "query":
{ "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL
{ } Now add a filter to it! 5
{ } Search Object s = Search(doc_type='question') 6
{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php')
7
{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python'))
8
{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match":
{"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9
{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)])
Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10
{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)})
11
{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12
{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'})
13
{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency',
'date_histogram', field='creation_date', interval='month') 14
{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15
{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term',
tag='published') my_glorious_query = s.to_dict() 16 query at a time
{ } Response response = s.execute() for hit in response:
print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!
{ } Persistence From Mapping to Model-like DocTypes 18
{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title',
String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19
{ } DocType class Article(DocType): title = String() created_date =
Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog' def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20
{ } Configuration 21
{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts':
['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22
{ } Future 23
{ } More DSLs! Index Analyzers Settings ... 24
{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment]
fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???
{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit
in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '<b>%s: %d</b>' print(mask % (name, count)) 26 Usage ????
{ } Django integration ? Model -> DocType signal handlers
to update management command to sync FacetedSearch -> Form view + template pattern 27
{ } Thank you! @honzakral
{ } This work is licensed under the Creative Commons
Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29