hotdog - yet another command line tool for DD

Transcript

1. Copyright ©2015 Treasure Data. All Rights Reserved. hotdog
 yet another command line tool for DD Yamashita, Yuu (@yyuu) 
 Treasure Data, Inc. yuu@treasure-data.com February 24, 2015
2. Copyright ©2015 Treasure Data. All Rights Reserved. about:yyuu • 2014/10 - Treasure Data Inc. • Past: Geisha Tokyo Entertainment Inc. • OSS • pyenv (github: yyuu/pyenv) • xtc (bitbucket: yyuu/xtc) • jetty-nosql-memcached (github: yyuu/jetty-nosql-memcached) • botornado (github: yyuu/botornado) • sbt-extras (github: paulp/sbt-extras) 2
3. Copyright ©2015 Treasure Data. All Rights Reserved. Topics • hotdog - yet another command line tool for DataDog 3
4. Copyright ©2015 Treasure Data. All Rights Reserved. Treasure Data: We need processors • We’re a Big Data company. • Big Data needs processors. • So, we have MANY servers 4
5. Copyright ©2015 Treasure Data. All Rights Reserved. DataDog tags are awesome • DataDog tags help us to find a host • We’re setting up agent tags in Chef’s cookbook • DataDog doesn’t allow us complex tag expressions • Multiple tags will be regarded as AND • We need OR and NOT 5
6. Copyright ©2015 Treasure Data. All Rights Reserved. DataDog search sucks • “/api/v1/search” doesn’t support query language • Unable to search hosts from their tags • Doesn’t support wildcard 6
7. Copyright ©2015 Treasure Data. All Rights Reserved. My solution; hotdog • Supports AND/OR/NOT expression • Supports wildcard characters • Supports cooperation with pssh/GNU Parallel 7
8. Copyright ©2015 Treasure Data. All Rights Reserved. How hotdog works • Parse given expression by PEG parser • Get inverted index from “/api/v1/tags/hosts” • Build host-tag index on on-memory SQLite • Evaluate the expression 8
9. Copyright ©2015 Treasure Data. All Rights Reserved. WE ARE HIRING! 9 Check: www.treasuredata.com