Apache Drillで 身の回りのログを集計してみる

Tokyo Apache Drill Meetup (2015/09/15)の資料です。

Transcript

1. "QBDIF
   %SJMMͰ
   ਎ͷճΓͷϩάΛूܭͯ͠ΈΔ
   -FU`T
   USZ
   UP
   BHHSFHBUF
   JOEJWJEVBM
   MPHT
   VTJOH
   "QBDIF
   %SJMM 5PLZP
   "QBDIF
   %SJMM
   .FFUVQ
   
   !BDJEMFNPO
2. "CPVU
   .F w 1SPHSBNNJOH
   (PMBOH
   1FSM
   $
   w %#%8)
   .Z42-3FETIJGU
   w "QBDIF
   %SJMM
   IJTUPSZ
   
   XFFLT !BDJEMFNPO
   .BTBUPTIJ
   ,BXB[PF 4PGUXBSF
   &OHJOFFS
   
   BU
   ,":"$
   *OD
3. "QBDIF
   %SJMM w 4DIFNBGSFF
   42-
   &OHJOF
   NPEF
   w %JTUSJCVUJPO
   .PEF
 XJUI
   )BEPPQ
   /P42-
   $MPVE4UPSBHF
   w &NCFEEFE
   .PEF
 XJUI
   -PDBM
   pMF
   +40/
   $47
   547
   147 5SZ
   UIJT
   UPEBZ
4. 8IZ
   %SJMM
   .Z
   3FBTPO w 8IFO
   *
   OFFE
   UP
   JNQSPWF
   XFCBQQ
   QFSGPSNBODFʜ
   w *
   XBOU
   UP
   BHHSFHBUF
   SFTQPOTF
   UJNF
   QFS
   FOEQPJOUʜ
   w %SJMM
   DPVME
   SVO
   RVFSZ
   UP
   +40/
   ZFBI
5. 2VJDL
   4UBSU w 4FF
   0GpDJBM
   5VUPSJBM
   %PDVNFOU
 l%SJMM
   JO
   
   NJOVUFTz
 IUUQTESJMMBQBDIFPSHEPDTESJMMJONJOVUFT
   w PS
   %FWFMPQFST*0
   #MPH
   &OUSZ
   BCPVU
   "QBDIF
   %SJMM
   
   *NQSFTTJPO
 IUUQEFWDMBTTNFUIPEKQDMPVEBXTBQBDIFESJMMpSTUJNQSFTTJPO
   w 'PS
   .BD
   6TFS
   VTF
   )PNFCSFX
 
 SFRVJSFT
   +%,
   
   PS
   MBUFS $ brew install apache-drill
6. $POWFSU
   OHJOY
   MPH w 8F
   VTF
   l-547z
   GPSNBU
   GPS
   OHJOY
   MPH
   w -547
   
   -BCFMFE
   5BCTFQBSBUFE
   7BMVFT
 IUUQMUTWPSH
   
   w "QBDIF
   %SJMM
   EPFT
   OPU
   TVQQPSU
   -547
   ZFU
   TP
   DPOWFSU
   MPH
   GSPN
   -547
   UP
   +40/
   w
   VTF
   MUTWJFX
   DPNNBOE
   WJB
   SVCZHFNT host:217.235.96.155 time:2015-09-14T00:00:00+09:00 method:POST uri:/api/hunting/challenge protocol:HTTP/1.1 status:200 size: 5790 request_time:0.383 upstream_response_time:0.438 referer:- user_agent:MyApp/11000 CFNetwork/711.3.18 Darwin/14.0.0 user_id: 76005617 geo:JP
7. %FNP .
   MJOFT
   .#
   +40/
8. $VU
   2VFSZ4USJOH w (SPVQ
   CZ
   VSJ
   
 CVU
   VSJ
   DPOUBJOT
   2VFSZ4USJOHʜ
   w 4PMVUJPO
   
 3&(&91@3&1-"$& VSJ `a   
   BT
   QBUI +------------------------------------------------------------------------------------------------------------+----------+ | uri | maxtime | +------------------------------------------------------------------------------------------------------------+----------+ | /api/costume/hunting_item/equip | 9.695 | | /api/hunting/families?hunting_id=4000222&session=XXXXXXXXe937077b8cbc06ad3aec80fe&client_data_version=1329 | 9.273 | | /api/mypage?os_type=1&platform_type=1&session=XXXXXXXX7968eb69dd20d2bb1aec3943&client_data_version=1329 | 8.316 | | /api/mypage?os_type=2&platform_type=2&session=XXXXXXXXb63727f12bd2724ab9725885&client_data_version=1329 | 8.279 | | /api/hunting/families?hunting_id=4026012&session=XXXXXXXX2dc4a6668288c5f4a675f071&client_data_version=1330 | 7.926 | | /api/hunting/families?hunting_id=4025518&session=XXXXXXXX04634d3293a8c2317454b6a3&client_data_version=1329 | 7.896 |
9. 1BSTF
   UJNF
   TUSJOH w UJNF5 
   w KVTU
   B
   TUSJOH
   OPU
   UJNFTUBNQʜ

   
   where `time` > ‘2015-09-14 07:00:00’
 EPFT
   OPU
   XPSL
   BT
   FYQFDUFEʜ w 4PMVUJPO
 50@5*.&45".1 AUJNFA `ZZZZ..EE5))NNTT; 
 "4
   BDDFTT@UJNF
10. %FNP
     (3061
    #:
    QFSDFOUJMF
    BOBMZTJT
11. $PODMVTJPO w "QBDIF
    %SJMM
    JT
    SFBTPOBCMZ
    GBTU
    w 5FNQPSBSZ
    UBCMF
    JT
    VTFGVM
    GPS
    MPOH
    RVFSZ
    w $"45
    JT
    JNQPSUBOU
    UP
    SVO
    RVFSZ
    
    w 4UJMM
    MJUUMF
    IBSE
    UP
    VTF
    ZFU
    SFBE
    EPDVNFOU
    DBSFGVMMZ
12. 5IBOL
    ZPV