Apache Drillで 身の回りのログを集計してみる

Tokyo Apache Drill Meetup (2015/09/15)の資料です。

Transcript

1. "QBDIF%SJMMͰ
   ਎ͷճΓͷϩάΛूܭͯ͠ΈΔ
   -FU`TUSZUPBHHSFHBUF
   JOEJWJEVBMMPHTVTJOH"QBDIF%SJMM
   5PLZP"QBDIF%SJMM.FFUVQ
   !BDJEMFNPO
   

   View Slide

2. "CPVU.F
   w 1SPHSBNNJOH(PMBOH 1FSM $
   w %#%8).Z42-3FETIJGU
   w "QBDIF%SJMMIJTUPSZXFFLT
   !BDJEMFNPO
   .BTBUPTIJ,BXB[PF
   4PGUXBSF&OHJOFFS
   BU,":"$*OD
   

   View Slide

3. "QBDIF%SJMM
   w 4DIFNBGSFF42-&OHJOF NPEF
   
   w %JTUSJCVUJPO.PEF

   XJUI)BEPPQ /P42- $MPVE4UPSBHF
   w &NCFEEFE.PEF

   XJUI-PDBMpMF +40/ $47 547 147
   
   5SZUIJTUPEBZ
   

   View Slide

4. 8IZ%SJMM .Z3FBTPO
   w 8IFO*OFFEUPJNQSPWFXFCBQQ
   QFSGPSNBODFʜ
   w *XBOUUPBHHSFHBUFSFTQPOTFUJNF
   QFSFOEQPJOUʜ
   w %SJMMDPVMESVORVFSZUP+40/ ZFBI
   

   View Slide

5. 2VJDL4UBSU
   w 4FF0GpDJBM5VUPSJBM%PDVNFOU

   l%SJMMJONJOVUFTz

   IUUQTESJMMBQBDIFPSHEPDTESJMMJONJOVUFT
   
   w PS%FWFMPQFST*0#MPH&OUSZBCPVU
   "QBDIF%SJMM*NQSFTTJPO

   IUUQEFWDMBTTNFUIPEKQDMPVEBXTBQBDIFESJMMpSTUJNQSFTTJPO
   w 'PS.BD6TFSVTF)PNFCSFX

   

   SFRVJSFT+%,PSMBUFS
   $ brew install apache-drill
   

   View Slide

6. $POWFSUOHJOYMPH
   w 8FVTFl-547zGPSNBUGPSOHJOYMPH
   w -547-BCFMFE5BCTFQBSBUFE7BMVFT

   IUUQMUTWPSH
   w "QBDIF%SJMMEPFTOPUTVQQPSU-547ZFU TP
   DPOWFSUMPHGSPN-547UP+40/
   w VTFMUTWJFXDPNNBOE WJBSVCZHFNT
   
   host:217.235.96.155 time:2015-09-14T00:00:00+09:00 method:POST
   uri:/api/hunting/challenge protocol:HTTP/1.1 status:200 size:
   5790 request_time:0.383 upstream_response_time:0.438 referer:-
   user_agent:MyApp/11000 CFNetwork/711.3.18 Darwin/14.0.0 user_id:
   76005617 geo:JP
   

   View Slide

7. %FNP
   .MJOFT
   .#+40/
   

   View Slide

8. $VU2VFSZ4USJOH
   w (SPVQCZVSJ

   CVUVSJDPOUBJOT2VFSZ4USJOHʜ
   w 4PMVUJPO

   3&(&91@3&1-"$& VSJ `a
   BTQBUI
   +------------------------------------------------------------------------------------------------------------+----------+
   | uri | maxtime |
   +------------------------------------------------------------------------------------------------------------+----------+
   | /api/costume/hunting_item/equip | 9.695 |
   | /api/hunting/families?hunting_id=4000222&session=XXXXXXXXe937077b8cbc06ad3aec80fe&client_data_version=1329 | 9.273 |
   | /api/mypage?os_type=1&platform_type=1&session=XXXXXXXX7968eb69dd20d2bb1aec3943&client_data_version=1329 | 8.316 |
   | /api/mypage?os_type=2&platform_type=2&session=XXXXXXXXb63727f12bd2724ab9725885&client_data_version=1329 | 8.279 |
   | /api/hunting/families?hunting_id=4026012&session=XXXXXXXX2dc4a6668288c5f4a675f071&client_data_version=1330 | 7.926 |
   | /api/hunting/families?hunting_id=4025518&session=XXXXXXXX04634d3293a8c2317454b6a3&client_data_version=1329 | 7.896 |
   

   View Slide

9. 1BSTFUJNFTUSJOH
   w UJNF5
   w KVTUBTUSJOH OPUUJNFTUBNQʜ

   where `time` > ‘2015-09-14 07:00:00’

   EPFTOPUXPSLBTFYQFDUFEʜ
   w 4PMVUJPO

   50@5*.&45".1 AUJNFA `ZZZZ..EE5))NNTT;
   

   "4BDDFTT@UJNF
   

   View Slide

10. %FNP
    (3061#:
    QFSDFOUJMFBOBMZTJT
    

    View Slide

11. $PODMVTJPO
    w "QBDIF%SJMMJTSFBTPOBCMZGBTU
    w 5FNQPSBSZUBCMFJTVTFGVMGPSMPOH
    RVFSZ
    w $"45JTJNQPSUBOUUPSVORVFSZ
    w 4UJMMMJUUMFIBSEUPVTFZFU SFBE
    EPDVNFOUDBSFGVMMZ
    

    View Slide

12. 5IBOLZPV
    

    View Slide