Apache Drillで 身の回りのログを集計してみる

9480e27b8d6ae2a668a98144516b5513?s=47 acidlemon
September 15, 2015

Apache Drillで 身の回りのログを集計してみる

Tokyo Apache Drill Meetup (2015/09/15)の資料です。

9480e27b8d6ae2a668a98144516b5513?s=128

acidlemon

September 15, 2015
Tweet

Transcript

  1. "QBDIF%SJMMͰ ਎ͷճΓͷϩάΛूܭͯ͠ΈΔ -FU`TUSZUPBHHSFHBUF JOEJWJEVBMMPHTVTJOH"QBDIF%SJMM 5PLZP"QBDIF%SJMM.FFUVQ !BDJEMFNPO

  2. "CPVU.F w 1SPHSBNNJOH(PMBOH 1FSM $  w %#%8).Z42-3FETIJGU w "QBDIF%SJMMIJTUPSZXFFLT

    !BDJEMFNPO .BTBUPTIJ,BXB[PF 4PGUXBSF&OHJOFFS BU,":"$*OD
  3. "QBDIF%SJMM w 4DIFNBGSFF42-&OHJOF NPEF  w %JTUSJCVUJPO.PEF
 XJUI)BEPPQ /P42- $MPVE4UPSBHF

    w &NCFEEFE.PEF
 XJUI-PDBMpMF +40/ $47 547 147 5SZUIJTUPEBZ
  4. 8IZ%SJMM .Z3FBTPO w 8IFO*OFFEUPJNQSPWFXFCBQQ QFSGPSNBODFʜ w *XBOUUPBHHSFHBUFSFTQPOTFUJNF QFSFOEQPJOUʜ w %SJMMDPVMESVORVFSZUP+40/

    ZFBI
  5. 2VJDL4UBSU w 4FF0GpDJBM5VUPSJBM%PDVNFOU
 l%SJMMJONJOVUFTz
 IUUQTESJMMBQBDIFPSHEPDTESJMMJONJOVUFT  w PS%FWFMPQFST*0#MPH&OUSZBCPVU "QBDIF%SJMM*NQSFTTJPO
 IUUQEFWDMBTTNFUIPEKQDMPVEBXTBQBDIFESJMMpSTUJNQSFTTJPO

    w 'PS.BD6TFSVTF)PNFCSFX
 
 SFRVJSFT+%,PSMBUFS $ brew install apache-drill
  6. $POWFSUOHJOYMPH w 8FVTFl-547zGPSNBUGPSOHJOYMPH w -547-BCFMFE5BCTFQBSBUFE7BMVFT
 IUUQMUTWPSH w "QBDIF%SJMMEPFTOPUTVQQPSU-547ZFU TP DPOWFSUMPHGSPN-547UP+40/

    w VTFMUTWJFXDPNNBOE WJBSVCZHFNT host:217.235.96.155 time:2015-09-14T00:00:00+09:00 method:POST uri:/api/hunting/challenge protocol:HTTP/1.1 status:200 size: 5790 request_time:0.383 upstream_response_time:0.438 referer:- user_agent:MyApp/11000 CFNetwork/711.3.18 Darwin/14.0.0 user_id: 76005617 geo:JP
  7. %FNP .MJOFT .#+40/

  8. $VU2VFSZ4USJOH w (SPVQCZVSJ
 CVUVSJDPOUBJOT2VFSZ4USJOHʜ w 4PMVUJPO
 3&(&91@3&1-"$& VSJ `a 

      BTQBUI +------------------------------------------------------------------------------------------------------------+----------+ | uri | maxtime | +------------------------------------------------------------------------------------------------------------+----------+ | /api/costume/hunting_item/equip | 9.695 | | /api/hunting/families?hunting_id=4000222&session=XXXXXXXXe937077b8cbc06ad3aec80fe&client_data_version=1329 | 9.273 | | /api/mypage?os_type=1&platform_type=1&session=XXXXXXXX7968eb69dd20d2bb1aec3943&client_data_version=1329 | 8.316 | | /api/mypage?os_type=2&platform_type=2&session=XXXXXXXXb63727f12bd2724ab9725885&client_data_version=1329 | 8.279 | | /api/hunting/families?hunting_id=4026012&session=XXXXXXXX2dc4a6668288c5f4a675f071&client_data_version=1330 | 7.926 | | /api/hunting/families?hunting_id=4025518&session=XXXXXXXX04634d3293a8c2317454b6a3&client_data_version=1329 | 7.896 |
  9. 1BSTFUJNFTUSJOH w UJNF5  w KVTUBTUSJOH OPUUJNFTUBNQʜ
 where `time` >

    ‘2015-09-14 07:00:00’
 EPFTOPUXPSLBTFYQFDUFEʜ w 4PMVUJPO
 50@5*.&45".1 AUJNFA `ZZZZ..EE5))NNTT; 
 "4BDDFTT@UJNF
  10. %FNP (3061#:  QFSDFOUJMFBOBMZTJT

  11. $PODMVTJPO w "QBDIF%SJMMJTSFBTPOBCMZGBTU w 5FNQPSBSZUBCMFJTVTFGVMGPSMPOH RVFSZ w $"45JTJNQPSUBOUUPSVORVFSZ w 4UJMMMJUUMFIBSEUPVTFZFU

    SFBE EPDVNFOUDBSFGVMMZ
  12. 5IBOLZPV