Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Apache Drillで 身の回りのログを集計してみる

acidlemon
September 15, 2015

Apache Drillで 身の回りのログを集計してみる

Tokyo Apache Drill Meetup (2015/09/15)の資料です。

acidlemon

September 15, 2015
Tweet

More Decks by acidlemon

Other Decks in Technology

Transcript

  1. "QBDIF%SJMMͰ
    ਎ͷճΓͷϩάΛूܭͯ͠ΈΔ
    -FU`TUSZUPBHHSFHBUF
    JOEJWJEVBMMPHTVTJOH"QBDIF%SJMM
    5PLZP"QBDIF%SJMM.FFUVQ
    !BDJEMFNPO

    View Slide

  2. "CPVU.F
    w 1SPHSBNNJOH(PMBOH 1FSM $
    w %#%8).Z42-3FETIJGU
    w "QBDIF%SJMMIJTUPSZXFFLT
    !BDJEMFNPO
    .BTBUPTIJ,BXB[PF
    4PGUXBSF&OHJOFFS
    BU,":"$*OD

    View Slide

  3. "QBDIF%SJMM
    w 4DIFNBGSFF42-&OHJOF NPEF

    w %JTUSJCVUJPO.PEF

    XJUI)BEPPQ /P42- $MPVE4UPSBHF
    w &NCFEEFE.PEF

    XJUI-PDBMpMF +40/ $47 547 147

    5SZUIJTUPEBZ

    View Slide

  4. 8IZ%SJMM .Z3FBTPO
    w 8IFO*OFFEUPJNQSPWFXFCBQQ
    QFSGPSNBODFʜ
    w *XBOUUPBHHSFHBUFSFTQPOTFUJNF
    QFSFOEQPJOUʜ
    w %SJMMDPVMESVORVFSZUP+40/ ZFBI

    View Slide

  5. 2VJDL4UBSU
    w 4FF0GpDJBM5VUPSJBM%PDVNFOU

    l%SJMMJONJOVUFTz

    IUUQTESJMMBQBDIFPSHEPDTESJMMJONJOVUFT

    w PS%FWFMPQFST*0#MPH&OUSZBCPVU
    "QBDIF%SJMM*NQSFTTJPO

    IUUQEFWDMBTTNFUIPEKQDMPVEBXTBQBDIFESJMMpSTUJNQSFTTJPO
    w 'PS.BD6TFSVTF)PNFCSFX


    SFRVJSFT+%,PSMBUFS
    $ brew install apache-drill

    View Slide

  6. $POWFSUOHJOYMPH
    w 8FVTFl-547zGPSNBUGPSOHJOYMPH
    w -547-BCFMFE5BCTFQBSBUFE7BMVFT

    IUUQMUTWPSH
    w "QBDIF%SJMMEPFTOPUTVQQPSU-547ZFU TP
    DPOWFSUMPHGSPN-547UP+40/
    w VTFMUTWJFXDPNNBOE WJBSVCZHFNT

    host:217.235.96.155 time:2015-09-14T00:00:00+09:00 method:POST
    uri:/api/hunting/challenge protocol:HTTP/1.1 status:200 size:
    5790 request_time:0.383 upstream_response_time:0.438 referer:-
    user_agent:MyApp/11000 CFNetwork/711.3.18 Darwin/14.0.0 user_id:
    76005617 geo:JP

    View Slide

  7. %FNP
    .MJOFT
    .#+40/

    View Slide

  8. $VU2VFSZ4USJOH
    w (SPVQCZVSJ

    CVUVSJDPOUBJOT2VFSZ4USJOHʜ
    w 4PMVUJPO

    3&(&[email protected]&1-"$& VSJ `a
    BTQBUI
    +------------------------------------------------------------------------------------------------------------+----------+
    | uri | maxtime |
    +------------------------------------------------------------------------------------------------------------+----------+
    | /api/costume/hunting_item/equip | 9.695 |
    | /api/hunting/families?hunting_id=4000222&session=XXXXXXXXe937077b8cbc06ad3aec80fe&client_data_version=1329 | 9.273 |
    | /api/mypage?os_type=1&platform_type=1&session=XXXXXXXX7968eb69dd20d2bb1aec3943&client_data_version=1329 | 8.316 |
    | /api/mypage?os_type=2&platform_type=2&session=XXXXXXXXb63727f12bd2724ab9725885&client_data_version=1329 | 8.279 |
    | /api/hunting/families?hunting_id=4026012&session=XXXXXXXX2dc4a6668288c5f4a675f071&client_data_version=1330 | 7.926 |
    | /api/hunting/families?hunting_id=4025518&session=XXXXXXXX04634d3293a8c2317454b6a3&client_data_version=1329 | 7.896 |

    View Slide

  9. 1BSTFUJNFTUSJOH
    w UJNF5
    w KVTUBTUSJOH OPUUJNFTUBNQʜ

    where `time` > ‘2015-09-14 07:00:00’

    EPFTOPUXPSLBTFYQFDUFEʜ
    w 4PMVUJPO

    [email protected]*.&45".1 AUJNFA `ZZZZ..EE5))NNTT;

    "[email protected]

    View Slide

  10. %FNP
    (3061#:
    QFSDFOUJMFBOBMZTJT

    View Slide

  11. $PODMVTJPO
    w "QBDIF%SJMMJTSFBTPOBCMZGBTU
    w 5FNQPSBSZUBCMFJTVTFGVMGPSMPOH
    RVFSZ
    w $"45JTJNQPSUBOUUPSVORVFSZ
    w 4UJMMMJUUMFIBSEUPVTFZFU SFBE
    EPDVNFOUDBSFGVMMZ

    View Slide

  12. 5IBOLZPV

    View Slide