Tweet
Tweet

Slide 1

Slide 1 text

Crocos, Inc. Sotaro Karasawa @sotarok http://facebook.com/sotarok εΩʔϚϨεͳ ϩάσʔλͷ ऩूͱूܭͷͨΊͷ σβΠϯύλʔϯ #ds2013 ·ͨ͸ Treasure Data ϋΠύʔ׆༻ज़

Slide 2

Slide 2 text

ࣗݾ঺հ 4PUBSP,BSBTBXB!TPUBSPL ฑ୔૱ଠ࿠ EIBUFOBOFKQTPUBSPL גࣜձࣾΫϩίε$SPDPT*OD 1)1 3FE#VMM

Slide 3

Slide 3 text

ࣾһਓͰ૑ۀ ։ൃऀ౰࣌ਓ ೥݄ʹαʔϏεϩʔϯν ೥݄ʹ5%ಋೖ

Slide 4

Slide 4 text

ࠓ೔࿩͍ͨ͜͠ͱ ΞϓϦέʔγϣϯϩάΛͲ͏ू ΊΔ͔ εΩʔϚϨεͳϩάͱ͸ جຊతͳϩάઃܭ ϩάऩूͷσβΠϯύλʔϯ

Slide 5

Slide 5 text

ࠓ೔ͷ໨త ৘ใڞ༗ɺ৘ใަ׵ ͏ͪͰ͸͜͏΍ͬͯΔΑɺͱ͍͏Ұྫ ܾͯ͠ߨࢣͱͯ͠ɺ͜͏΍Γ·͠ΐ͏ͱݴ ͍ʹདྷͨΘ͚Ͱ͸ͳ͘ɻ ࠓޙɺ͜͏͍͏ωλ͕σΟεΧογϣϯͰ ͖Ε͹͍͍ͳͱ

Slide 6

Slide 6 text

ओʹ 8FCΞϓϦέʔγϣϯ ͷ࿩ Ͱ͕͢ɺ8FCΞϓϦέʔγϣϯ΋ଟ༷Խ͠ ͍ͯ·͢ ޙ൒ͷσβΠϯύλʔϯͷͳ͔Ͱ͍͔ͭ͘ ৮ΕΒΕΔ͔ͳʁ

Slide 7

Slide 7 text

2ϩάऩूΛ͍ͯ͠Δ 2qVFOUEΛ࢖͍ͬͯΔ 25%Λ࢖͍ͬͯΔ

Slide 8

Slide 8 text

ͲΜͳϩάΛूΊͯΔʁ

Slide 9

Slide 9 text

8FCαʔόͷϩά

Slide 10

Slide 10 text

ϩάͱ͍͑͹ 8FCαʔόʔͷϩά 5SFBTVSF%BUBͷνϡʔτϦ Ξϧ΋"QBDIFͷϩά http://docs.treasure-data.com/articles/quickstart

Slide 11

Slide 11 text

͚ͩͲຊ౰ʹཉ͍͠ͷ͸

Slide 12

Slide 12 text

ͲΜͳϢʔβʔ͕ʁ ͲΜͳ୺຤ͰʁͲ͔͜Βʁ ͍ͭԿΛͨ͠ͷ͔ʁ ͲΜͳϘλϯΛΫϦοΫͨ͠ ͷ͔ʁλοϓͨ͠ͷ͔ʁ

Slide 13

Slide 13 text

ΞϓϦέʔγϣϯϩά

Slide 14

Slide 14 text

ͲΜͳϢʔβʔ͕ʁ ɹˠϢʔβʔొ࿥৘ใ ͲΜͳ୺຤ͰʁͲ͔͜Βʁ ɹˠ6"(&0 ͍ͭԿΛͨ͠ͷ͔ʁ ɹˠ63*ΞΫγϣϯ

Slide 15

Slide 15 text

ΞϓϦέʔγϣϯϩάΛ Ͳ͏ूΊΔ͔

Slide 16

Slide 16 text

εΩʔϚϨεϩάͱ͸ʁ

Slide 17

Slide 17 text

εΩʔϚϨεϩάͱ͸ʁ εΩʔϚͷແ͍ϩά

Slide 18

Slide 18 text

ϩάͷεΩʔϚ ͜Ε·Ͱ ˠྫ͑͹547

Slide 19

Slide 19 text

ΧϥϜ໨UJNF ΧϥϜ໨TUBUVT ΧϥϜ໨VSJ ΧϥϜ໨VTFS@JE JOEFY

Slide 20

Slide 20 text

ΧϥϜ໨UJNF ΧϥϜ໨TUBUVT ΧϥϜ໨VSJ ΧϥϜ໨VTFS@JE JOEFY εΩʔϚ

Slide 21

Slide 21 text

for line in open('app.log', 'r'): columns = line.split("\t") time = columns[0] ...

Slide 22

Slide 22 text

߲໨ͷΘ͔ΓͮΒ͞ εΩʔϚมߋͷ೉͠͞ ෼ੳऀͱऩूऀͷೝࣝࠩҟʹ ΑΔࣄނ

Slide 23

Slide 23 text

5%ͷϩά ͱ͍͏͔qVFOUE +40/ { "time":1373876885, "status":200, "request_uri":"/52495/facebook", "session_id":"kn6avn2fuh21r25a65mgm3rjh3", "fb_id":"7c40c5dd2e55cde37a8c40ed80e1", ... }

Slide 24

Slide 24 text

Θ͔Γ΍͍͢ ߲໨Λ௥ՃͰ͖Δ σʔλྔ͸૿͑Δɾ৑௕

Slide 25

Slide 25 text

ΞϓϦέʔγϣϯϩάΛ Ͳ͏ूΊΔ͔

Slide 26

Slide 26 text

جຊతͳϩάઃܭ

Slide 27

Slide 27 text

ΠϕϯτϨίʔυͱͳΔΑ ͏ʹه࿥͢Δ ˞8FCΞϓϦέʔγϣϯͷ৔߹ɺΞΫηε

Slide 28

Slide 28 text

Πϕϯτͱ͸ 8FCΞϓϦέʔγϣϯͳΒ ɾΞΫηε ωΠςΟϒΞϓϦͳΒ ɾΠϕϯτ

Slide 29

Slide 29 text

جຊతͳεΩʔϚΛܾΊΔ

Slide 30

Slide 30 text

Slide 31

Slide 31 text

εΩʔϚϨεͱ͍ͬͯ΋ Ͳ͏͍͏ϩάΛѻ͍ͬͯΔͷ͔ ֤ϨίʔυͰҙຯ͕ҧͬͯ͸ҙ ຯ͕ແ͍

Slide 32

Slide 32 text

جຊతͳεΩʔϚΛܾΊΔ UJNF TUBUVT VSJ VB SFGFSSFS

Slide 33

Slide 33 text

جຊతͳεΩʔϚΛܾΊΔ UJNF TUBUVT VSJ VB SFGFSSFS LTSVͬΆ໊͍લʹ ߹Θ͓ͤͯ͘ͱΘ ͔Γ΍͍͔͢΋

Slide 34

Slide 34 text

8FCαʔόʹ͋Δϩά ͚ͩͰͳ͘ BQQ SPVUF DPOUSPMMFS QSPDFTT@UJNF EFWJDF

Slide 35

Slide 35 text

8FCαʔόʹ͋Δϩά ͚ͩͰͳ͘ BQQ SPVUF DPOUSPMMFS QSPDFTT@UJNF EFWJDF ϑϨʔϜϫʔΫ಺Ͱͷϧʔ ςΟϯά໊ͱ͔ɺίϯτ ϩʔϥ໊ͱ͔ (uri ʹϊΠζ͕͋ͬͯ΋ routing ໊ͰूܭͰ͖Δ)

Slide 36

Slide 36 text

ΞϓϦέʔγϣϯͷ஌Γ͏Δ ଐੑΛඇਖ਼نԽͯ͠Ϩίʔυ ʹؚΊΔ

Slide 37

Slide 37 text

ඇਖ਼نԽ͞ΕͨϨίʔυ TFTTJPO@JE VTFS@JE HFOEFS BHF EFWJDF

Slide 38

Slide 38 text

ͳͥඇਖ਼نԽ͔ͷϝϦοτ +0*/ͤͣʹूܭؔ਺ʹ͔ΔͨΊ

Slide 39

Slide 39 text

ͪͳΈʹ VTFS@JE TFTTJPO@JE ͳͲ͸IBTIԽ͓ͯ͘͠

Slide 40

Slide 40 text

·ͱΊΔͱ ΠϕϯτϨίʔυͱͳΔΑ͏ ʹه࿥͢Δूܭؔ਺ʹ͔ΔͨΊ جຊతͳεΩʔϚΛܾΊΔ ΞϓϦέʔγϣϯͷ஌Γ͏Δଐ ੑΛඇਖ਼نԽͯ͠ϨίʔυʹؚΊΔ

Slide 41

Slide 41 text

͜͜·ͰདྷΔͱɺ΋͏෼ੳ͕Մೳ

Slide 42

Slide 42 text

෼ੳͷྫ SELECT AVG(v[‘process_time’]) FROM access WHERE v[‘route’] = ‘crocos_index’

Slide 43

Slide 43 text

෼ੳͷྫ SELECT v[‘gender’], COUNT(*) FROM access GROUP BY v[‘gender’] ඇਖ਼نԽ͓͍ͯ͠ ͯΑ͔ͬͨʂ

Slide 44

Slide 44 text

෼ੳͷྫ SELECT v[‘gender’], COUNT(*) FROM access GROUP BY v[‘gender’]

Slide 45

Slide 45 text

෼ੳͷྫ Τϥʔͷௐࠪʹ΋ SELECT v[‘route’], v[‘status’], v[‘ua’] FROM access WHERE v[‘user_id’] = ‘xxx’

Slide 46

Slide 46 text

˞௕͘ͳΔͷͰ೔෇ؔ࿈ͷॲཧ͸লུͯ͠·͢ ɹຊ౰͸೔ผʹ(3061#:ͨ͠Γ8&)&3۟ͰߜͬͨΓ

Slide 47

Slide 47 text

εΩʔϚϨεͳ ΞϓϦέʔγϣϯϩά ͷͨΊͷ σβΠϯύλʔϯ Λߟ͑Δ

Slide 48

Slide 48 text

ͯ͞ جຊతͳεΩʔϚΛ࣋ͭ ϩά͕ͨ·Γ࢝Ί·ͨ͠

Slide 49

Slide 49 text

͔͜͜Βઌ͸ ԿΛ෼ੳΛ͍ͨ͠৔߹ʹ ͲΜͳϩάΛೖΕ͓͚ͯ͹ྑ ͍͔ ύλʔϯʹ෼͚ͯߟ͑·͢

Slide 50

Slide 50 text

εΩʔϚϨεͷग़൪

Slide 51

Slide 51 text

جຊతͳεΩʔϚ UJNF TUBUVT VSJ VB SFGFSSFS ͳΜͪΌΒ ͔ΜͪΌΒ

Slide 52

Slide 52 text

جຊతͳεΩʔϚ UJNF TUBUVT VSJ VB SFGFSSFS ͳΜͪΌΒ ͔ΜͪΌΒ ಛఆͷϨίʔυʹɺಛ ผͳҙຯΛ΋ͨͤΔ͜ͱ ͕Ͱ͖Δʂ ͔͠΋ଞͷϨίʔυʹӨ ڹΛ͋ͨ͑Δ͜ͱͳ͘ɻ

Slide 53

Slide 53 text

ύλʔϯ τϥϯβΫγϣϯ

Slide 54

Slide 54 text

ಛผͳҙຯΛ࣋ͭ ΞΫγϣϯͷ੒ޭͳͲΛ ه࿥͍ͨ͠

Slide 55

Slide 55 text

τϥϯβΫγϣϯ uri ΍ route: ϦΫΤετ͕དྷͨ͜ͱ͸Θ͔Δ ͔͠͠ɺຊ౰ʹ੒ޭ͔ͨ͠͸ɺ ΞϓϦέʔγϣϯͰ͔͠Θ͔Β ͳ͍

Slide 56

Slide 56 text

τϥϯβΫγϣϯ key_action key_attr_*

Slide 57

Slide 57 text

τϥϯβΫγϣϯ key_action present:entry:completed ΞϓϦ:ಈ࡞:ঢ়گ ※͜ͷྫ͸ʮొ࿥׬ྃʯ

Slide 58

Slide 58 text

τϥϯβΫγϣϯ key_attr_* τϥϯβΫγϣϯʹؔΘΔ෇Ճ తͳ৘ใΛͭͬ͜Ή εΩʔϚ͸ɺkey_action ͝ͱʹ ҟͳΔ

Slide 59

Slide 59 text

τϥϯβΫγϣϯྫ key_action = shop:register:completed key_attr_user_id = xxxxx key_attr_ref = fb_share

Slide 60

Slide 60 text

τϥϯβΫγϣϯ෼ੳͷྫ SELECT v[‘key_attr_ref’], COUNT(*) FROM access WHERE v[‘key_action’] = ‘...’ GROUP BY v[‘key_attr_ref’]

Slide 61

Slide 61 text

τϥϯβΫγϣϯ෼ੳ ࠷ۙΑ͘ݟͯΔσʔλ ... Ͳͷࢪࡦ͕Ұ൪ޮ͍ͨͷ͔

Slide 62

Slide 62 text

ύλʔϯ Πϕϯτ

Slide 63

Slide 63 text

ΞΫηεʹґଘ͠ͳ͍ ΠϕϯτͷൃੜΛ஌Γ͍ͨ

Slide 64

Slide 64 text

ɾ+BWB4DSJQUʹΑΔΠϕϯτ ɾϞʔμϧͷදࣔ ɾ5XJUUFS΍'BDFCPPL΁ͷ γΣΞ ɾωΠςΟϒΞϓϦ

Slide 65

Slide 65 text

Πϕϯτ tag = app:action:location & some attributes

Slide 66

Slide 66 text

Πϕϯτྫ tag = shop:tweet:shop_item item_id = 1234 tweet_id = xxxxx

Slide 67

Slide 67 text

Πϕϯτ෼ੳͷྫ SELECT v[‘item_id’], COUNT(*) FROM events WHERE v[‘tag’] = ‘shop:tweet:shop_item’ GROUP BY v[‘item_id’]

Slide 68

Slide 68 text

τϥϯβΫγϣϯͱ ࣮͸࢓૊Έ͸͔ΘΒͳ͍

Slide 69

Slide 69 text

εΩʔϚϨεϩάͷѻ͍ํͰ ࠷΋ॏཁͳͷ͸ ղऍͷϧʔϧΛܾΊΔ͜ͱ

Slide 70

Slide 70 text

ଟ෼͕࣌ؒແ͍ͷͰ ͜ͷ΁ΜͰ

Slide 71

Slide 71 text

͜͏͍͏࣌ʹ͸ ͜͏͍͏෩ʹσʔλͷूΊͯ ͜͏ղੳ͠Α͏ ͱ͍͏ͷΛڞ༗͍ͨ͠

Slide 72

Slide 72 text

஫ҙ͍ͨ͠ͱ͜Ζ

Slide 73

Slide 73 text

εΩʔϚϨεͱ͍͑Ͳ ࣄલͷϩάઃܭΛ͔ͬ͠Γ΍Δ ϩά͸Ұ౓ೖΕΔͱมߋ͕೉͍͠ ˠ෼ੳ͍߲ͨ͠໨ͷ࿙Ε͕ແ͍͔ ϓϥΠόγʔ໰୊ʹ΋ؾΛ͚ͭΔ

Slide 74

Slide 74 text

No content