Tweet
Tweet

Slide 1

Slide 1 text

@৽॓Geek Lounge#3 ෼ੳج൫Meetup @kimutyam 2017/09/19 ηϓςʔχͰ෼ੳج൫ (Treasure Data)Λಋೖͨ͠࿩

Slide 2

Slide 2 text

໊લ໦ଜজ޺ 5XJUUFS!LJNVUZBN ձࣾηϓςʔχɾΦϦδφϧ %%%ͱ*$0/*9Λ࢖ͬͨ෼ੳઃܭΛ޷Έͱ͍ͯ͠Δ4DBMBΤϯδχΞ ۀ຿ࢧԉͷΞʔΩςΫνϟΛߟҊ͢Δ͏ͪʹ෼ੳج൫ܥͷۀ຿ʹ୧Γண͖·ͨ͠ ࣗݾ঺հ

Slide 3

Slide 3 text

PYXISͷྺ࢙ͱ෼ੳج൫ಋೖ·Ͱͷഎܠ

Slide 4

Slide 4 text

PYXISͱ͸ ޿ࠂ഑৴ ࠷దԽ ୅ཧళۀ ຿վળ σʔλ෼ ੳ

Slide 5

Slide 5 text

σʔλ෼ੳॳظ PYXISͷσʔλ ηϓςʔχͷ֤෦໳DB ഔମࣾͷσʔλ ސ٬ͷσʔλ ࣾһͷσʔλ ֎෦ϕϯμʔͷσʔλ ಛఆͷ໨తʹԊͬͯߏ଄Խ͞Εͨσʔλ͕෼ࢄ؅ཧ͞Εɺ࣍ར༻͞Ε͍ͯͨ

Slide 6

Slide 6 text

σʔλͷ࣍ར༻͕ͮ͠Β͍ σʔλ෼ੳॳظͰͷ໰୊఺ ύϑΥʔϚϯε͕ѱ͍ σʔλ͕෼ࢄ؅ཧ͞Ε͍ͯΔ &5-ॲཧ"11ͷཚཱ

Slide 7

Slide 7 text

σʔλͷ࣍ར༻͕ͮ͠Β͍ վળ΁ ύϑΥʔϚϯε͕ѱ͍ &5-ॲཧ"11ͷཚཱ ඇߏ଄Խσʔλͷ஝ੵ ෼ࢄॲཧ "11ͷ؆ུԽπʔϧར༻ σʔλͷू໿Խ σʔλ͕෼ࢄ؅ཧ͞Ε͍ͯΔ

Slide 8

Slide 8 text

σʔλͷ࣍ར༻͕ͮ͠Β͍ ιϦϡʔγϣϯ: σʔλϨΠΫ ύϑΥʔϚϯε͕ѱ͍ &5-ॲཧ"11ͷཚཱ ඇߏ଄Խσʔλͷ஝ੵ ෼ࢄॲཧ "11ͷ؆ུԽπʔϧར༻ σʔλͷू໿Խ σʔλ͕෼ࢄ؅ཧ͞Ε͍ͯΔ

Slide 9

Slide 9 text

σʔλͷ࣍ར༻͕ͮ͠Β͍ ιϦϡʔγϣϯ: σʔλ΢ΣΞϋ΢ε ύϑΥʔϚϯε͕ѱ͍ σʔλ͕෼ࢄ؅ཧ͞Ε͍ͯΔ &5-ॲཧ"11ͷཚཱ ඇߏ଄Խσʔλͷ஝ੵ ෼ࢄॲཧ "11ͷ؆ུԽπʔϧར༻ σʔλͷू໿Խ

Slide 10

Slide 10 text

σʔλͷ࣍ར༻͕ͮ͠Β͍ ιϦϡʔγϣϯ: Treasure Data ύϑΥʔϚϯε͕ѱ͍ σʔλ͕෼ࢄ؅ཧ͞Ε͍ͯΔ &5-ॲཧ"11ͷཚཱ ඇߏ଄Խσʔλͷ஝ੵ ෼ࢄॲཧ "11ͷ؆ུԽπʔϧར༻ σʔλͷू໿Խ

Slide 11

Slide 11 text

Treasure DataΛར༻ͨ͠ΞʔΩςΫνϟ

Slide 12

Slide 12 text

ΞʔΩςΫνϟߏ੒ 1MB[NB%# %BUB5BOL

Slide 13

Slide 13 text

S3Λ1࣍੥͚ʹ͢Δ ετϨʔδ͸ͳ͍ͷͰ4Λར༻ ੜσʔλͷσʔλܗࣜ +40/$479.-ͳͲ ͷ··อଘ 4ϑΝʔετʹ͢Δ͜ͱͰ"UIFOB3FETIJGU&.3ͳͲͷ αʔϏεͱͷ࿈ܞ͕ඞཁʹԠͯ͡Մೳʹ ੜσʔλอଘͷࡍʹσʔλεΩʔϚΛߟྀ͠ͳ͍

Slide 14

Slide 14 text

ߏ଄Խσʔλ͸ڥք͚ͮΒΕͨίϯςΩετʹด͡Δ ޿ࠂ഑৴ ࠷దԽ ୅ཧళۀ ຿վળ σʔλ෼ ੳ ʮ୅ཧళۀ຿վળʯ΍ʮ޿ࠂ഑৴࠷దԽʯͷίϯς Ωετ಺ʹσʔλ͸1:9*4಺ʹݶఆ ʮσʔλ෼ੳʯͷίϯςΩετͰ͸࣍ར༻͢Δͨ ΊʹੜσʔλΛ4ʹอଘ

Slide 15

Slide 15 text

S3ͷϨΪϡϨʔγϣϯ #VDLFU1:9*4ͷΞϓϦέʔγϣϯڥքͷ໋໊ʹ͢Δ 1BUI\5%ͷσʔλϕʔε໊^\5%ͷςʔϒϧ໊^ZZZZ..ZZZZ..EE@ ΞϓϦέʔγϣϯؒͷڝ߹Λճආ͢ΔͨΊ &NCVMLଆͰQBUI@QSFpYͰ೔࣌ΛಈతʹࢦఆͰ͖ΔΑ͏ʹ "UIFOBͷύʔςΟγϣϯͰ೔෇ࢦఆ͠΍͘͢ͳΔΑ͏ʹ

Slide 16

Slide 16 text

S3 -> TD

Slide 17

Slide 17 text

Treasure DataͷྖҬ 1MB[NB%# %BUB5BOL 5SFBTVSF%BUBͷϑϧϚωʔδͷαʔϏεΛར༻ 5SFBTVSF8PSLqPX %JHEBH %BUB$POOFDUPS &NCVML 5SFBTVSF8PSLqPXΛར༻͢Δ͜ͱͰɺಠࣗʹ%JHEBH 4FSWFSΛཱͯΔߏஙɾอकίετΛ࡟ݮ 5SFBTVSF8PSLqPX͔Βར༻͠ɺඞཁʹԠͯ͡%JHEBH 4FSWFSΛཱͯΔΑ͏ʹϓϩδΣΫτελʔτ

Slide 18

Slide 18 text

Treasure DataͷྖҬ 1MB[NB%# %BUB5BOL 4͔ΒUE@MPBE 1MB[NB%#͔Β1SFTUPͰूܭͯ͠൚ԽσʔλΛ1MB[NB%# ʹอଘ %BUB5BOL σʔλϚʔτ ʹར༻༻్ʹԊͬͨಛԽ͞Εͨ σʔλΛ֨ೲ ˞%BUB5BOL͸5SFBTVSF%BUB͕ఏڙ͍ͯ͠Δ1PTUHSFTRM

Slide 19

Slide 19 text

ࣗલͰDigdag ServerΛݕ౼ͨ͠ྫ 1MB[NB%# %BUB5BOL 42-͚ͩͰ͸ूܭग़དྷͳ͍৔߹ %JHEBHͷQZUIPOSVCZTIFMMΦϖϨʔλʔΛར༻ ͯ͠ɺಠࣗʹ࣮૷ 5SFBTVSF8PSLqPX͸ະରԠ %BUB$POOFDUPSͰ͸ཉ͍͠ूܭཻ౓͕ಘΒΕͳ͍৔߹ &NCVMLϓϥάΠϯΛ૊ΈࠐΈPS։ൃ

Slide 20

Slide 20 text

Treasure DataΛར༻ͨ͠ۀ຿ϑϩʔ σʔλ஝ੵ σʔλ෼ੳɾՁ஋ݕূ %JHEBHʹΑΔΦʔτϝʔγϣϯԽ ։ൃऀ͕5SFBTVSF%BUBʹσʔλ֨ೲ σʔλΞφϦετ ฐࣾͰ͸ඇΤϯδχΞ͕ଟ͍ ͕5SFBTVSF%BUBͷ(6*Λར༻ ։ൃऀ͕؅ཧ͢Δ 5SFBTVSF%BUBΛհͯ͠։ൃऀͱσʔλΞφϦετͷۀ຿࿈ܞΛ͢Δ

Slide 21

Slide 21 text

Treasure DataΛબ୒ͨ͠Α͔ͬͨ఺

Slide 22

Slide 22 text

ϑϧϚωʔδυͳ෼ੳج൫ΛεϐʔσΟʔʹखʹೖΓɺ෼ੳۀ຿ʹूத͕Ͱ͖Δ "84΍($1Ͱ෼ੳج൫Λࣗલߏங͢Δ৔߹͸ɺ αʔϏεͷ஌ݟΛ্࣋ͬͨͰอकɾӡ༻Λߦ͏ඞཁ͕͋Δ ˞ಋೖޙʹඇػೳཁٻ͕໌֬Խͨ͠Βࣗલ΋ݕ౼΋ ෼ੳۀ຿ʹूதͰ͖Δ

Slide 23

Slide 23 text

σʔλͷΠϯϙʔτΤΫεϙʔτͷΤίγεςϜͷॆ࣮͍ͯ͠·͢ ֤छσʔλϕʔε΍ϑΝΠϧαʔόʔͱͷૄ௨ͷόϦΤʔγϣϯ͕ଟ͘ɺ ਺ଟ͘ͷ֎෦γεςϜͱͷ࿈ܞΛ༰қͰ͢ ֎෦γεςϜͱͷ࿈ܞ͕༰қ

Slide 24

Slide 24 text

5SFBTVSF%BUBʹσʔλ஝ੵΛ͢Ε͹ɺσʔλΞφϦετ͕෼ੳͰ͖Δঢ়ଶʹ ۃ࿦ ΫΤϦ՝ۚ΍సૹྔ՝ۚͰ͸ͳ͍ͷͰ42-ʹਫ਼௨͍ͯ͠ͳ͍͕ΫΤϦൃߦͯ͠΋Ϋϥ΢υഁ࢈͠ͳ ͍ σʔλ෼ੳɾՁ஋ݕূ·Ͱͷεϐʔυײͱ҆৺ײ

Slide 25

Slide 25 text

͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠