Data preprocessing for MachineLearning/BI by Golang and MySQL UDF

Slide 1

Slide 1 text

mecompany Inc. 1 2018728 ੈքॳʂ (Pݴޠͱ.Z42-6%'Ͱ࣮ݱ͢Δ ௒ߴ଎σʔλલॲཧ෼ੳج൫ %FWFMPQFST4VNNJU4VNNFS $4FTTJPOEFWTVNJ$ ా໺ޱେथ ϛʔΧϯύχʔגࣜձࣾ (PMBOH0VUTUBOEJOH%BUBFOHJOFFS (0%

Slide 2

Slide 2 text

2 2018728 lલॲཧ൛(PPHMF#JH2VFSZ .-z Λ.Z42-্ʹ(PݴޠͰ࣮૷ͯ͠ ӡ༻·Ͱճ͍ͯ͠·ͨ͠ %FWFMPQFST4VNNJU4VNNFS $4FTTJPOEFWTVNJ$ ా໺ޱେथ ϛʔΧϯύχʔגࣜձࣾ (PMBOH0VUTUBOEJOH%BUBFOHJOFFS (0% mecompany Inc.

Slide 3

Slide 3 text

2018728 mecompany Inc. 3 ϓϩϑΟʔϧ ా໺ޱେथ %BJLJ5BOPHVDIJ !EQSPKFDU ϛʔΧϯύχʔגࣜձࣾ (PMBOH0VUTUBOEJOH%BUBFOHJOFFS (0% ྵࡉιϑτϋ΢ε 7#/&5 4*FS 8FCαʔϏεاۀ ྲྀ࿘ͷຽ ओݴޠ $+BWB ػցֶश *P5τϥΠΞϧࢀը 1ZUIPO (PMBOH લॲཧϥΠϒϥϦ։ൃͳͲ "LJCBHP (PMBOH5PLZP $PEFGPS+BQBO Ϧʔμʔक़ %FW-07& 1VC 50$G& #PPUDBNQ 8"$"5& ϛʔΧϯύχʔגࣜձࣾ

Slide 4

Slide 4 text

ϓϩϑΟʔϧ mecompany Inc. 4 8 C: % # G+K $/& /0B -! 8 XVWV WX)Y( ,J'EZSYYSXQ XeQ 3 I ¡A>#¢ P* 9 ¡bci¢ WRVVV W_ 51N?D M6F ldg ean mjg "L@ B" VYS\Z[]S^[]_ VYS\\\]SZ_XX t||yÙUvrpxvyow~Tvr t||yÙU{p}ruTvr t||yÙU{p}ruqo|oTvr t||yÙU{p}ruZvzTvr L@ ?51 U7DU? =2H / / / fl / / ; / 4O?.< kbmdg kbmdgQsxzQhj 2018 7)28(

Slide 5

Slide 5 text

mecompany Inc. 5 ҩྍػؔɾༀہɾհޢࣄۀॴͷࢪઃɾ๏ਓσʔλϕʔε ϥΠηϯεͷൢചɾϝϯςφϯεΛߦ͍ͬͯ·͢ɻ 2018728 ϓϩϑΟʔϧ

Slide 6

Slide 6 text

<3S mecompany Inc. 6 7> QP#? y!9/w (Az <[R'_Uw4KwF6_HX Z^\Wkxg* Y^]r`xmoic_\'_%X y!9/w (Az 'U Cefjt@ ;B.E@ ;3uxbx ;3S MJ O2( SCUEL Projectpxlnx VL $=Ndxqf I8G &G ;wMwOdxqdx wwA 1Dsvoah M0 WEB WEB WEB SCUEL Projectpxlnx kxg58 )+ :T] HP 2018"7-28, ϓϩϑΟʔϧ

Slide 7

Slide 7 text

5-K ࠓ೔ͷ࿩͸ʜ mecompany Inc. 7 0 7 IH8 r2)p #:s r2)p #:s "M 9 4-nq_q 4-K EB G,# SCUEL Projectjqghq ND 6F`qkb A1@ !@ 4pEpG`qk`q pp: +=loi^d E* WEB WEB WEB SCUEL Projectjqghq fqc.1 $% 3LZ HP 20187'28& J"]?Q\R[Y CWXSR fqc /YTPUZDVQO

Slide 8

Slide 8 text

6.I ݁࿦͸ʜ mecompany Inc. 8 1 8 HG9 k3*i #;l k3*i #;l "K =Z[^f: 5<)?: 5.gjXj 5.I DB F-# SCUEL Projectcj`aj LC 7EYjd[ A2@ !@ 5iDiFYjdYj ii; ,>ehbW] D+ WEB WEB WEB SCUEL Projectcj`aj _j\/2 $% 4JU HP 20187(28' &_j\U 0 PRMQNOVSST

Slide 9

Slide 9 text

2018728 mecompany Inc. 9 σʔλ։ൃͷݱ৔

Slide 10

Slide 10 text

σʔλ։ൃͷݱ৔ mecompany Inc. 10 ެతػ͔ؔΒ։ࣔ͞Ε͍ͯΔσʔλ &YDFM $47ͷଞ 1%' 8FCαΠτ ࢴʢ෺ཧʣ͕͋Δ 2018728 ࢴͱઓ͏ͨΊʹ0$3Λ࢖ͬͨ͋ͱ͸ ޡೝࣝσʔλͱઓΘͳ͚Ε͹͍͚ͳ͍ɻ

Slide 11

Slide 11 text

mecompany Inc. 11 ެతػ͔ؔΒ։ࣔ͞Ε͍ͯΔσʔλ தʹ͸ zόουσʔλz΋ଘࡏ͢Δ 2018728 • ޡࣈ୤ࣈ • දهΏΕ • ҟମࣈ • ༣ศ൪߸ؒҧ͍ • ి࿩൪߸ ͳ͔ͥ&ϝʔϧΞυϨε • ʙࢢʙ۠ ͷ͕۠ൈ͚͍ͯΔ σʔλ։ൃͷݱ৔

Slide 12

Slide 12 text

! mecompany Inc. 12 / -0 / -0 " 2018728 લॲཧର৅σʔλ ԯ௒߲໨ ʢ਺ेສϨίʔυ º਺ສ߲໨ʣ %'+.* +.*,.) ࠃ಺๏ਓ ສϨίʔυ ެతػ͔ؔΒ։ࣔ͞Ε͍ͯΔσʔλΛ౷߹͠ l࢖͑Δzσʔλʹ͢ΔͨΊɺલॲཧ͕ඞཁ #( $& +.*,.) ଞ͔ࣾΒߪೖͨ͠σʔλϕʔε ਺ԯϨίʔυ σʔλ։ൃͷݱ৔

Slide 13

Slide 13 text

mecompany Inc. 13 ެతͳެ։৘ใ͔Β z࢖͑Δzσʔλʹ࢓্ཱͯ͛ͨ ҩྍػؔɾༀہɾհޢࣄۀॴͷࢪઃɾ๏ਓσʔλϕʔε ϥΠηϯεͷൢചɾϝϯςφϯεΛߦ͍ͬͯ·͢ɻ 2018728 σʔλ։ൃͷݱ৔

Slide 14

Slide 14 text

! mecompany Inc. 14 / -0 ਺ԯʙԯ݅௒ͷσʔλલॲཧ͕ఆৗۀ຿ / -0 " 2018728 %'+.* +.*,.) #( $& +.*,.) • ຖ೥มΘΔ੍౓มߋʹ௥ै • σʔλͷ෼ׂɺ౷߹ɺഇࢭ͕೔ৗతʹൃੜ • ໊শมߋɺࢢ۠ொଜ߹ซͳͲ ॅॴมߋ΋ൃੜ͢Δ σʔλ։ൃͷݱ৔

Slide 15

Slide 15 text

! σʔλ։ൃͷ࣮ࡍ mecompany Inc. 15 / -0 ߴ଎ɺߴਫ਼౓ɺशಘ༰қͳલॲཧ͕ඞཁ / -0 " 2018728 %'+.* +.*,.) #( $& +.*,.) • ࣄۀ͕֦େ͢Δͱѻ͏ྖҬ΍߲໨͕૿͑Δ • ৽ϝϯόʔͰ΋͙͢ʹ ΩϟονΞοϓͰ͖Δ࢓૊Έ͕ඞཁ • σʔλ෼ੳɺઃܭɺ࣮૷શͯͰ ಉ͡࢓૊ΈΛ࢖͍ੜ࢈ੑΛߴΊ͍ͨ

Slide 16

Slide 16 text

2018728 mecompany Inc. 16 ͳͥ.Z42-6%' ʢϢʔβʔఆٛؔ਺ʣ ͳͷ͔

Slide 17

Slide 17 text

ͳͥ.Z42-6%'ͳͷ͔ mecompany Inc. 17 42-͕ෆಘҙͳจࣈܕσʔλͷॲཧΛ Ͳ͏ʹ͔͍ͨ͠ 2018728 • JGOVMM ΍ SFQMBDF͕Ԇʑͱଓ͘42- • ਺ઍߦͷετΞυϓϩγʔδϟ • ૿͑ΔҰํͷσʔλྔʹॲཧ͕௥͍෇͔ͳ͍

Slide 18

Slide 18 text

ͳͥ.Z42-6%'ͳͷ͔ mecompany Inc. 18 42-͕ෆಘҙͳจࣈܕσʔλͷॲཧΛ 6%'ʢϢʔβʔఆٛؔ਺ʣͰ࿫͏ 2018728 • ৽ϝϯόʔͰ΋͙͢ʹΩϟονΞοϓͰ͖Δ࢓૊Έ͕ඞཁ ˠࣾ಺શମͷڞ௨εΩϧ͕42- • σʔλ෼ੳɺઃܭɺ࣮૷શͯͰಉ͡࢓૊ΈΛ࢖͍͍ͨ ˠ6%'ͳΒ42-ͷͳ͔Ͱѻ͏͜ͱ͕Ͱ͖Δ • 42-͕ෆಘҙͳจࣈܕσʔλͷՃ޻ΛͲ͏ʹ͔͍ͨ͠ ˠจࣈܕσʔλॲཧ༻ͷ6%'Λఆٛ͢Δ

Slide 19

Slide 19 text

2018728 mecompany Inc. 19 ͳͥ (Pݴޠͳͷ͔

Slide 20

Slide 20 text

ͳͥ(Pݴޠͳͷ͔ mecompany Inc. 20 ػցֶशͳΒɺࠓͷ࣌͝ੈͩͱ1ZUIPO ػցֶश #*σʔλ෼ੳͷ ͞ΒʹલʹདྷΔॲཧͳͷͰ 1ZUIPOͰͳͯ͘΋Α͍ 2018728

Slide 21

Slide 21 text

ͳͥ(Pݴޠͳͷ͔ mecompany Inc. 21 $$ͱͷ਌࿨ੑ͕ߴ͘ɺॻ͖΍͍͢ 2018728 • .Z42-6%'͸௨ৗ $$Ͱॻ͔ΕΔ • $$ͱͷม׵෦෼͕ ଞݴޠͱൺ΂ͯൺֱతॻ͖΍͍͢ ʢ'PSFJHOGVODUJPOJOUFSGBDF''*(PݴޠͰ͸DHPʣ • ϝϞϦ؅ཧ͸($͕΍ͬͯ͘ΕΔ • ϥΠϒϥϦ΋͍͍۩߹ʹἧͬͯΔ

Slide 22

Slide 22 text

ͳͥ(Pݴޠͳͷ͔ mecompany Inc. 22 ѻ͏σʔλͱॲཧ଎౓ͷཁ݅ʹϚονͨ͠ 2018728 • όΠφϦʹ࣮ͯ͠ߦͰ͖Δ • αϩήʔτϖΞͱ͔ؾʹ͠ͳͯ͘Α͍ • $$ͱಉ౳ͷ଎౓Ͱɺ 3VCZ1ZUIPOͱಉ౳ͷѻ͍΍͢͞ ͕ಘΒΕΔͷ͕(Pݴޠ • จࣈܕσʔλͷॲཧ͕ओͳ໨తͳͷͰɺ 1BOEBT΍/VNQZΛ࢖͏ඞཁੑ͕ͳ͔ͬͨ

Slide 23

Slide 23 text

ͳͥ(Pݴޠͳͷ͔ mecompany Inc. 23 কདྷతͳ֦ுੑ 2018728 • $-*΍8FC"1*ϑϨʔϜϫʔΫͷ ϥΠϒϥϦ͕ἧ͍ͬͯΔ • $MFBO"SDIJUFDUVSFͰ࢓ཱͯΕ͹ɺ $-*΍8FC"1*ͳͲ΁ͷల։΋༰қ 6%'΋&YUFSOBM*OUFSGBDFT ͱͯ͠ѻ͏͜ͱ͕Ͱ͖Δɻ

Slide 24

Slide 24 text

2018728 mecompany Inc. 24 ։ൃ੒Ռ

Slide 25

Slide 25 text

։ൃ੒Ռ mecompany Inc. 25 2018728 ։ൃͨ͠൚༻จࣈྻॲཧ6%' ● DPODBU@OVMMBCMF OVMM΋͍͍۩߹ʹѻ͑ΔDPODBU ● TQMJUΧϯϚ۠੾ΓσʔλͷY൪໨͚ͩΛऔΓग़͢ ● 8FCαΠτεΫϨΠϐϯά݁Ռͷॲཧ ● ਺ࣈɾΞϧϑΝϕοτɾΧλΧφͷશ֯൒֯ม׵ ● ׽ࣈɾه߸ͷҟମࣈม׵ ● ͻΒ͕ͳΧλΧφ૬ޓม׵ ͳͲଟ਺

Slide 26

Slide 26 text

։ൃ੒Ռ mecompany Inc. 26 2018728 ྫʣ OVMM΋͍͍۩߹ʹѻ͑ΔDPODBU FWFOU@OBNF QMBDF σϒαϛՆ ޚ஡ϊਫιϥγςΟ σϒαϛౙ OVMM OVMM ϕϧαʔϧ౦ژ೔ຊڮ select concat( ifnull(event_name, ’’), ifnull(place, ‘’) ) from event; select concat_nullable( event_name, place ) from event;

Slide 27

Slide 27 text

։ൃ੒Ռ mecompany Inc. 27 2018728 ॲཧ଎౓ 8FCαΠτεΫϨΠϐϯά݁Ռͷॲཧ ● εΫϨΠϐϯά݁Ռͷ +40/σʔλԯ݅Λ࣌ؒͰॲཧ ਺ࣈɾΞϧϑΝϕοτɾΧλΧφͷશ֯൒֯ม׵ ● 1P$ͷ࣌఺ͰετΞυϓϩγʔδϟ͔Β ഒߴ଎Խ ʢ೥݄ʣ ● ݱࡏ͸ετΞυϓϩγʔδϟ͔Β ഒߴ଎Խ

Slide 28

Slide 28 text

2018728 mecompany Inc. 28 ߋͳΔ։ൃ੒Ռ

Slide 29

Slide 29 text

ߋͳΔ։ൃ੒Ռ mecompany Inc. 29 .Z42-6%'ͷཪଆͰ ී௨ͷγεςϜ͕ಈ͘ 2018728 • ཪͰ 3FEJT ʹॲཧ݁ՌΩϟογϡΛ࣋ͨͤΔ • ଞ%#΁42-ΛྲྀͤΔ • ଞࣾ8FC"1*΋ୟ͚Δ 6%' 3FEJT ଞࣾ8FC"1* ଞ%# શͯ &YUFSOBM*OUFSGBDFT $MFBO"SDIJUFDUVSF͸͍͍ͧ .Z42-6%'lͳͷʹz

Slide 30

Slide 30 text

ߋͳΔ։ൃ੒Ռ mecompany Inc. 30 2018728 l߲໨zʹಛԽͨ͠จࣈྻॲཧ6%' • ి࿩൪߸ ͳ͔ͥ&ϝʔϧΞυϨε • ࢢ֎ہ൪͕ൈ͚͍ͯΔ ͳͲͷόουి࿩൪߸σʔλ͔Βਖ਼͍͠ి࿩൪߸Λநग़͢Δ ޡࣈ୤ࣈɺදهΏΕɺҟମࣈɺ༣ศ൪߸ؒҧ͍ɺ ʙࢢʙ۠ ͷ͕۠ൈ͚͍ͯΔ ͳͲͷόουॅॴσʔλΛͳΜͱ͔͍ͨ͠ ॅॴਖ਼نԽ ి࿩൪߸ਖ਼نԽ

Slide 31

Slide 31 text

ߋͳΔ։ൃ੒Ռ mecompany Inc. 31 2018728 l߲໨zʹಛԽͨ͠จࣈྻॲཧ6%' ΄΅ਖ਼͍͠ి࿩൪߸Λநग़Մೳ • ࣌ؒ͋ͨΓສ݅ΛॲཧՄೳ • ࠃ಺࠷ߴͷิ׬ਫ਼౓Λୡ੒ ˞ϥϯμϜʹநग़ͨ͠σʔλ݅Λଞ਺ࣾͷॅॴਖ਼نԽ݁Ռͱൺֱ ॅॴਖ਼نԽ ి࿩൪߸ਖ਼نԽ ͜͜·Ͱ঺հͨ͠ཁૉٕज़ͷ্ʹ੒Γཱ͍ͬͯΔ

Slide 32

Slide 32 text

2018728 mecompany Inc. 32 ͜ΕΒΛ ؾܰʹѻ͑Δ

Slide 33

Slide 33 text

6.I ݁࿦ mecompany Inc. 33 1 8 HG9 k3*i #;l k3*i #;l "K =Z[^f: 5<)?: 5.gjXj 5.I DB F-# SCUEL Projectcj`aj LC 7EYjd[ A2@ !@ 5iDiFYjdYj ii; ,>ehbW] D+ WEB WEB WEB SCUEL Projectcj`aj _j\/2 $% 4JU HP 20187(28' &_j\U 0 PRMQNOVSST