SQLベースのMLパイプライン / ML pipeline based on SQL

LT
https://connehito.connpass.com/event/149552/

Transcript

1. SQLϕʔεͷ MLύΠϓϥΠϯ @hatunina 2019/11/05 Connehito Marche vol.6
2. ࣗݾ঺հ • hatunina • ࣄۀձࣾͰσʔλΤϯδχΞ • σʔλج൫, ML, WEB։ൃ • Kaggle Expert • vtuber͕޷͖
3. ΞυϗοΫͳMLҊ݅ʹ ͲͷΑ͏ͳύΠϓϥΠϯͰ ରԠ͍ͯ͠·͔͢ʂ
4. ΞυϗοΫʁ • ͱΓ͋͑ͣͲΕ͘Β͍ਫ਼౓͕ग़Δͷ͔ݟ͍ͨ • Өڹྗͷڧ͍ಛ௃͕஌Γ͍ͨ • Ϧετग़͠ ͳͲͳͲ
5. ඞཁͳॲཧ • σʔληοτ࡞੒ • ಛ௃ྔΤϯδχΞϦϯά • ϋΠύʔύϥϝʔλௐ੔ • ֶश/ਪ࿦ ͜͜Λ42-Ͱʂ
6. ύΠϓϥΠϯ֓ཁ USBJODTW UFTUDTW TDPSFDTW SVOQZ 42-ϨϙδτϦ
   DSFBUF@EBUBTFU NPEFM TDPSFDTW NPEFM είΞϦϯάαʔό ࣮ߦ༻αʔό ग़ྗ ࣮ߦ ςʔϒϧ࡞੒ ഑෍είΞϦϯά தؒςʔϒϧ
   είΞϦϯάςʔϒϧ Ҋ݅͝ͱʹ42-Λม͑Δ͚ͩ
7. create_dataset !"" common !"" annkenn01 !"" annkenn02 #"" annkenn03 common !"" create_feats $ !"" feats_01.sql $ !"" feats_02.sql $ !"" feats_03.sql $ !"" feats_04.sql $ #"" feats_05.sql #"" create_score !"" create_table.sql #"" insert_score.sql annkenn01 !"" create_train_list $ !"" create_train_table.sql $ !"" insert_pos.sql $ #"" insert_neg.sql !"" create_feats -> ../common/create_feats !"" create_score_list $ !"" create_score_table.sql $ #"" insert_score.sql !"" create_score -> ../common/create_score #"" select_dataset #"" select.sql σΟϨΫτϦߏ੒ • commonʹ֤Ҋ݅ͷڞ௨SQLʢಛ௃ྔΤϯδχΞϦϯάɺείΞϦϯά݁ՌςʔϒϧͳͲʣ • Ҋ݅͝ͱʹσΟϨΫτϦ/SQLΛ࡞੒ • ڞ௨SQL͸commonΛࢀর • ਖ਼ྫ/ෛྫͷநग़ • ࢖༻͢Δಛ௃ͷநग़ͳͲ
8. SQLΛϕʔεʹ͢ΔϝϦσϝ • ࢖͍ճ͠͠΍͍͢ • ڞ༗/Ҿܧ͕͠΍͍͢ • ςʔϒϧఆٛॻ͕ͦͷ··࢖༻ಛ௃ͷઆ໌ॻʹͳΔ • DB͕ࠞΜͰ͍Δͱσʔλ࡞੒͕ϘτϧωοΫʹͳΓ͕ͪ
9. Ҏ্Ͱ͢ʂ ࣭໰͋Ε͹࠙਌ձͰ ͓੠͕͚͍ͩ͘͞ʂʂ