SQLベースのMLパイプライン / ML pipeline based on SQL

6787be6eb0bef8575926f97e2104cc02?s=47 hatunina
November 05, 2019

SQLベースのMLパイプライン / ML pipeline based on SQL

6787be6eb0bef8575926f97e2104cc02?s=128

hatunina

November 05, 2019
Tweet

Transcript

  1. SQLϕʔεͷ MLύΠϓϥΠϯ @hatunina 2019/11/05 Connehito Marche vol.6

  2. ࣗݾ঺հ • hatunina • ࣄۀձࣾͰσʔλΤϯδχΞ • σʔλج൫, ML, WEB։ൃ •

    Kaggle Expert • vtuber͕޷͖
  3. ΞυϗοΫͳMLҊ݅ʹ ͲͷΑ͏ͳύΠϓϥΠϯͰ ରԠ͍ͯ͠·͔͢ʂ

  4. ΞυϗοΫʁ • ͱΓ͋͑ͣͲΕ͘Β͍ਫ਼౓͕ग़Δͷ͔ݟ͍ͨ • Өڹྗͷڧ͍ಛ௃͕஌Γ͍ͨ • Ϧετग़͠ ͳͲͳͲ

  5. ඞཁͳॲཧ • σʔληοτ࡞੒ • ಛ௃ྔΤϯδχΞϦϯά • ϋΠύʔύϥϝʔλௐ੔ • ֶश/ਪ࿦ ͜͜Λ42-Ͱʂ

  6. ύΠϓϥΠϯ֓ཁ USBJODTW UFTUDTW TDPSFDTW SVOQZ 42-ϨϙδτϦ DSFBUF@EBUBTFU NPEFM TDPSFDTW NPEFM

    είΞϦϯάαʔό ࣮ߦ༻αʔό ग़ྗ ࣮ߦ ςʔϒϧ࡞੒ ഑෍είΞϦϯά தؒςʔϒϧ είΞϦϯάςʔϒϧ Ҋ݅͝ͱʹ42-Λม͑Δ͚ͩ
  7. create_dataset !"" common !"" annkenn01 !"" annkenn02 #"" annkenn03 common

    !"" create_feats $ !"" feats_01.sql $ !"" feats_02.sql $ !"" feats_03.sql $ !"" feats_04.sql $ #"" feats_05.sql #"" create_score !"" create_table.sql #"" insert_score.sql annkenn01 !"" create_train_list $ !"" create_train_table.sql $ !"" insert_pos.sql $ #"" insert_neg.sql !"" create_feats -> ../common/create_feats !"" create_score_list $ !"" create_score_table.sql $ #"" insert_score.sql !"" create_score -> ../common/create_score #"" select_dataset #"" select.sql σΟϨΫτϦߏ੒ • commonʹ֤Ҋ݅ͷڞ௨SQLʢಛ௃ྔΤϯδχΞϦϯάɺείΞϦϯά݁ՌςʔϒϧͳͲʣ • Ҋ݅͝ͱʹσΟϨΫτϦ/SQLΛ࡞੒ • ڞ௨SQL͸commonΛࢀর • ਖ਼ྫ/ෛྫͷநग़ • ࢖༻͢Δಛ௃ͷநग़ͳͲ
  8. SQLΛϕʔεʹ͢ΔϝϦσϝ • ࢖͍ճ͠͠΍͍͢ • ڞ༗/Ҿܧ͕͠΍͍͢ • ςʔϒϧఆٛॻ͕ͦͷ··࢖༻ಛ௃ͷઆ໌ॻʹͳΔ • DB͕ࠞΜͰ͍Δͱσʔλ࡞੒͕ϘτϧωοΫʹͳΓ͕ͪ

  9. Ҏ্Ͱ͢ʂ ࣭໰͋Ε͹࠙਌ձͰ ͓੠͕͚͍ͩ͘͞ʂʂ