データを中心とするWebサービスの 開発・運用を実現する技術

E60aa4f80303f3f386898546ddb3686a?s=47 Livesense Inc.
June 26, 2018
79

データを中心とするWebサービスの 開発・運用を実現する技術

2018/6/23 【CAMPHOR x LIVESENSE】 Webサービスの開発・運用におけるデータエンジニアリング

E60aa4f80303f3f386898546ddb3686a?s=128

Livesense Inc.

June 26, 2018
Tweet

Transcript

  1. σʔλΛத৺ͱ͢ΔWebαʔϏεͷ ։ൃɾӡ༻Λ࣮ݱ͢Δٕज़ ాத ঵ଠ࿠ (@yubessy) CAMPHOR x LIVESENSE WebαʔϏεͷ։ൃɾӡ༻ʹ͓͚ΔσʔλΤϯδχΞϦϯά

  2. About me ాத ঵ଠ࿠ (@yubessy) ςΫϊϩδΧϧϚʔέςΟϯά෦ σʔλϓϥοτϑΥʔϜάϧʔϓ • 2010 -

    2014 ژ౎େֶ޻ֶ෦৘ใֶՊ ܭࢉػՊֶίʔε • 2014 - 2016 ژ౎େֶେֶӃ৘ใֶݚڀՊ ࣾձ৘ใֶઐ߈ • 2016 - גࣜձࣾϦϒηϯε • σʔλ෼ੳج൫ Livesense Analytics ͷ։ൃɾӡ༻ • ػցֶशج൫ Livesense Brain ͷ։ൃɾӡ༻
  3. Works • ࣄۀʹدΓఴ͏σʔλج൫ͷҭͯํ • ػցֶशγεςϜͷ৴པੑΤϯδχΞϦϯά • σʔλ෼ੳΛࢧ͑ΔʮศརΧϥϜʯͷ໰୊఺ͱͦͷղܾࡦ • Ϛϧνίϯςφߏ੒ʹΑΔػցֶशΞϧΰϦζϜͱΞϓϦέʔγϣϯͷૄ݁߹Խ •

    ػցֶशγεςϜͷӡ༻՝୊ͱίϯςφΦʔέετϨʔγϣϯ͕΋ͨΒ͢΋ͷ • Apache Sparkͷ3ͭͷAPI: RDD, DataFrame͔ΒDataset΁ • Facebookͷߴ଎ϕΫτϧܭࢉϥΠϒϥϦfaissΛϨίϝϯυAPIʹ࢖ͬͯΈΔ
  4. ࠓ೔఻͍͑ͨ͜ͱ αʔϏε։ൃͰͳͥσʔλ͕ॏཁʁ αʔϏεͷσʔλ׆༻͸Ͳ͏͋Δ΂͖ʁ σʔλʹؔΘΔ࢓ࣄʹٻΊΒΕΔ͜ͱ͸ʁ

  5. ͦͷલʹ

  6. ͳΜͷͨΊʹαʔϏεΛͭ͘Δͷ͔ʁ

  7. αʔϏεΛͭ͘Δཧ༝ Ϣʔβͷ໾ʹཱͬͯɺ͓ۚΛ΋Β͏

  8. ϦϒηϯεͷαʔϏε͸ Ͳ͏΍ͬͯϢʔβͷ໾ʹཱ͍ͬͯΔʁ Ͳ͏΍͓ͬͯۚΛ΋Β͍ͬͯΔʁ

  9. Ωʔϫʔυ : ੒Ռใुܕ

  10. ͨͱ͑͹ : ϚοϋόΠτͷ͘͠Έ ܝࡌ՝ۚ (ଞࣾαʔϏε) 1. ΞϧόΠτΛޏ͍͍ͨ 2. ٻਓΛܝࡌ ←

    අ༻ൃੜ ࠾༻՝ۚ (ϚοϋόΠτ) 1. ΞϧόΠτΛޏ͍͍ͨ 2. ٻਓΛܝࡌ ← ແྉ 3. Ԡืऀ͕࠾༻͞Εͯॳग़ۈ ← අ༻ൃੜ
  11. ੒ޭใुܕ = αʔϏεͷར༻։࢝Ͱ͸ͳ͘໨తୡ੒ͷ࣌఺ͰใुΛ΋Β͏

  12. ੒ޭใुܕͷϝϦοτ ࠾༻͠ͳ͚Ε͹අ༻͸ൃੜ͠ͳ͍ = اۀͷ͓͕ۚແବʹͳΒͳ͍

  13. σϝϦοτ͸ʁ

  14. ੒ޭใुܕͷσϝϦοτ ࠾༻͠ͳ͚Ε͹අ༻͸ൃੜ͠ͳ͍ = ܝࡌ͕Ϧϒηϯεͷച্ʹͳΒͳ͍

  15. Ͳ͏͢Δʁ

  16. Ϧϒηϯε͕΍Δ΂͖͜ͱ ܝࡌ͚ͩͰͳ͘Ԡืɾ࠾༻·Ͱ੒ޭͤ͞Δ = ϢʔβͱاۀΛ͏·͘Ϛονϯά͢Δ

  17. Ͳ͏΍ͬͯʁ

  18. Ϧϒηϯε͕஫ྗ͍ͯ͠Δ͜ͱ σʔλΛ׆༻͢Δ

  19. ఻͍͑ͨ͜ͱ1 αʔϏε։ൃͰͳͥσʔλ͕ॏཁʁ

  20. σʔλͱฉ͍ͯɾɾɾ • Ϗοάσʔλ, ϦΞϧλΠϜσʔλ, .... • σʔλ෼ੳ, σʔλαΠΤϯε, .... •

    ਓ޻஌ೳ, ू߹஌, .... • ػցֶश, ࣗવݴޠॲཧ, ... • σΟʔϓϥʔχϯά, ... • Hadoop, Spark, ...
  21. Ұ୴๨Ε͍ͯͩ͘͞

  22. ͦ΋ͦ΋ WebαʔϏε͕ѻ͏σʔλͬͯʁ

  23. WebαʔϏε͕ѻ͏σʔλ ίϯςϯπ : αʔϏε͕Ϣʔβʹఏڙ͢Δ৘ใ • ΞϧόΠτٻਓථ • ΞϧόΠτઌͷΫνίϛ Ϣʔβσʔλ :

    αʔϏε͕Ϣʔβ͔Βड͚औΔ৘ใ • ر๬৚݅ (৬छ, ۈ຿஍, ...ʣ • ݕࡧͷΩʔϫʔυ • ϖʔδͷΞΫηεϩά
  24. Ϣʔβσʔλ͸Ͳ͏໾ཱͭʁ

  25. Ϣʔβσʔλͷొ৔ ੲͷWeb : ϦϯΫΛͨͲͬͯࣗྗͰίϯςϯπʹͨͲΓண͘ • αΠτϚοϓɾϦϯΫू • Yahoo! σΟϨΫτϦݕࡧ Google

    ݕࡧͷొ৔ • ίϯςϯπʹϦϯΫΛషΕ͹ Google Bot ͕ΫϩʔϦϯά • Ϣʔβ͸΄͍͠৘ใΛ Google ʹݕࡧΩʔϫʔυͱͯ͠༩͑Δ
  26. Ϣʔβσʔλ΁ͷ஫໨ Ϣʔβ͕༷ʑͳσʔλΛఏڙ͢ΔΑ͏ʹͳͬͨ • Google, Twitter, Instagram, LINE, ... σʔλ׆༻ͷͨΊͷٕज़͕ൃୡͨ͠ •

    େྔσʔλͷ஝ੵ : ෼ࢄॲཧ, Ϋϥ΢υ, ... • ༗༻ͳ৘ใͷநग़ : ػցֶश, ࣗવݴޠॲཧ, ... → ϢʔβσʔλΛར༻ͯ͠ίϯςϯπΛ͏·͘ಧ͚͍ͨ
  27. ϦϒηϯεͰ͸Ͳ͏࢖͍ͬͯΔʁ

  28. ͨͱ͑͹ Ձ஋ͷߴ͍ٻਓΛݟ͚ͭ΍͘͢͢Δ

  29. (࠶ܝ) ϚοϋόΠτͷ͘͠Έ ܝࡌ՝ۚ (ଞࣾαʔϏε) 1. ΞϧόΠτΛޏ͍͍ͨ 2. ٻਓΛܝࡌ ← අ༻ൃੜ

    ࠾༻՝ۚ (ϚοϋόΠτ) 1. ΞϧόΠτΛޏ͍͍ͨ 2. ٻਓΛܝࡌ ← ແྉ 3. Ԡืऀ͕࠾༻͞Εͯॳग़ۈ ← අ༻ൃੜ
  30. ʮՁ஋ͷߴ͍ٻਓʯͱ͸ʁ

  31. Ϣʔβ : ࠾༻͞Ε΍͍͢ اۀ : Ԡื͞Ε΍͍͢ ϚοϋόΠτ : Ԡื &

    ࠾༻͞Ε΍͍͢
  32. ʮՁ஋ͷߴ͍ٻਓʯΛͲ͏΍ͬͯ஌Δʁ

  33. ʮՁ஋ͷߴ͍ٻਓʯͷࢦඪ Ԡื཰ɾ࠾༻཰

  34. Ԡื཰ͱ࠾༻཰ Ԡื཰ = Ԡื਺ / Ӿཡ਺ ࠾༻཰ = ࠾༻਺ /

    Ԡื਺ ٻਓͷՁ஋ ∝ Ԡื཰ ✕ ࠾༻཰ • Ԡื཰͕ߴ͍ → Ϣʔβʹਓؾ͕͋Δ • ࠾༻཰͕ߴ͍ → اۀ͕ਓΛޏ͍͍ͨ
  35. ӾཡɾԠื͕·ͩθϩͳΒʁ

  36. ৽ணٻਓͷԠื཰ɾ࠾༻཰͸ܭࢉͰ͖ͳ͍ → աڈσʔλ͔Βػցֶश

  37. Ԡื཰ɾ࠾༻཰ͷ༧ଌ • ΞϧΰϦζϜ : ϩδεςΟοΫճؼ • ಛ௃ : ৬छ, ۈ຿஍,

    ళฮ਺, ٳ೔, γϑτ਺, ... • ڭࢣ : աڈͷଟ਺ͷٻਓͷӾཡɾԠืɾ࠾༻ ← Ϣʔβσʔλ (ߦಈϩά)
  38. ༧ଌͨ͠஋ΛͲ͏࢖͏ʁ → ݕࡧ݁ՌͷϥϯΩϯά

  39. ݕࡧ݁ՌͷϥϯΩϯά 1. ϢʔβͷݕࡧΩʔϫʔυʹϚον͢ΔٻਓͷϦετΛऔಘ 2. ༧Ί༧ଌ͓͍ͯͨ͠Ԡื཰ɾ࠾༻཰ʹج͍ͮͯϦετΛฒͼସ͑ public void addDocs(TopFieldDocs mainDocs, int

    reSortDocsNum) throws JsenPageLimitExceedingException, JsenInvalidReSortUnitException { int sortUnitIndex = getIndexOf(dhondtUnit, mainDocs.fields); this.docCountTable = new DocCountTable(mainDocs.fields, reSortUnits, reSortDocsNum); java.util.ListIterator<FieldDoc> iter = fieldDocsIter(mainDocs.scoreDocs, reSortDocsNum); HashMap<FieldDoc, Float> reSortedDocs = new HashMap(reSortDocsNum); while (iter.hasNext()) { FieldDoc doc = iter.next(); float score = ((Integer) (doc.fields[sortUnitIndex])).floatValue(); int count = docCountTable.getCountOf(doc); reSortedDocs.put(doc, score / (1 + count * dhondtCoef)); docCountTable.add(doc); } ArrayList<Entry<FieldDoc, Float>> priorityList = new ArrayList<>(reSortedDocs.entrySet()); Collections.sort(priorityList, (obj1, obj2) -> obj2.getValue().compareTo(obj1.getValue())); this.allDocs = priorityList.stream().map(Entry::getKey).toArray(FieldDoc[]::new); }
  40. Q. େม͡Όͳ͍ʁ

  41. A. େมͰ͢

  42. σʔλ׆༻ͷ೉͠͞ ͜ͷγεςϜ͸ൺֱత୯७ͳ෦ྨ ͦΕͰ΋େมͳ͜ͱ͸ͨ͘͞Μ͋Δ • ͲΜͳΞϧΰϦζϜɾಛ௃ΛબͿ͔ʁ • ӾཡɾԠืɾ࠾༻σʔλΛͲ͔͜ΒऔΔ͔ʁ • ༧ଌ஋ΛݕࡧγεςϜʹͲ͏औΓࠐΉ͔ʁ •

    ݕࡧ݁ՌͷιʔτͷܭࢉྔΛͲ͏ݮΒ͔͢ʁ • ͜Ε͕ຊ౰ʹϢʔβ΍اۀʹ໾ʹཱͭͷ͔ʁ
  43. ͳΜͰ೉͍͠ͷʁ

  44. σʔλ׆༻Λ೉͘͢͠ΔݪҼ γεςϜͷෆ࣮֬ੑ͕ߴ͍ ༷ʑͳઐ໳ੑ͕ඞཁʹͳΔ

  45. γεςϜͷෆ࣮֬ੑ͕ߴ͍ ී௨ͷ৔߹ = σʔλ͕มΘͬͯ΋ϩδοΫ͸มΘΒͳ͍ • ϩδοΫ͸ίʔυ͚ͩͰܾ·Δ • ίʔυΛݟΕ͹ೖྗʹର͢Δग़ྗ͸Θ͔Δ ػցֶश =

    σʔλ͕มΘΔͱϩδοΫ͕มΘΔ • ಉ͡ίʔυͰ΋ֶशσʔλ͕ҧ͑͹ผͷϞσϧ • Ϟσϧͷύϥϝʔλ͸ਓ͕ؒཧղ͠ʹ͍͘
  46. γεςϜͷෆ࣮֬ੑ͕ߴ͍ → αʔϏεΠϯ͢Δ·Ͱ݁Ռ͕ݟ͑ͳ͍

  47. ༷ʑͳઐ໳ੑ͕ඞཁʹͳΔ యܕతͳWeb։ൃ • αʔόαΠυ, ϑϩϯτ, Πϯϑϥ, ΞϓϦ σʔλΛ׆༻ͨ͠Web։ൃ • αʔόαΠυ,

    ϑϩϯτ, Πϯϑϥ, ΞϓϦ • ػցֶशΤϯδχΞ, σʔλΤϯδχΞ • ΞφϦετ, σʔλαΠΤϯςΟετ
  48. ༷ʑͳઐ໳ੑ͕ඞཁʹͳΔ → ਓࡐͷෆ଍ɾ։ൃίετͷ૿େ

  49. Α͋͘Δࣦഊ

  50. Ғ͍ਓ ʮਓ޻஌ೳͰ͍͍ײ͡ͷ੒ՌΛग़ͯ͘͠Εʯ

  51. σʔλΤϯδχΞ (Λ೚͞Εͯ͠·ͬͨਓ) ʮΞΫηεϩά͸શ෦͋Γ·͢ʂʂʂʯ

  52. ݱ࣮ ϑΝΠϧαʔόʹཷ·ͬͨ਺ສͷϑΝΠϧ /var/log/apache.log-2012-05-19.gz ͳ͓ɺ͋Δ೔͔ΒಥવϑΥʔϚοτ͕มΘΔ

  53. σʔλαΠΤϯςΟετ (Λ೚͞Εͯ͠·ͬͨਓ) ʮ͜ͷΞϧΰϦζϜͰਫ਼౓͕100%ʹʂʂʂʯ

  54. ݱ࣮ ֶशσʔλ͸40݅ ֶश΋ςετ΋ಉ͡σʔλ ௐ΂ͨΒܾఆ໦ΞϧΰϦζϜ͕աֶश͍ͯͨ͠

  55. ͜͏ͳΒͳ͍ͨΊʹ

  56. ఻͍͑ͨ͜ͱ2 αʔϏεͷσʔλ׆༻͸Ͳ͏͋Δ΂͖ʁ

  57. ͦͷલʹ : େมͳΒ΍Βͳͯ͘Α͍ͷͰ͸ʁ

  58. σʔλ׆༻͸େม͔ͩΒ΍Βͳͯ͘Α͍ʁ ৑ஊͰ͸ͳ͘ɺ͜Ε΋Ұͭͷཱ೿ͳબ୒ σʔλΛ࢖Θͳͯ͘΋αʔϏε͕͏·͍͘͘ͳΒͦΕͰ͍͍ σʔλ͕ແ͍ͷʹσΟʔϓϥʔχϯάͳͲʹखΛग़ͯ͠΋࢓ํͳ͍ ٯʹݴ͑͹ɺσʔλ׆༻Λ΍͍ͬͯ͘ʹ͸ͦΕͳΓͷ౒ྗ͕ඞཁ ͦΕͰ΋ɺσʔλ׆༻ͰαʔϏε͕ΑΓྑ͘ͳΔͳΒ΍Δ΂͖

  59. ղܾ͢΂͖՝୊ ෆ࣮֬ੑΛ͏·͘ѻ͏ ઐ໳ੑΛ੒Ռʹͭͳ͛Δ

  60. ෆ࣮֬ੑΛ͏·͘ѻ͏ʹ͸ʁ

  61. ෆ࣮֬ੑΛ͏·͘ѻ͏ʹ͸ʁ → ҆શɾޮ཰తͳ։ൃͱݕূΛՄೳʹ

  62. γεςϜͷෆ࣮֬ੑΛѻ͏ํ๏ ෆ࣮֬ੑ͸෭࡞༻Ͱ͸ͳ͘ɺຊ࣭తͰආ͚ΒΕͳ͍΋ͷ ෆ࣮֬ੑΛ཈੍͢ΔͷͰ͸ͳ͘ɺ্ख͘ѻ͑ΔΑ͏ʹ͢Δ • σʔλ͕มΘΔͱϞσϧ΋มΘΔ → ։ൃͰ΋ຊ൪ͱಉ͡σʔλΛ࢖͏ • σʔλநग़΍੔ܗʹख͕͔͔ؒΔ →

    σʔλͷ৔ॴͱલॲཧΛҰݩԽ → ͜ΕΛՄೳʹ͢Δσʔλ෼ੳج൫Λͭ͘Δ
  63. Ϧϒηϯεͷσʔλ෼ੳج൫ ௨শ : Livsense Analytics • ϚοϋόΠτɾస৬ձٞɾస৬φϏͳͲɺ֤αʔϏεͷσʔλΛू໿ • ίϯςϯπ (ٻਓ৘ใ౳)

    ͱϢʔβσʔλ (ߦಈϩά౳) ͷ݁߹෼ੳΛՄೳʹ όοΫΤϯυ : Amazon Redshift • ෼ੳ༻్ʹಛԽͨ͠ྻࢤ޲ܕ෼ࢄσʔλ΢ΣΞϋ΢ε • ϨΠςϯγΑΓεϧʔϓοτΛॏࢹͨ͠ΫΤϦΤϯδϯ • 500+ ςʔϒϧ / 100ԯ+ Ϩίʔυ / 100+ ࣾ಺Ϣʔβ
  64. Livesense Analytics ͷશ๴

  65. ೔ʑ౤͛ΒΕ͍ͯΔΫΤϦ SELECT DATE(n_cpl.pull_at) AS dt, CASE WHEN n_cpl.pull_condition_id = 47

    THEN '1' WHEN n_cpl.pull_condition_id = 48 THEN '2' WHEN n_cpl.pull_condition_id = 49 THEN '3' ELSE 'other' END AS list, COUNT(DISTINCT n_c.id) AS contacts, COUNT(DISTINCT CASE WHEN service_start.contact_id IS NOT NULL THEN n_c.id ELSE NULL END) AS agreements, COUNT(DISTINCT CASE WHEN application_proxy.summarizable_id IS NOT NULL THEN n_c.id ELSE NULL END) AS agencies, COUNT(DISTINCT CASE WHEN cv.id IS NOT NULL THEN n_c.id ELSE NULL END) AS conversions FROM nkd.contact_pull_logs AS n_cpl INNER JOIN nkd.contacts AS n_c ON n_cpl.contact_id = n_c.id LEFT JOIN ( SELECT contact_id, staff_id, action_at FROM nkd.histories AS n_h INNER JOIN nkd.histories_history_actions AS n_hha ON n_h.id = n_hha.history_id WHERE n_hha.history_action_id = 1 AND n_h.history_result_id = 1 AND n_h.created_at >= '2017-12-19' ) AS service_start ON n_c.id = service_start.contact_id LEFT JOIN ( SELECT * FROM nkd.kpi_raw_data WHERE name = 'application_proxy' AND created_at >= '2017-12-19' ) AS application_proxy ON n_c.member_id = application_proxy.member_id LEFT JOIN ( SELECT id FROM nkd.applications WHERE recruit_state IN ('hire','pre_hire_confirm_admin') ) AS cv ON application_proxy.summarizable_id = cv.id WHERE n_cpl.pull_at >= '2017-12-19' GROUP BY dt, list ;
  66. ઐ໳ੑΛ੒Ռʹͭͳ͛Δʹ͸ʁ

  67. ઐ໳ੑΛ੒Ռʹͭͳ͛Δʹ͸ʁ → ٕज़બ୒ͷࣗ༝౓Λ޲্ͤ͞Δ

  68. σʔλ׆༻ٕज़ͷଟ༷ੑ ՝୊ɾ໨తɾಘҙ෼໺ʹΑͬͯద͢Δٕज़͸ҟͳΔ ຊདྷ͸֤͕ࣗ100%ͷྗΛൃشͰ͖Δٕज़Λ࢖͍͍ͨ • Web։ൃ : Ruby etc. 㲗 σʔλ׆༻

    : Python etc. • σʔλ׆༻Ͱ΋ R, Python, Julia ͳͲ͸Ұ௕Ұ୹ → ॊೈͳٕज़બ୒͕Ͱ͖Δ࣮ߦ؀ڥΛ੔උ͢Δ
  69. Ϧϒηϯεͷػցֶशج൫ ௨শ : Livsense Brain • ༧ଌɾϨίϝϯυͳͲ֤αʔϏεͷػցֶशγεςϜΛ࣮ߦ • ؀ڥΛίϯςφԽ͠ɺ༷ʑͳγεςϜΛ؆୯ʹσϓϩΠ όοΫΤϯυ

    : Google Kubernetes Engine • ෳ਺ίϯςφͷ࣮ߦ΍ڠௐΛՄೳʹ͢ΔΦʔέετϨʔγϣϯαʔϏε • ҟͳΔγεςϜͰ΋ϦιʔεΛڞ༗ɾ༗ޮ׆༻ • 16+core CPU / 100+GB RAM / 15+छྨ ίϯςφ
  70. Livesense Brain ͷશ๴

  71. R / Shiny ʹΑΔޮՌ༧ଌWebΞϓϦ

  72. Julia ʹΑΔϨίϝϯυΞϧΰϦζϜ࣮૷ function bpmf( users::Vector{Int}, items::Vector{Int}, rates::Vector{Float64}, N::Int, M::Int, D::Int,

    n_sample::Int) rateidx = sparse(users, items, 1:length(rates)) U = zeros(n_sample, N, D) V = zeros(n_sample, M, D) for t in 1:(n_sample-1) SS = (1. / N) * ΣXXt(U[t, :, :]) SU = (1. / N) * sum(U[t, :, :], 1)[:] W0ast = inv(inv(W0) + N * SS + (β0 * N) / (β0 + N) * (μ0 - SU) * (μ0 - SU)') λ_u = rand(Wishart(ν0 + N, symmetric(W0ast))) μ0ast = (β0 * μ0 + N * SU) / (β0 + N) μ_u = rand(MvNormal(μ0ast, symmetric(inv((β0 + N) * λ_u)))) ... for i in 1:N js = find(rateidx[i, :]) SV_i = ΣXXt(V[t, js, :]) λast_inv = inv(λ_u + α * SV_i) SVR_i = sum(V[t, js, :] .* rates[rateidx[i, js].nzval], 1)[:] μast = λast_inv * (α * SVR_i + λ_u * μ_u) U[t+1, i, :] = rand(MvNormal(μast, symmetric(λast_inv))) end ... end return U, V end
  73. Q. ΍ͬͺΓେม͡Όͳ͍ʁ

  74. A. ͦΕͳΓʹେมͰ͢

  75. େม͚ͩͲɾɾɾ ϦϒηϯεͰ͸ɺ͜Ε·Ͱଟ͘ͷࢼߦࡨޡΛੵΈॏͶΔதͰ σʔλ׆༻ʹΑΓϢʔβʹ໾ཱͭαʔϏεΛ࡞ΕΔΑ͏ʹͳ͖ͬͯͨ ߴ͍ઐ໳ੑΛۦ࢖ͯ͠αʔϏεʹߩݙͰ͖Δਓࡐ΋ଟ͍͘Δ ҟͳΔઐ໳ੑΛ࣋ͬͨϝϯόʔ͕ڠྗͰ͖Δ͘͠ΈΛ࡞Γɺ ͦΕΛ׆͔ͯ͠αʔϏεΛΑΓྑ͍͖͍ͯͨ͘͠

  76. ࠓ೔఻͍͑ͨ͜ͱ3 σʔλʹؔΘΔ࢓ࣄʹٻΊΒΕΔ͜ͱ͸ʁ

  77. σʔλ׆༻ͱઐ໳ੑ σʔλʹؔΘΔઐ໳෼໺͸ඇৗʹͨ͘͞Μ͋Δ • ΞφϦετ, σʔλαΠΤϯςΟετ, Ϛʔέλʔ, ... • ػցֶशΤϯδχΞ, σʔλΤϯδχΞ,

    ... ʮ͜Ε͕Ͱ͖Ε͹શ෦Ͱ͖Δʯͱ͍ͬͨۜͷ஄ؙ͸ଘࡏ͠ͳ͍ • PythonͰػցֶश͚ͩͰ͖Ε͹OK → ❌ • RͰσʔλ෼ੳ͚ͩͰ͖Ε͹OK → ❌
  78. Ͳ͏΍ͬͯઐ໳ੑΛ਎ʹ͚ͭΔʁ ͸ͨΒ͘ਓશһ͕৘ใܥग़਎Ͱ΋ͳ͍͠ɺ࠷ॳ͔ΒWebʹৄ͍͠Θ͚Ͱ΋ͳ͍ ͱ͸͍͑ɺ୭Ͱ΋ԿΒ͔ͷσʔλ΍ϩδοΫʹ৮ΕΔػձ͸ҙ֎ͱ͋Δ • ੜ෺ܥͷݚڀࣨͰɺ࣮ݧσʔλͷ౷ܭݕఆΛ͍ͯͨ͠ • લ৬͕ӦۀͰɺച্਺஋Λ΋ͱʹࢪࡦΛཱ͍ͯͯͨ • εϙʔπ׆ಈͰɺ૬खνʔϜͷઓྗ෼ੳΛ͍ͯͨ͠ →

    ԿΒ͔ͷ͖͔͚͔ͬΒσʔλʹڵຯΛ࣋ͭ
  79. σʔλͱ͏·͘෇͖߹͏ʹ͸ʁ τϨϯυʹ૸Βͣɺ·ͣ͸ࣗ෼ͷ໨తͷͨΊʹσʔλΛͲ͏࢖͑Δ͔Λߟ͑Δ ͦͷ্Ͱɺࣗ෼ʹͰ͖ΔखஈΛߟ͑ͯࢼߦࡨޡ͢Δ • ࠓ͋Δσʔλ͔ΒɺͲΜͳ݁Ռ͕ಘΒΕΕ͹໨త͕ୡ੒Ͱ͖Δʁ • ͦͷͨΊʹ͸σʔλʹͲΜͳख๏΍ཧ࿦Λద༻͢Δʁ • ࠓ͋Δσʔλ͕ෆे෼ͳΒɺͲ͏΍Ε͹ิ͏͜ͱ͕Ͱ͖Δʁ ࠷ॳ͔Β͏·͍͘͘͜ͱ͸গͳ͍͕ɺ্ख͍͔͘ͳ͍ཧ༝Λߟ͑Δͷ͕େࣄ

  80. σʔλʹؔΘΔ࢓ࣄʹͭ͘ʹ͸ʁ ࣮ࡍͷ࢓ࣄͰσʔλ͕Ͳ͏࢖ΘΕΔ͔஌Γ͍ͨ → Πϯλʔϯ΍ΞϧόΠτΛ׆༻͢Δ ΠϯλʔϯɾΞϧόΠτબͼͷϙΠϯτ • ֶੜ͚ͩͷϋοΧιϯͰ͸ͳ͘ɺ࣮຿ʹؔΘΕΔػձΛݟ͚ͭΔ • ʮ͜Ε͔Β΍͍͖͍ͬͯͨʯͰ͸ͳ͘ʮ͢Ͱʹ΍͍ͬͯΔʯձࣾΛݟ͚ͭΔ •

    σʔλʹؔΘΔͳΒɺαϙʔτ͢Δ࢓૊Έ͕੔͍ͬͯΔ͔Λฉ͍ͯΈΔ
  81. ͍͞͝ʹ

  82. ࠓ೔ͷ࿩ʹڵຯΛ࣋ͬͯ΋Β͑ͨΒ…

  83. αϚʔΠϯλʔϯ • ظؒ : Ԡ૬ஊ • ର৅ : ػցֶशɾσʔλ෼ੳɾαʔόαΠυ։ൃͷܦݧऀ •

    څ༩ : ࣌څ1200ԁʙ (ަ௨අࢧڅɾԕํͷ৔߹͸॓ധࢪઃख഑) • ಺༰ྫ : • ΫνίϛΛར༻ͨࣗ͠વݴޠॲཧʹΑΔ৽ػೳͷ։ൃ • Factorization Machine ͳͲͷΞϧΰϦζϜΛར༻ͨ͠Ϩίϝϯυ։ൃ • GCP্ͰͷABςετɾόϯσΟοτج൫ͷ։ൃ
  84. Q & A