WantedlyでFeature Storeを導入する際に考えたこと

Slide 1

Slide 1 text

Slide 2

Slide 2 text

©2023 Wantedly, Inc. 'FBUVSF4UPSFͱ͸ͳʹ͔ 8BOUFEMZʹಋೖͯ͠Ͳ͏͔ͩͬͨ ຊ೔ͷ͓࿩͢͠Δ͜ͱ Part1 w 'FBUVSF4UPSFͱ͸ͳʹ͔ w 'FBUVSF4UPSFͷඪ४తͳ࣮૷ߏ੒ w 'FBUVSF4UPSFϓϩμΫτͷ঺հ 8BOUFEMZͰ'FBUVSF4UPSFΛಋೖͨ͠എܠ Part2 w 8BOUFEMZͷਪનγεςϜͷཁ݅ w ͜Ε·Ͱͷ՝୊ w 'FBUVSF4UPSFͷࣗ࡞ʹࢸͬͨഎܠ ࣗ࡞ͨ͠'FBUVSF4UPSFͷߏ੒ Part3 w શମߏ੒ w ֤ػೳͷ࣮૷ߏ੒ Part4 w ࡞ͬͯྑ͔ͬͨ͜ͱ w ࣗ࡞ͯ͠ྑ͔ͬͨ͜ͱ w ΋ͬͱΑ͍ͨ͘͠ͱ͜Ζ ࡞ͬͨॴײ 'FBUVSF4UPSF֓ཁ

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

©2023 Wantedly, Inc. Կ͕Ͱ͖Δ Ұ؏ͨ͠ํ๏ͰσʔλΛॲཧ͠ɺ ෳ਺ͷϞσϧؒͰڞ༗Մೳͳ   ಛ௃ྔΛੜ੒ɾ؅ཧͰ͖Δ 'FBUVSF4UPSFͱ͸ ͲΜͳγεςϜ ಛ௃ྔσʔλࣗମ΍ɺ   ͦͷϝλσʔλΛޮ཰తʹอଘɺ ΞΫηεɺ࠶ར༻͢ΔͨΊʹઃܭ ͞ΕͨγεςϜ ࢖͏ͱͲ͏ͳΔ ػցֶशϞσϧ։ൃͷεϐʔυ ΞοϓɺσʔλͷҰ؏ੑͷ֬อɺ ͓ΑͼϞσϧੑೳͷվળʹد༩ ػցֶशͷͨΊͷಛ௃ྔσʔλͷதԝ؅ཧɾڞ༗γεςϜ

Slide 11

Slide 11 text

©2023 Wantedly, Inc. 'FBUVSF4UPSFͷඪ४తͳ࣮૷ߏ੒ ओཁͳͭͷίϯϙʔωϯτ 'FBUVSF3FHJTUSZ ಛ௃ྔ΍ϝλ৘ใͳͲΛҰཡͰ͖Δ ར༻ऀ޲͚ͷΠϯλʔϑΣʔεͷఏڙ 0QFSBUJPOBM.POJUPSJOH $PODFQU%SJGUͳͲϞσϧͷܦ࣌ྼԽ Λݕ஌͢Δಛ௃ྔͷϞχλϦϯάػߏ ref: https://neptune.ai/blog/feature-stores-components-of-a-data-science-factory-guide 5SBOTGPSN ੜσʔλΛಛ௃ྔʹม׵͢Δػߏ ཁ݅ʹΑΓม׵ํ๏͕ҟͳΔ 4USFBNPS#BUDIͳͲ 4UPSBHF ಛ௃ྔΛอ؅͢Δػߏ#BUDIͳΒ %8) ϦΞϧλΠϜͳΒ3FEJTͳͲ 4FSWJOH ಛ௃ྔΛఏڙ͢ΔػߏϦʔΫ͕   ى͖ͳ͍Α͏ʹ࣌ܥྻΛ   ࢀর͢Δ΋ͷ΋͋Δ QPJOUJOUJNF

Slide 12

Slide 12 text

©2023 Wantedly, Inc. 044 w ೥ʹ(PKFLͱ(PPHMF$MPVE͕ϦϦʔε .-ͱ%BUB*OGSBͷ෼཭ w σʔλΠϯϑϥΛந৅Խ೚ҙͷΠϯϑϥ   ($1 "84 4OPX fl BLF౳ Ͱར༻Մೳ σʔλϦʔΫΛճආՄೳ w QPJOUJOUJNFKPJO ޙड़ Λαϙʔτ ֶशͱαʔϏϯάͷҰ؏ੑ୲อ w ֶशɾਪ࿦࣌ʹಉ͡ΠϯλʔϑΣʔεͰར༻Մೳ 'FBUVSF4UPSFϓϩμΫτ঺հd'FBTUd 3FGIUUQTEPDTGFBTUEFWWNBTUFSXIBUJTGFBTU

Slide 13

Slide 13 text

©2023 Wantedly, Inc. ϑϧϚωʔδυαʔϏε w #2ͷσʔλΞΫηεɾΨόφϯεΛ׆༻Ͱ͖Δ ௿ϨΠςϯγͷαʔϏϯά w ಛ௃ྔͷϦΫΤετͷ͕NTͰ׬ྃ .-ɾੜ੒"*ͷ྆ํʹରԠ w ಛ௃ྔͱಉ͡Α͏ʹϕΫτϧຒΊࠐΈ΋֨ೲՄೳ σʔλϦʔΫΛճආՄೳ w QPJOUJOUJNFKPJO ޙड़ Λαϙʔτ 'FBUVSF4UPSFϓϩμΫτ঺հd7FSUFY"*d 3FGIUUQTDMPVEHPPHMFDPNWFSUFYBJEPDTGFBUVSFTUPSF IMKBWBJGT IUUQTDMPVEHPPHMFDPNCMPHKBQSPEVDUTBJNBDIJOFMFBSOJOHOFXWFSUFYBJGFBUVSFTUPSFCJHRVFSZQPXFSFEHFOBJSFBEZ

Slide 14

Slide 14 text

Slide 15

Slide 15 text

©2023 Wantedly, Inc. ิ଍ಛ௃ྔͷάϧʔϐϯά w ݕࡧੑ΍Ϧιʔε؅ཧͷͨΊɺಛ௃ྔ͸͍͔ͭ͘ͷ؍఺ Ͱάϧʔϐϯά͠͞ΕΔ͜ͱ͕ଟ͍ 7FSUFY"*ʹ͓͚Δάϧʔϐϯάྫ w &OUJUZ'FBUVSFΛඥ෇͚Δ୯Ґ w FY Ϣʔβʔ ΞΠςϜ w 'FBUVSF(SPVQσʔλιʔεͷϩέʔγϣϯʹ   جͮ͘άϧʔϓ w 'FBUVSF7JFXҙຯͷ୯Ґʹجͮ͘άϧʔϓ 3FGIUUQTDMPVEHPPHMFDPNWFSUFYBJEPDTGFBUVSFTUPSFMBUFTUPWFSWJFX

Slide 16

Slide 16 text

Slide 17

Slide 17 text

©2023 Wantedly, Inc. ఏڙ͢ΔϢʔβʔମݧ w ϢʔβʹืूΛɺϦΫϧʔλʔʹީิऀΛਪન͢Δ w ީิͱͳΔΞΠςϜʹείΞ͚ͮ͠ɺϥϯΩϯάͱͯ͠ఏڙ͢Δ νʔϜߏ੒ w .-1E.ਓ%4ਓਪનج൫ΤϯδχΞਓ ݱঢ়ͷγεςϜߏ੒ w 1ZUIPOͷKPCΛఆظ࣮ߦ͠ɺਪ࿦݁ՌΛ#JH2VFSZʹ֨ೲ ओʹ%4 w #2ͷ݁ՌΛ3FEJTܦ༝ͰɺϢʔβʹ഑৴ ओʹਪનج൫ΤϯδχΞ ࣮૷࣌ͷཁٻ w 'FBUVSF4UPSF௥Ճਫ਼౓վળΛؚΊϲ݄ਓͰϦϦʔε w ಛ௃ྔ࡞੒Λߦ͏%4ࣗ਎͕ϝϯςͰ͖ΔγεςϜ͕๬·͍͠ 'FBUVSF4UPSFͷಋೖର৅ͱͳͬͨਪનγεςϜͷཁ݅ɾલఏ

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

©2023 Wantedly, Inc. ՝୊ͦͷಛ௃ྔʹ༧ଌ௚લͷσʔλΛར༻Ͱ͖ͳ͍ w ैདྷͷ༧ଌλεΫ͸5SBJO7BMJE5FTUΛ࣌ܥྻ෼ׂ͠ɺ ༧ଌσʔλऩूظؒΛઃ͚͍ͯͨ w ༧ଌσʔλऩूظ͕ؒ͋ΔͨΊɺ௚લͷߦಈ·ͰΛ   ೖྗʹؚΊΔ͜ͱ͕ग़དྷͳ͍ w ਫ਼౓վળͷͨΊʹɺϢʔβͷΠϕϯτൃੜλΠϛϯάͷ ௚લ·ͰΛೖྗσʔλͷऩूظؒͱ͍ͨ͠ w ಛ௃ྔΛఆظతʹอଘ͠ɺϢʔβຖʹར༻͢Δ   ೖྗظؒΛม͑Δඞཁ͕ग़͖ͯͨ

Slide 26

Slide 26 text

©2023 Wantedly, Inc. 'FBUVSF4UPSFͷػೳͱ8BOUFEMZͷ՝୊ͱඞཁੑ ػೳ ݱঢ়ͷ՝୊ ඞཁੑ Feature Registry Ұݩ؅ཧ͍ͯ͠ͳ͍ͨΊɺ   Ͳ͜ͰͲͷಛ௃ྔ͕࢖ΘΕ͍ͯΔͷ͔୳ͮ͠Β͍&࠶ར༻͠ʹ͍͘ ߴ Operational Monitoring ܽଛͳͲͷσάϨʹؾ͚ͮͳ͍ ͨͩ͠ΦϑϥΠϯςετͰ͋Δఔ౓ΧόʔͰ͖Δ ௿ Storage ಛ௃ྔ͕Ϋϥ΢υετϨʔδʹ͔͠࢒͓ͬͯΒͣɺ   σόοά΍࠶ར༻ɺϢʔβຖʹऔಘ࣌ظΛม͑Δ͜ͱ͕೉͍͠ ߴ Transform PythonϨΠϠʔͰBatchॲཧ͍ͯ͠ΔͷͰ஗͍ த Serving Servingػߏ͸طʹ͋ΔͷͰඞཁͳ͍ ແ Ұൠతͳ'FBUVSF4UPSFͷػೳ͕શͯඞཁͰ͋ΔΘ͚Ͱ͸ແ͍ͱΘ͔ͬͨ💡

Slide 27

Slide 27 text

©2023 Wantedly, Inc. ಺੡WT֎෦πʔϧ ಺੡ VertexAI Feast or ͦͷଞOSS Pros ΧελϚΠζ͠΍͍͢   ඞཁͳ෦෼͚࣮ͩ૷Ͱ͖Δ εέʔϥϒϧ   ϦΞϧλΠϜͳಛ௃ྔͷαʔϏϯά ແྉɻίʔυΛશͯಡΊΔ Cons ࣮૷ίετ͕͔͔ΔɻυΩϡϝϯτ౳͸શͯ   ࣗ෼Ͱ༻ҙ͢Δඞཁ͕͋Δ 2023/10ϦϦʔεͰ   ৘ใ͕গͳ͍   ྉ͕͔͔ۚΔ   Transformػߏ͸݁ہ࣮૷͕ඞཁɻ ґଘϥΠϒϥϦͷߋ৽͕   ΍΍஗͍   ࠃ಺ͷར༻ࣄྫ͕͋·Γͳ͍

Slide 28

Slide 28 text

Slide 29

Slide 29 text

©2023 Wantedly, Inc. ࣗ࡞͕ಋೖ΋อक΋   ࠷΋ίετ͕খͦ͞͏ ࣗ࡞͢Δ͜ͱʹͨࠜ͠ڌ ࣗ࡞ˠطଘ੡඼΁ͷ Ҡߦ͸͠΍ͦ͢͏ w طଘ੡඼͸େن໛αʔϏεʹ΋దԠͰ͖ΔΑ͏ʹػೳ͕ଟ͍ w FY ಛ௃ྔͷάϧʔϐϯά΍ϦΞϧλΠϜαʔϏϯά w ϝϯςφϯε͞Ε͍ͯΔϥΠϒϥϦ͚ͩΛબΜͰಋೖͰ͖Δ w $IBU(15ʹΑΓɺ࣮૷ίετ͕Լ͕͍ͬͯΔ w 7FSUFY"*͸ಛ௃ྔ͕#2ʹΞοϓϩʔυ͞Ε͍ͯΔඞཁ͋Γ w ͲͷΈͪɺˢΛߦ͏ػߏ͸ඞཁ w ΑΓଞࣾࣄྫ͕ग़͖͔ͯͯΒஈ֊తʹҠߦ΋Մೳ

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

©2023 Wantedly, Inc. ࣗ࡞ͯ͠ྑ͔ͬͨ͜ͱ w ઃܭΛܦͯɺࣗࣾͰٻΊ͍ͯΔ΋ͷͷཁ͕݅໌֬ʹͳͬͨ w 1ZUIPOͷΈͰ࣮૷͞Ε͍ͯΔͷͰɺ%4ࣗ਎͕ཉ͍͠ػೳΛؾܰʹ13Ͱ௥ՃͰ͖ΔΑ͏ʹͳͬͨ w ಛ௃ྔੜ੒ʹؔ͢Δ։ൃੜ࢈ੑͷ޲্ͷऔΓ૊Έ͕ҰΧॴʹूத͠ɺεέʔϧ͢ΔΑ͏ʹͳͬͨ w େن໛ͳνʔϜߏ੒Λ૝ఆ͍ͯ͠Δπʔϧ͕ଟ͍ͳ͔ɺඞཁे෼ͳػೳΛબ୒ͯ͠ಋೖ͢Δ͜ͱ͕ग़དྷͨ w $IBU(15ͷ୆಄ʹΑΓɺίʔυΛॻ͘ίετ͕֨ஈʹԼ͕͍ͬͯΔͨΊɺૉૣ࣮͘૷Ͱ͖ͨ w 1BOEBT⁶#JH2VFSZ⁶ZBNMؒͷܕΛࣄલʹόϦσʔγϣϯͰ͖ΔͨΊɺ&SSPS΍ܽଛ͕ݮͬͨ ࣗ࡞ͯ͠೉͍͠ͱײͨ͜͡ͱ w ಛ௃ྔΛϦΞϧλΠϜͰαʔϏϯά͢Δػߏͷࣗ࡞͸͔ͳΓ೉ͦ͠͏͓ͦΒ͘7FSUFY"*ʹ৐ͬͨํ͕Α͍ w ੜ੒ͨ͠ಛ௃ྔಉ࢜ΛKPJO͢ΔΫΤϦΛಈతʹ࡞Δػߏ͸ෳࡶԽ͠΍͍͢ɻKJOKBUFNQMBUFͷ׆༻ͳͲ͕ඞཁͦ͏ ࣗ࡞ͨ͠ॴײ

Slide 45

Slide 45 text

©2023 Wantedly, Inc. ·ͱΊ 'FBUVSF4UPSFʹؚ·ΕΔػೳ΍ؔ࿈ϓϩμΫτʹ͍ͭͯ঺հͨ͠ w ಛ௃ྔσʔλࣗମ΍ɺϝλσʔλΛޮ཰తʹอଘɺΞΫηεɺ࠶ར༻͢ΔͨΊʹઃܭ͞ΕͨγεςϜ w Ұ؏ͨ͠ํ๏ͰσʔλΛॲཧ͠ɺෳ਺ͷϞσϧؒͰڞ༗Մೳͳಛ௃ྔΛੜ੒Ͱ͖Δ 8BOUFEMZͰ͸ൺֱݕ౼ͨ݁͠Ռ'FBUVSF4UPSFΛ಺੡Ͱ։ൃͨ͠ w ؔ࿈ϓϩμΫτͷશͯͷػೳ͕ඞཁͰແ͍͜ͱ͕Θ͔ͬͨͨΊ w ࣗ࡞͢Δ͜ͱͰඞཁे෼ͳػೳ͚ͩΛ࣮૷Ͱ͖ͨ 'FBUVSF4UPSFΛಋೖ͢Δ͜ͱͰߴ଎ͳ։ൃͱਫ਼౓ͷվળͰ͖ͨ w σόοά΍ϞχλϦϯάɺϨϏϡʔͷ͠΍͢͞ͳͲɺӡ༻্ͷϖΠϯ͕େ෯ʹվળ͞Εͨ w Ϣʔβຖʹ࠷৽ͷσʔλΛඥ෇͚Δ͜ͱ͕ग़དྷΔͨΊɺਫ਼౓͕վળ͞Εͨ