range over funcの使い道と非同期N+1リゾルバーの夢 / about a range over func

kamakura.go #7

1. range over funcͷ࢖͍ಓͱ ඇಉظN+1Ϧκϧόʔͷເ @mackee_w
2. ஻Δਓ • X: @mackee_w • GitHub: @mackee • ໘ന๏ਓΧϠοΫ • όοΫΤϯυ & SRE • ࠷ۙ͸RAGύΠϓϥΠϯͳͲΛࢼ࡞ ͍ͯ͠·͢
3. range over func͓͞Β͍ + ࢖͍ಓ
4. Go 1.23 Release NotesΑΓ
5. ͬ͘͟Γ range over ** Go language specΑΓ
6. ͬ͘͟Γrange over func • for range … ͷ … ʹೖΕΒΕΔ΍ͭͰɺ೚ҙͷڍಈΛ͢Δ΍͕ͭ࡞ΕΔΑ͏ ʹͳΓ·ͨ͠Α • ݴޠઃܭଆͷҙਤ: ൓෮ॲཧͷҰൠԽ͞Εͨॲཧύλʔϯɺͭ·ΓΠςϨʔ λʔΛݴޠػೳͱͯ͠૊ΈࠐΜͩ • *database/sql.Rows ΍ *bu fi o.ScannerͳͲ͸ΠςϨʔλʔΛಠࣗʹ࣮૷͠ ͍ͯΔ(ͦͯͦ͠ΕͧΕΠϯλʔϑΣΠε͕ҧ͏) • ͜ΕΒΛେ౷Ұ͢Δҙਤ͕͋ΔͷͰ͸ͳ͍͔(ཁग़య)
7. ͬ͘͟Γ࢖͍ํ
8. iter.Seq͸ࣗ࡞ͯ͠΋ྑ͍ • ೚ҙͷ਺ྻ΍จࣈྻΛϧʔϓ͝ͱʹ ੜ੒͠ଓ͚Δ܅΋࡞ΕΔ • ଞͷrange over **ͱҧ͍ɺແݶʹ ஋Λు͖ग़͢΋ͷ΋࡞ΕΔ • 👉ͷ͸਺ྻΛՃ޻ͯ͠౉͍ͯ͠Δ
9. օ͞Μ͕͙͢ʹ࢖͑Δศརؔ਺ slices.Chunk
10. iter.Seq͸ผʹfor rangeͰ࢖Θͳͯ͘ྑ͍ • iter.Seq ͸ slices.Collect ͰεϥΠεʹม׵Ͱ͖Δ • ແݶʹ஋Λు͘΍ͭ͸ऴΘΒͳ͍ͷͰ஫ҙ • εϥΠεͷΑ͏ͳσʔλྻΛදݱ͢ΔσʔλΛ͞Βʹந৅Խͨ͠ܕͱݴ͑Δ • github.com/BooleanCat/go-functional/v2/it ʹศརϢʔςΟϦςΟؔ਺͕͋Δ • ͍ΘΏΔ github.com/samber/lo ͷ iter.Seq/iter.Seq2൛ • it.Map/it.Map2, it.Filter/it.Filter2 ͳͲ • x/exp/xiter ύοέʔδʹಉछͷؔ਺͕ೖΔ͜ͱ͕ٞ࿦͞Ε͍ͯΔ
11. ࠓ·Ͱͷ࿩ + ΋ͬͱத਎ͷ࿩͕ҎԼͷεϥΠυ ʹ͋Δ • Go Conference 2024: ΠςϨʔλʹΑͬͯGo͸Ͳ͏มΘΔͷ͔ • https://gocon.jp/2024/sessions/2/
12. TIPS: օ͞Μ͕͙͢ʹ࢖͑Δศརؔ਺ slices.Chunk • sliceΛ෼ׂͯ͠ෳ਺ݸͷsliceΛฦ͢iter.Seq[[]E]Λ࡞Δ • Ͳ͏͍͏৔໘ʹศར͔ʁ => όϧΫͰॲཧ͍͕ͨ͠ɺҰؾʹ΍Γ͗͢ Δͷ͸ྑ͘ͳ͍࣌ • SQLͰIN۟Λ༻͍ͨόϧΫऔಘ • Ұ౓ʹ౤͛ΒΕΔID͕nݸʹ੍ݶ͞Ε͍ͯΔόϧΫऔಘAPI
13. ඇಉظN+1Ϧκϧόʔͷເ
14. N+1໰୊ • ιγϟήͷྫ: users ςʔϒϧʹ user_weapons(૷උ෢ث) ςʔϒϧ͕ has-many Ͱ ͍͍ͭͯΔɻҰཡͰuserͱ૷උ෢ثΛηοτͰग़͍ͨ͠ • φΠʔϒͳ࣮૷: SELECT * FROM users WHERE … • ্هͰҾͬு͖ͬͯͨ݁ՌΛߦ͝ͱʹSELECT * FROM user_weapons WHERE user_id = ? • usersͷߦ͝ͱʹSQL͕ൃߦ͞Εͯ͠·͏ͨΊɺRDBMS·Ͱͷ௨৴Φʔόʔϔο υɺSQLύʔείετɺͳͲͳͲ͕ੵΈॏͳΓϨεϙϯελΠϜ͕ྼԽ͢Δ
15. GraphQL͸N+1͕ຊ࣭తʹى͜Γ͕ͪ • GraphQLͰ͸ɺ͋Δϊʔυʹඥͮ͘ࢠڙͷϊʔυΛऔΔΑ͏ͳΫΤϦΛΫϥΠ Ξϯτ͕ಈతʹ૊Έཱͯͯൃߦ͢Δ͜ͱ͕Ͱ͖Δ • ී௨ʹφΠʔϒʹ࣮૷͢ΔͱN+1͕ൃੜ͢Δʂ • ͦ͜ͰGraphQLքͰ͸dataloaderͱ͍͏ςΫ͕޿·͍ͬͯΔ • GoͷdataloaderͰ͸ɺϊʔυͷղܾΛඇಉظʹ͢Δ + छྨ͝ͱͷϊʔυͷղ ܾϦΫΤετΛόοϑΝϦϯάͯ͠όϧΫͰॲཧ͢Δ • dataloaderʹ৐͍ͬͯΕ͹΄ͱΜͲͷN+1͸͍͍ײ͡ʹղܾ͞ΕΔ
16. ҰํRESTfulͷN+1ղܾํ๏ • Ϩεϙϯεͷ૊Έཱͯํ͸ϑϨʔϜϫʔΫͰ͸ͳ͘ΞϓϦέʔγϣϯ ίʔυͷ੹຿ • => dataloaderͷΑ͏ʹఆܕԽͰ͖ͳ͍ • ΄ͱΜͲͷέʔεͰεϥΠεΛฦ͢ͱ͖ʹಉظతʹεϥΠεͷத਎Λ ॱ൪ʹ૊Έཱͯͯߦ͍ͬͯΔͷͰdataloaderͷద༻΋Ͱ͖ͳ͍ • => εϥΠεͷղܾΛඇಉظʹ͢Ε͹ྑ͍ͷͰ͸ʁ
17. github.com/mackee/iterutils/async.Map • go-functional/v2/it.Mapͷඇಉظ൛ • async.Map2΋͋ΔΑʂ • ͜ΕͰεϥΠε಺ͷࢠͷϊʔυͷղܾΛdataloaderʹ೚ͤΔ͜ͱ͕Ͱ ͖ΔͷͰ͸ͳ͍͔
18. ࢖͍ํ
19. sync.WaitGroupͰࣄ଍ΓΔͷͰ͸ʁ • ҧ͍ͱͯ͠͸ɺॏͶ͕͚͕Մೳ • async.Map͸iterΛड͚ೖΕΔ ͭ·Γit.Map΍async.MapΛෳ਺ճ ֻ͚߹ΘͤΔ͜ͱ͕Ͱ͖Δ • ॲཧͷ࣮ߦࣗମ͸ɺslices.Collect ͞ΕΔ·Ͱ஗Ԇ͞ΕΔ • ͪͳΈʹॱ൪͸ೖΕͨiter.Seqͷॱ൪ʹ੔ྻͯ͠ฦ͢Α͏ʹͳͬͯ ͍·͢
20. async.Map2ͷ࢖͍ํ • errgroup.Groupʹ૬౰ • iter.Seq2[V, error] Λฦ͢ • it.TryCollect͸iter.Seq2[V, error]Λ৯΂ͯͲΕ͔ҰͭͰ ΋error͕͋Ε͹ଧͪ੾ͬͯ errorΛฦ͢
21. github.com/vikstrous/dataloadgen • GoͰdataloaderΛߦ͏΍ͭ • GenericsΛ࢖͍ͬͯΔͷ Ͱੜ੒͠ͳͯ͘ศར(ର vektah/dataloaden)
22. ࣮ࡍૣ͘ͳΔ͔ݕূͯ͠ΈΔ • ӈਤͷςʔϒϧΛ༻ҙ • fi xtureΛೖΕͨ users: 10000, items: 10000 user_items: 100000 • ϥϯμϜͰ500݅IDࢦఆͨ͠users ΛऔΓɺ͞Βʹ͍࣋ͬͯΔitemΛ ͱΔૢ࡞ • Fk΍ΠϯσοΫε͸ష͍ͬͯͳ͍
23. N+1͕͋Δঢ়ଶͷίʔυ
24. Dataloaderʹͨ͠ͱ͖ͷಉ༷ͷίʔυ
25. Dataloaderʹͨ͠ͱ͖ͷಉ༷ͷίʔυ
26. ϕϯν݁Ռ 100ഒۙ͘ͷ͕͍ࠩͭͨ
27. ͦͷଞͷߟ࡯ • ࣮͸ISUCONͷաڈ໰Ͱ΍ͬͯ΋είΞ͕ͦΜͳʹ্͕Βͳ͍ • ਪଌ: CPU/ϝϞϦ੍͕ݶ͞Ε͍ͯͨΓ͢Δ͔Βʁ • ͔ͳΓgoroutineΛىಈ͢ΔͨΊʁ • ͔࣮͠͠ࡍͷݱ৔Ͱ͸gqlgen/dataloader͕ػೳ͍ͯ͠ΔҎ্ɺ͜ͷ ख๏΋༗ޮͦ͏ͱݴ͑Δ • (͕͜͜λΠτϧͷʮເʯཁૉͰ͢)
28. ·ͱΊ • GraphQLͰ͸ͳ͍৔໘ͰdataloaderΛ࢖͏৔߹ʹɺasync.Map/ async.Map2͕࢖༻Ͱ͖Δ͜ͱΛࣔͨ͠ • ͦͯ͋͠Δఔ౓ޮՌతͰ͋Δ͜ͱ͕Θ͔ͬͨ • async.Map/dataloaderΛ࢖͏ͱطଘͷσʔλ૊Έཱͯͷߏ଄͔Β͋ ·Γม͑ͳͯ͘ྑ͍ͷͰϦϑΝΫλʹศར • AΛݺͿͷʹBͷmap͕ඞཁΈ͍ͨͳґଘؔ܎΋ݮΒͤΔ
29. Ҏ্Ͱ͢ʂ • ͦͷଞ͓஌Βͤ: gitHub.com/mackee/iterutils ʹطଘͷΠςϨʔλͬ Ά͍interfaceΛiter.Seqʹม׵͢Δؔ਺͕͋ΔͷͰݟͯ͘Εʂ