range over funcの使い道と非同期N+1リゾルバーの夢 / about a range over func

by mackee

Slide 1

Slide 1 text

range over funcͷ࢖͍ಓͱ ඇಉظN+1Ϧκϧόʔͷເ @mackee_w

Slide 2

Slide 2 text

஻Δਓ • X: @mackee_w • GitHub: @mackee • ໘ന๏ਓΧϠοΫ • όοΫΤϯυ & SRE • ࠷ۙ͸RAGύΠϓϥΠϯͳͲΛࢼ࡞ ͍ͯ͠·͢

Slide 3

Slide 3 text

range over func͓͞Β͍ + ࢖͍ಓ

Slide 4

Slide 4 text

Go 1.23 Release NotesΑΓ

Slide 5

Slide 5 text

ͬ͘͟Γ range over ** Go language specΑΓ

Slide 6

Slide 6 text

ͬ͘͟Γrange over func • for range … ͷ … ʹೖΕΒΕΔ΍ͭͰɺ೚ҙͷڍಈΛ͢Δ΍͕ͭ࡞ΕΔΑ͏ ʹͳΓ·ͨ͠Α • ݴޠઃܭଆͷҙਤ: ൓෮ॲཧͷҰൠԽ͞Εͨॲཧύλʔϯɺͭ·ΓΠςϨʔ λʔΛݴޠػೳͱͯ͠૊ΈࠐΜͩ • *database/sql.Rows ΍ *bu fi o.ScannerͳͲ͸ΠςϨʔλʔΛಠࣗʹ࣮૷͠ ͍ͯΔ(ͦͯͦ͠ΕͧΕΠϯλʔϑΣΠε͕ҧ͏) • ͜ΕΒΛେ౷Ұ͢Δҙਤ͕͋ΔͷͰ͸ͳ͍͔(ཁग़య)

Slide 7

Slide 7 text

ͬ͘͟Γ࢖͍ํ

Slide 8

Slide 8 text

iter.Seq͸ࣗ࡞ͯ͠΋ྑ͍ • ೚ҙͷ਺ྻ΍จࣈྻΛϧʔϓ͝ͱʹ ੜ੒͠ଓ͚Δ܅΋࡞ΕΔ • ଞͷrange over **ͱҧ͍ɺແݶʹ ஋Λు͖ग़͢΋ͷ΋࡞ΕΔ • 👉ͷ͸਺ྻΛՃ޻ͯ͠౉͍ͯ͠Δ

Slide 9

Slide 9 text

օ͞Μ͕͙͢ʹ࢖͑Δศརؔ਺ slices.Chunk

Slide 10

Slide 10 text

iter.Seq͸ผʹfor rangeͰ࢖Θͳͯ͘ྑ͍ • iter.Seq ͸ slices.Collect ͰεϥΠεʹม׵Ͱ͖Δ • ແݶʹ஋Λు͘΍ͭ͸ऴΘΒͳ͍ͷͰ஫ҙ • εϥΠεͷΑ͏ͳσʔλྻΛදݱ͢ΔσʔλΛ͞Βʹந৅Խͨ͠ܕͱݴ͑Δ • github.com/BooleanCat/go-functional/v2/it ʹศརϢʔςΟϦςΟؔ਺͕͋Δ • ͍ΘΏΔ github.com/samber/lo ͷ iter.Seq/iter.Seq2൛ • it.Map/it.Map2, it.Filter/it.Filter2 ͳͲ • x/exp/xiter ύοέʔδʹಉछͷؔ਺͕ೖΔ͜ͱ͕ٞ࿦͞Ε͍ͯΔ

Slide 11

Slide 11 text

ࠓ·Ͱͷ࿩ + ΋ͬͱத਎ͷ࿩͕ҎԼͷεϥΠυ ʹ͋Δ • Go Conference 2024: ΠςϨʔλʹΑͬͯGo͸Ͳ͏มΘΔͷ͔ • https://gocon.jp/2024/sessions/2/

Slide 12

Slide 12 text

TIPS: օ͞Μ͕͙͢ʹ࢖͑Δศརؔ਺ slices.Chunk • sliceΛ෼ׂͯ͠ෳ਺ݸͷsliceΛฦ͢iter.Seq[[]E]Λ࡞Δ • Ͳ͏͍͏৔໘ʹศར͔ʁ => όϧΫͰॲཧ͍͕ͨ͠ɺҰؾʹ΍Γ͗͢ Δͷ͸ྑ͘ͳ͍࣌ • SQLͰIN۟Λ༻͍ͨόϧΫऔಘ • Ұ౓ʹ౤͛ΒΕΔID͕nݸʹ੍ݶ͞Ε͍ͯΔόϧΫऔಘAPI

Slide 13

Slide 13 text

ඇಉظN+1Ϧκϧόʔͷເ

Slide 14

Slide 14 text

N+1໰୊ • ιγϟήͷྫ: users ςʔϒϧʹ user_weapons(૷උ෢ث) ςʔϒϧ͕ has-many Ͱ ͍͍ͭͯΔɻҰཡͰuserͱ૷උ෢ثΛηοτͰग़͍ͨ͠ • φΠʔϒͳ࣮૷: SELECT * FROM users WHERE … • ্هͰҾͬு͖ͬͯͨ݁ՌΛߦ͝ͱʹSELECT * FROM user_weapons WHERE user_id = ? • usersͷߦ͝ͱʹSQL͕ൃߦ͞Εͯ͠·͏ͨΊɺRDBMS·Ͱͷ௨৴Φʔόʔϔο υɺSQLύʔείετɺͳͲͳͲ͕ੵΈॏͳΓϨεϙϯελΠϜ͕ྼԽ͢Δ

Slide 15

Slide 15 text

GraphQL͸N+1͕ຊ࣭తʹى͜Γ͕ͪ • GraphQLͰ͸ɺ͋Δϊʔυʹඥͮ͘ࢠڙͷϊʔυΛऔΔΑ͏ͳΫΤϦΛΫϥΠ Ξϯτ͕ಈతʹ૊Έཱͯͯൃߦ͢Δ͜ͱ͕Ͱ͖Δ • ී௨ʹφΠʔϒʹ࣮૷͢ΔͱN+1͕ൃੜ͢Δʂ • ͦ͜ͰGraphQLքͰ͸dataloaderͱ͍͏ςΫ͕޿·͍ͬͯΔ • GoͷdataloaderͰ͸ɺϊʔυͷղܾΛඇಉظʹ͢Δ + छྨ͝ͱͷϊʔυͷղ ܾϦΫΤετΛόοϑΝϦϯάͯ͠όϧΫͰॲཧ͢Δ • dataloaderʹ৐͍ͬͯΕ͹΄ͱΜͲͷN+1͸͍͍ײ͡ʹղܾ͞ΕΔ

Slide 16

Slide 16 text

ҰํRESTfulͷN+1ղܾํ๏ • Ϩεϙϯεͷ૊Έཱͯํ͸ϑϨʔϜϫʔΫͰ͸ͳ͘ΞϓϦέʔγϣϯ ίʔυͷ੹຿ • => dataloaderͷΑ͏ʹఆܕԽͰ͖ͳ͍ • ΄ͱΜͲͷέʔεͰεϥΠεΛฦ͢ͱ͖ʹಉظతʹεϥΠεͷத਎Λ ॱ൪ʹ૊Έཱͯͯߦ͍ͬͯΔͷͰdataloaderͷద༻΋Ͱ͖ͳ͍ • => εϥΠεͷղܾΛඇಉظʹ͢Ε͹ྑ͍ͷͰ͸ʁ

Slide 17

Slide 17 text

github.com/mackee/iterutils/async.Map • go-functional/v2/it.Mapͷඇಉظ൛ • async.Map2΋͋ΔΑʂ • ͜ΕͰεϥΠε಺ͷࢠͷϊʔυͷղܾΛdataloaderʹ೚ͤΔ͜ͱ͕Ͱ ͖ΔͷͰ͸ͳ͍͔

Slide 18

Slide 18 text

࢖͍ํ

Slide 19

Slide 19 text

sync.WaitGroupͰࣄ଍ΓΔͷͰ͸ʁ • ҧ͍ͱͯ͠͸ɺॏͶ͕͚͕Մೳ • async.Map͸iterΛड͚ೖΕΔ ͭ·Γit.Map΍async.MapΛෳ਺ճ ֻ͚߹ΘͤΔ͜ͱ͕Ͱ͖Δ • ॲཧͷ࣮ߦࣗମ͸ɺslices.Collect ͞ΕΔ·Ͱ஗Ԇ͞ΕΔ • ͪͳΈʹॱ൪͸ೖΕͨiter.Seqͷॱ൪ʹ੔ྻͯ͠ฦ͢Α͏ʹͳͬͯ ͍·͢

Slide 20

Slide 20 text

async.Map2ͷ࢖͍ํ • errgroup.Groupʹ૬౰ • iter.Seq2[V, error] Λฦ͢ • it.TryCollect͸iter.Seq2[V, error]Λ৯΂ͯͲΕ͔ҰͭͰ ΋error͕͋Ε͹ଧͪ੾ͬͯ errorΛฦ͢

Slide 21

Slide 21 text

github.com/vikstrous/dataloadgen • GoͰdataloaderΛߦ͏΍ͭ • GenericsΛ࢖͍ͬͯΔͷ Ͱੜ੒͠ͳͯ͘ศར(ର vektah/dataloaden)

Slide 22

Slide 22 text

࣮ࡍૣ͘ͳΔ͔ݕূͯ͠ΈΔ • ӈਤͷςʔϒϧΛ༻ҙ • fi xtureΛೖΕͨ users: 10000, items: 10000 user_items: 100000 • ϥϯμϜͰ500݅IDࢦఆͨ͠users ΛऔΓɺ͞Βʹ͍࣋ͬͯΔitemΛ ͱΔૢ࡞ • Fk΍ΠϯσοΫε͸ష͍ͬͯͳ͍

Slide 23

Slide 23 text

N+1͕͋Δঢ়ଶͷίʔυ

Slide 24

Slide 24 text

Dataloaderʹͨ͠ͱ͖ͷಉ༷ͷίʔυ

Slide 25

Slide 25 text

Dataloaderʹͨ͠ͱ͖ͷಉ༷ͷίʔυ

Slide 26

Slide 26 text

ϕϯν݁Ռ 100ഒۙ͘ͷ͕͍ࠩͭͨ

Slide 27

Slide 27 text

ͦͷଞͷߟ࡯ • ࣮͸ISUCONͷաڈ໰Ͱ΍ͬͯ΋είΞ͕ͦΜͳʹ্͕Βͳ͍ • ਪଌ: CPU/ϝϞϦ੍͕ݶ͞Ε͍ͯͨΓ͢Δ͔Βʁ • ͔ͳΓgoroutineΛىಈ͢ΔͨΊʁ • ͔࣮͠͠ࡍͷݱ৔Ͱ͸gqlgen/dataloader͕ػೳ͍ͯ͠ΔҎ্ɺ͜ͷ ख๏΋༗ޮͦ͏ͱݴ͑Δ • (͕͜͜λΠτϧͷʮເʯཁૉͰ͢)

Slide 28

Slide 28 text

·ͱΊ • GraphQLͰ͸ͳ͍৔໘ͰdataloaderΛ࢖͏৔߹ʹɺasync.Map/ async.Map2͕࢖༻Ͱ͖Δ͜ͱΛࣔͨ͠ • ͦͯ͋͠Δఔ౓ޮՌతͰ͋Δ͜ͱ͕Θ͔ͬͨ • async.Map/dataloaderΛ࢖͏ͱطଘͷσʔλ૊Έཱͯͷߏ଄͔Β͋ ·Γม͑ͳͯ͘ྑ͍ͷͰϦϑΝΫλʹศར • AΛݺͿͷʹBͷmap͕ඞཁΈ͍ͨͳґଘؔ܎΋ݮΒͤΔ

Slide 29

Slide 29 text

Ҏ্Ͱ͢ʂ • ͦͷଞ͓஌Βͤ: gitHub.com/mackee/iterutils ʹطଘͷΠςϨʔλͬ Ά͍interfaceΛiter.Seqʹม׵͢Δؔ਺͕͋ΔͷͰݟͯ͘Εʂ