Rubyの会社でRustを書くということ

Bcbc80799bc32840662a366768318a9b?s=47 Kobayashi
February 10, 2018

 Rubyの会社でRustを書くということ

Cookpad TechConf 2018 ( https://techconf.cookpad.com/2018/ ) で発表した資料です。

Bcbc80799bc32840662a366768318a9b?s=128

Kobayashi

February 10, 2018
Tweet

Transcript

  1. KOBA789ʢখྛल࿨ʣ ΠϯϑϥετϥΫνϟʔ෦ RubyͷձࣾͰ
 RustΛॻ͘ͱ͍͏͜ͱ Cookpad Inc. Feb 10th, 2018

  2. ࣗݾ঺հ 2 ɾ2017೥৽ଔ ɾΠϯϑϥετϥΫνϟʔ෦
 σʔλج൫άϧʔϓ KOBA789ʢখྛल࿨ʣ

  3. RubyͷձࣾͰ
 RustΛॻ͘ͱ͍͏͜ͱ

  4. 4 Ruby

  5. 5 Go Java Ruby

  6. 6 Go Java Hako (DockerσϓϩΠج൫) Ruby

  7. 7 Go Java Hako (DockerσϓϩΠج൫) Ruby

  8. 8 Go Java Ruby Rust Hello !

  9. 9 Ruby͚ͩͷձࣾ
 Ͱ͸ͳ͍

  10. 10 Push௨஌഑৴ج൫

  11. ɾΠϕϯτൃੜ͝ͱʹૹ৴ ɾʮͭ͘ΕΆ͕ಧ͍ͨʯ ɾ഑৴਺͸গͳ͍ ɾಛఆͷ૚ʹҰ੪഑৴ ɾٳ຾Ϣʔβʔ΁ͷૌٻ ɾ഑৴਺͸ଟ͍ ɾ਺ඦສ௨ن໛ ɾݟੵ΋Γ͕େ੾ ϓογϡ௨஌ 11

    ౎౓഑৴ Ұ੪഑৴
  12. 12 App S3 ج൫ SNS 1. ड৴ઃఆΛΫΤϦ 2. ARNΛΫΤϦ 3.

    S3ʹॻ͖ࠐΈ 4. S3͔ΒಡΈग़͢ 5. SNS΁ૹ৴ ᶃ ᶄ ᶅ ᶇ ᶆ MySQL
  13. 13 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNS΁ૹ৴ S3ʹॻ͖ࠐΉ S3͔ΒಡΈग़͢

  14. 14 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNSʹૹ৴

  15. 15 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNSʹૹ৴ { چج൫ { App

  16. 16 World's Largest
 Rails Monolith

  17. S3 ج൫ SNS 17 App New! MySQL

  18. S3 ج൫ SNS New! DBڞ༗ 18 ίϐϖϩδοΫ App MySQL

  19. 19 ཧ૝ͷPush௨஌഑৴ج൫

  20. 20 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNSʹૹ৴ { } چج൫ ৽ج൫

  21. 21 ϢʔβʔIDࢦఆ ARNࢦఆ λʔήοτ

  22. 22 ड৴ઃఆͰϑΟϧλ

  23. 23 S3 ج൫ SNS MySQL େྔ഑৴ ౎౓഑৴ HTTP API

  24. 24 S3 ج൫ SNS MySQL େྔ഑৴ ౎౓഑৴ HTTP API ϢʔβʔIDͱϝοηʔδ͑͞


    ์ΓࠐΊ͹഑৴Ͱ͖Δ
  25. ɾΠϕϯτൃੜ͝ͱʹૹ৴ ɾʮͭ͘ΕΆ͕ಧ͍ͨʯ ɾ഑৴਺͸গͳ͍ ɾಛఆͷ૚ʹҰ੪഑৴ ɾٳ຾Ϣʔβʔ΁ͷૌٻ ɾ഑৴਺͸ଟ͍ ɾ਺ඦສ௨ن໛ ɾݟੵ΋Γ͕େ੾ ϓογϡ௨஌ 25

    ౎౓഑৴ Ұ੪഑৴
  26. ɾΠϕϯτൃੜ͝ͱʹૹ৴ ɾʮͭ͘ΕΆ͕ಧ͍ͨʯ ɾ഑৴਺͸গͳ͍ ɾಛఆͷ૚ʹҰ੪ૹ৴ ɾٳ຾Ϣʔβʔ΁ͷૌٻ ɾ഑৴਺͸ଟ͍ ɾ਺ඦສ௨ن໛ ɾݟੵ΋Γ͕େ੾ ϓογϡ௨஌ 26

    ౎౓഑৴ ΞϓϦ಺ϚʔέςΟϯά
  27. ɾΠϕϯτൃੜ͝ͱʹૹ৴ ɾʮͭ͘ΕΆ͕ಧ͍ͨʯ ɾ഑৴਺͸গͳ͍ ɾಛఆͷ૚ʹҰ੪ૹ৴ ɾٳ຾Ϣʔβʔ΁ͷૌٻ ɾ഑৴਺͸ଟ͍ ɾ਺ඦສ௨ن໛ ɾݟੵ΋Γ͕େ੾ ϓογϡ௨஌ 27

    ౎౓഑৴ ΞϓϦ಺ϚʔέςΟϯά ߴ଎ͳdry-run ਺ඦສ௨Λ਺෼Ͱ
  28. 28 RubyͰ࡞Δͷ͸େม

  29. Ruby or else 29 ɾ଎౓ɾ҆શੑɾฒߦੑʹϑΥʔΧε ɾίϯύΠϥ͕σʔλڝ߹Λݕࠪ ɾτϨΠτʹΑΔδΣωϦΫε Rust

  30. 30 Rust is not magic

  31. 31 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNSʹૹ৴

  32. 32 ड৴ઃఆΛΫΤϦ ARNΛΫΤϦ SNSʹૹ৴ 6ms 6ms }✕ 100ສ = 3࣌ؒ

  33. 33 Inspired by facebook/dataloader https://github.com/facebook/dataloader

  34. 34 ΫΤϦΛ·ͱΊΔ

  35. 35 ... WHERE user_id = 1; ... WHERE user_id =

    2; ... WHERE user_id = 3; ... WHERE user_id IN (1, 2, 3);
  36. ΫΤϦΛ·ͱΊΔ 36 WHERE user_id = ? →6ms (only 1 id)

    WHERE user_id IN (?, ?, ... ?) →11ms (1000 ids) ϕϯνϚʔΫͷ݁Ռɺ਺ઍ͘Β͍͕Α͍ ࠷దͳόοναΠζ
  37. ΫΤϦΛ·ͱΊΔ 37 ɾΩϡʔʹuser_idΛೖΕΔ ɾΩϡʔʹཷ·͍ͬͯΔuser_idΛҰؾʹऔΓग़͢ ɾIN (...) ͷதʹల։ͯ͠DB΁ΫΤϦ ΞϧΰϦζϜ

  38. 38 KOBA789/batch-recv-rs https://github.com/KOBA789/batch-recv-rs

  39. KOBA789/batch-recv-rs 39 MFU UY SY DSPTTCFBN@DIBOOFMVOCPVOE  MFUDIVOL7FD@SY CBUDI@SFDW 

     VOXSBQ  DPMMFDU  ࢖͍ํ
  40. εϨουؒͷϝοηʔδύογϯάʹ࢖͑Δ
 Multi-producer multi-consumerͳFIFOΩϡʔ SenderͱReceiver͔ΒͳΔ MFU UY SY DSPTTCFBN@DIBOOFMVOCPVOE  KOBA789/batch-recv-rs

    40 crossbeam_channel
  41. KOBA789/batch-recv-rs 41 fn recv(&self) -> Result<T, RecvError> Ωϡʔͷத͔Β࠷ॳͷཁૉΛऔΓग़͢ϝιου ཁૉ͕ͳ͚Ε͹εϨουΛϒϩοΫ͢Δʢॏཁʣ crossbeam_channel::Receiver

  42. KOBA789/batch-recv-rs 42 fn try_iter(&self) -> TryIter<T> Ωϡʔͷத਎ΛΠςϨʔλͱͯ͠ಘΔϝιου impl<'a, T> Iterator

    for TryIter<'a, T> ΋ͪΖΜIteratorτϨΠτΛ࣮૷͍ͯ͠Δ crossbeam_channel::Receiver
  43. KOBA789/batch-recv-rs 43 fn take(self, n: usize) -> Take<Self> ΠςϨʔλ͔Βઌ಄nݸͷཁૉ͚ͩͷΠςϨʔλΛ࡞Δϝιου std::iter::Iterator

  44. KOBA789/batch-recv-rs 44 MFUGJSTUTFMGSFDW  MFUSFTUTFMGUSZ@JUFS UBLF O  MFUJUFSWFD<GJSTU>
 JOUP@JUFS

     DIBJO SFTU  0L JUFS ૊Έ߹ΘͤΔ
  45. 45 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  46. batch_recv(3) 46 1 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  47. 47 1 working... Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  48. 48 1 working... 2 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  49. 49 1 working... 2 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  50. 50 1 working... 2 5 4 3 Ωϡʔ ϝΠϯ
 εϨου

    ΫΤϦ
 εϨου
  51. 51 1 working... 2 4 3 5 Ωϡʔ ϝΠϯ
 εϨου

    ΫΤϦ
 εϨου
  52. 52 2 4 3 5 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  53. batch_recv(3) 53 2 4 3 5 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ


    εϨου
  54. 54 2 4 3 5 Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  55. 55 2 4 3 5 working... } batched Ωϡʔ ϝΠϯ


    εϨου
  56. 56 Q. ΫΤϦ݁Ռ͸
 Ͳ͏΍ͬͯฦ͢?

  57. ΫΤϦ݁Ռ͸Ͳ͏΍ͬͯฦ͢? 57 futures::sync::oneshot Ұ౓͚͔ͩ͠ૹΕͳ͍ΩϡʔͷΑ͏ͳ΋ͷ SenderɹɹͱReceiverɹɹ͔ΒͳΔ ೖ ग़ ʹ஋ΛೖΕΔͱ ͔Βग़ͯ͘Δ ೖ

  58. 58 ೖ 1 ग़ Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  59. 59 ೖ 1 ग़ Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  60. 60 ೖ 1 ग़ A Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ
 εϨου

  61. 61 ೖ 1 ग़ A A Ωϡʔ ϝΠϯ
 εϨου ΫΤϦ


    εϨου
  62. 62 Q.ΫΤϦ͕
 ͨ͘͞Μ͋ͬͨΒ?

  63. 63 ೖ 2 ग़ ೖ 1 ೖ 3 ग़ ग़

  64. 64 ೖ 2 ग़ ೖ 1 ೖ 3 ग़ ग़

    όοναΠζ෼ͷ
 εϨου͕ඞཁ ͜ΕͰ͸ࠔΔ
  65. 65 Tokio + futures-rs https://tokio.rs/

  66. Tokio + futures-rs 66 ɾΠϕϯτϧʔϓͷ࣮૷ ɾ1ݸҎ্ͷFutureΛ࣮ߦ ɾϊϯϒϩοΩϯά I/O API ɾFutureτϨΠτ

    ɾJSͷPromiseͱ
 ಉ༷ͷ֓೦ ɾFutureͷίϯϏωʔλ ɾɹɹɹɹ΋͜Εͷػೳ Tokio futures-rs ೖ ग़
  67. Tokio + futures-rs 67 futures::sync::oneshot Ұ౓͚͔ͩ͠ૹΕͳ͍ΩϡʔͷΑ͏ͳ΋ͷ ɹɹ͸ΠϕϯτϧʔϓͰ؂ࢹͰ͖Δ ग़

  68. 68 ΠϕϯτϧʔϓΛ࢖͑͹
 1εϨουͰͨ͘͞Μͷ
 "ग़"Λ଴ͪ߹ΘͤͰ͖Δ

  69. 69 ೖ 2 ग़ ೖ 1 ೖ 3 ग़ ग़

  70. Tokio + futures-rs 70 ɾΩϡʔͷ޲͜͏ͷεϨου͔Β஋Λฦͯ͠΋Β͏͜ͱ͕Ͱ͖Δ ɾϒϩοΩϯάͳॲཧΛFutureʹม׵͍ͯ͠ΔͱΈͳͤΔ oneshot::Senderɹ ΛΩϡʔ΁ೖΕΔύλʔϯ ೖ

  71. 71 KOBA789/batch-loader-rs https://github.com/KOBA789/batch-loader-rs

  72. KOBA789/batch-loader-rs 72 ɾKeyͷΠςϨʔλ͔ΒValueͷ഑ྻΛฦ͢ϝιουΛ
 ࣮૷Ͱ͖Ε͹SQLҎ֎ʹ΋ద༻Մೳ ɾࠓճͷྫͰ͸Key=ϢʔβʔID, Value=ςʔϒϧͷߦ Key-Valueతͳ೚ҙͷΫΤϦΛόονԽ

  73. ࠷ޙʹ 73 ɾͱʹ͔͘ϚϧνεϨου͕҆શ ɾϚϧνεϨουͱΠϕϯτۦಈͷ͍͍ͱ͜औΓ͕Ͱ͖Δ ɾ҆৺ͯ͠ߴ଎ԽʹऔΓ૊ΊΔ ɾ൚༻ϥΠϒϥϦΛ࡞Γ΍͍͢ܕγεςϜ Rustͷ͍͍ͱ͜Ζ

  74. 74 ͜͏ͯ͠
 ϓογϡ௨஌഑৴ج൫͸
 RustͰॻ͖௚͞Ε·ͨ͠

  75. 75 Ruby͚ͩ͡Όͳ͍ ద੾ͳݴޠͰ
 Ϋʔϧʹ՝୊ղܾ

  76. 76 RustΛॻ͘Πϯλʔϯγοϓ https://internship.cookpad.com/2018/spring/