$30 off During Our Annual Pro Sale. View Details »

Ruby Technologies Activating The Development of COOKPAD

Kenta Murata
December 10, 2012

Ruby Technologies Activating The Development of COOKPAD

Kenta Murata

December 10, 2012
Tweet

More Decks by Kenta Murata

Other Decks in Technology

Transcript

  1. Ruby Technologies
    Activating The Development of
    Kenta Murata
    ͷ։ൃΛࢧ͑Δ Ruby ͷٕज़
    2012/12/10 Ruby Business Seminar
    Tuesday, December 11, 12

    View Slide

  2. @mrkn
    Tuesday, December 11, 12

    View Slide

  3. Tuesday, December 11, 12

    View Slide

  4. In COOKPAD:
    DevInfra Dept.
    In Private:
    CRuby committer
    Tuesday, December 11, 12

    View Slide

  5. What is
    Development of
    Testing in
    Tuesday, December 11, 12

    View Slide

  6. What is
    Tuesday, December 11, 12

    View Slide

  7. Tuesday, December 11, 12

    View Slide

  8. Tuesday, December 11, 12

    View Slide

  9. 20,000K+ users/month
    1,300K+ recipes
    Tuesday, December 11, 12

    View Slide

  10. Recipe Sharing
    only?
    Tuesday, December 11, 12

    View Slide

  11. Tuesday, December 11, 12

    View Slide

  12. Tuesday, December 11, 12

    View Slide

  13. Sharing Recipes
    Tuesday, December 11, 12

    View Slide

  14. Tuesday, December 11, 12

    View Slide

  15. Tuesday, December 11, 12

    View Slide

  16. Premium Categories
    Tuesday, December 11, 12

    View Slide

  17. Tuesday, December 11, 12

    View Slide

  18. Recommended Menus
    Tuesday, December 11, 12

    View Slide

  19. Tuesday, December 11, 12

    View Slide

  20. Tuesday, December 11, 12

    View Slide

  21. Pro’s Recipe Store
    Tuesday, December 11, 12

    View Slide

  22. Tuesday, December 11, 12

    View Slide

  23. Deliver Ingredients
    Tuesday, December 11, 12

    View Slide

  24. Tuesday, December 11, 12

    View Slide

  25. Cooking Schools
    Search & Reservation
    Tuesday, December 11, 12

    View Slide

  26. Sales Information
    Tuesday, December 11, 12

    View Slide

  27. Tuesday, December 11, 12

    View Slide

  28. Tuesday, December 11, 12

    View Slide

  29. Tuesday, December 11, 12

    View Slide

  30. User Collaboration
    Tuesday, December 11, 12

    View Slide

  31. Tuesday, December 11, 12

    View Slide

  32. Tuesday, December 11, 12

    View Slide

  33. Tuesday, December 11, 12

    View Slide

  34. 1. Recipe Sharing
    2. Premium Categories
    3. Recommended Menus
    4. Pro’s Recipe Store
    5. Deliver Ingredients
    6. Cooking Schools
    7. Recipe Request
    8. Q & A
    9. Everyone’s Cafe
    10. Sales Information
    Tuesday, December 11, 12

    View Slide

  35. These features are
    developed in
    the last half-year
    Tuesday, December 11, 12

    View Slide

  36. Ruby and Rails
    Accelerate
    Service Development
    Tuesday, December 11, 12

    View Slide

  37. How do we develop
    COOKPAD?
    Tuesday, December 11, 12

    View Slide

  38. Development of
    Tuesday, December 11, 12

    View Slide

  39. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  40. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  41. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  42. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  43. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  44. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  45. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  46. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  47. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  48. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  49. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  50. Talk
    today
    Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  51. Talk
    today
    Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  52. Talk
    today
    Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  53. Talk
    today
    Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  54. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  55. Idea
    Measure
    Tuesday, December 11, 12

    View Slide

  56. Idea
    Measure
    Ruby
    Rails
    Chanko
    Sara
    RSpec
    Jenkins
    Capistrano
    Unicorn
    Fluentd
    Tuesday, December 11, 12

    View Slide

  57. Idea
    Measure
    Ruby
    Rails
    Chanko
    Sara
    RSpec
    Jenkins
    Capistrano
    Unicorn
    Fluentd
    Tuesday, December 11, 12

    View Slide

  58. http://www.flickr.com/photos/june29/3396011694/
    Chanko
    Tuesday, December 11, 12

    View Slide

  59. Chanko is:
    Tuesday, December 11, 12

    View Slide

  60. Chanko is:
    Plug-in for Rails
    Tuesday, December 11, 12

    View Slide

  61. Chanko is:
    Plug-in for Rails
    Feature Flipping System
    Tuesday, December 11, 12

    View Slide

  62. Chanko is:
    Plug-in for Rails
    Feature Flipping System
    Open Source
    Tuesday, December 11, 12

    View Slide

  63. Tuesday, December 11, 12

    View Slide

  64. http://github.com/cookpad/chanko
    Tuesday, December 11, 12

    View Slide

  65. Tuesday, December 11, 12

    View Slide

  66. http://github.com/cookpad/chanko_sample
    Tuesday, December 11, 12

    View Slide

  67. Building the new feature
    Tuesday, December 11, 12

    View Slide

  68. Building the new feature
    Using topic branch
    Tuesday, December 11, 12

    View Slide

  69. Building the new feature
    Using topic branch
    Modifying the main code
    Tuesday, December 11, 12

    View Slide

  70. Building the new feature
    Using topic branch
    Modifying the main code
    Deploying to staging
    Tuesday, December 11, 12

    View Slide

  71. Master branch
    Tuesday, December 11, 12

    View Slide

  72. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  73. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  74. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  75. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  76. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  77. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  78. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  79. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  80. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  81. Master branch
    Topic branch
    Tuesday, December 11, 12

    View Slide

  82. Problems:
    Tuesday, December 11, 12

    View Slide

  83. Problems:
    Degrade Stability
    Tuesday, December 11, 12

    View Slide

  84. Problems:
    Degrade Stability
    Low-Quality Code
    Tuesday, December 11, 12

    View Slide

  85. Problems:
    Degrade Stability
    Low-Quality Code
    Slowdown Development
    Tuesday, December 11, 12

    View Slide

  86. Requirements:
    Tuesday, December 11, 12

    View Slide

  87. Requirements:
    Service Stability
    Tuesday, December 11, 12

    View Slide

  88. Requirements:
    Service Stability
    Code Quality
    Tuesday, December 11, 12

    View Slide

  89. Requirements:
    Service Stability
    Code Quality
    Fast Development
    Tuesday, December 11, 12

    View Slide

  90. http://www.flickr.com/photos/june29/3396011694/
    Chanko
    Tuesday, December 11, 12

    View Slide

  91. Tuesday, December 11, 12

    View Slide

  92. http://bit.ly/chanko-railsconf2012
    Tuesday, December 11, 12

    View Slide

  93. In COOKPAD
    Tuesday, December 11, 12

    View Slide

  94. 10+ Teams
    In COOKPAD
    Tuesday, December 11, 12

    View Slide

  95. 10+ Teams
    1-3 Developers per team
    In COOKPAD
    Tuesday, December 11, 12

    View Slide

  96. 10+ Teams
    30-40 Chanko units
    1-3 Developers per team
    In COOKPAD
    Tuesday, December 11, 12

    View Slide

  97. Usage
    Tuesday, December 11, 12

    View Slide

  98. Time-limited Feature
    Usage
    Tuesday, December 11, 12

    View Slide

  99. Time-limited Feature
    User-limited Feature
    Usage
    Tuesday, December 11, 12

    View Slide

  100. Time-limited Feature
    User-limited Feature
    A/B Testing
    Usage
    Tuesday, December 11, 12

    View Slide

  101. Tuesday, December 11, 12

    View Slide

  102. Tuesday, December 11, 12

    View Slide

  103. Service Stability
    Code Quality
    Fast Development
    Chanko
    Tuesday, December 11, 12

    View Slide

  104. Happy ending?
    (20min should have passed since the beginning)
    Tuesday, December 11, 12

    View Slide

  105. Testing in
    Tuesday, December 11, 12

    View Slide

  106. Service Stability
    Code Quality
    Fast Development
    Chanko
    Tuesday, December 11, 12

    View Slide

  107. For what purpose?
    Tuesday, December 11, 12

    View Slide

  108. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  109. For Users
    Tuesday, December 11, 12

    View Slide

  110. 10+ Teams
    30-40 Chanko units
    1-3 Developers per team
    In COOKPAD
    Tuesday, December 11, 12

    View Slide

  111. You can deploy
    whenever you want
    the revisions at which
    the CI was passed
    Tuesday, December 11, 12

    View Slide

  112. You can deploy
    whenever you want
    the revisions at which
    the CI was passed
    Tuesday, December 11, 12

    View Slide

  113. CI build should be fast
    Tuesday, December 11, 12

    View Slide

  114. git push git push
    CI
    git
    build #k build #k+1
    Tuesday, December 11, 12

    View Slide

  115. git push
    10min
    git push
    CI
    git
    build #k build #k+1
    Tuesday, December 11, 12

    View Slide

  116. git push
    10min 10min
    git push
    CI
    git
    build #k build #k+1
    Tuesday, December 11, 12

    View Slide

  117. git push
    10min 10min
    git push
    CI
    git
    This commit can be deployed after 20min
    build #k build #k+1
    Tuesday, December 11, 12

    View Slide

  118. git push
    build #k build #k+1
    N min N min
    git push
    CI
    git
    This commit can be deployed after 2N min
    Tuesday, December 11, 12

    View Slide

  119. The maximum CI waiting minutes 2N min
    Prescribed working time
    8 hours
    == 480 min
    The maximum count of deploy
    480 / 2N
    == 240 / N
    N : The CI waiting time to deploy 40 times
    240 / 40
    == 6 min
    Tuesday, December 11, 12

    View Slide

  120. The maximum CI waiting minutes 2N min
    Prescribed working time
    8 hours
    == 480 min
    The maximum count of deploy
    480 / 2N
    == 240 / N
    N : The CI waiting time to deploy 40 times
    240 / 40
    == 6 min
    Tuesday, December 11, 12

    View Slide

  121. The maximum CI waiting minutes 2N min
    Prescribed working time
    8 hours
    == 480 min
    The maximum count of deploy
    480 / 2N
    == 240 / N
    N : The CI waiting time to deploy 40 times
    240 / 40
    == 6 min
    Tuesday, December 11, 12

    View Slide

  122. The maximum CI waiting minutes 2N min
    Prescribed working time
    8 hours
    == 480 min
    The maximum count of deploy
    480 / 2N
    == 240 / N
    N : The CI waiting time to deploy 40 times
    240 / 40
    == 6 min
    Tuesday, December 11, 12

    View Slide

  123. Developers should guarantee
    their changes doesn’t break any tests
    when they pushes the changes
    into the master branch
    Tuesday, December 11, 12

    View Slide

  124. Requirements:
    Tuesday, December 11, 12

    View Slide

  125. Fast CI
    Requirements:
    Tuesday, December 11, 12

    View Slide

  126. Fast CI
    Fast developer test
    Requirements:
    Tuesday, December 11, 12

    View Slide

  127. Fast CI
    Fast developer test
    Accept coincidence of testing
    Requirements:
    Tuesday, December 11, 12

    View Slide

  128. 6000
    6500
    7000
    7500

































    ● ●





































































































    ● ●●




























































































































    ● ●●●

























































































































    ● ●


    ●●





















































































































































    ●●
    ● ●




















































































































    ● ●






















    ●●
































































































    ● ●




































































































    ●●
    ● ●




















































































































    ● ●





























    05/01 06/01 07/01
    Ti
    Examples count
    6000
    6500
    7000
    7500

































    ● ●





































































































    ● ●●




























































































































    ● ●●●

























































































































    ● ●


    ●●





















































































































































    ●●
    ● ●




















































































































    ● ●






















    ●●
































































































    ● ●




































































































    ●●
    ● ●




















































































































    ● ●


































    05/01 06/01 07/01
    Ti
    Examples count
    The number of examples in
    COOKPAD’s spec files
    Tuesday, December 11, 12

    View Slide

  129. ●●●

























































































































    ● ●


    ●●





















































































































































    ●●
    ● ●




















































































































    ● ●






















    ●●
































































































    ● ●




































































































    ●●
    ● ●




















































































































    ● ●











































































































    ● ●





































































































    ●●

    ● ●


























































































    ● ●








































































































    ● ●







































































































    06/01 07/01 08/01
    Timestamp
    6000
    6500
    7000
    7500

































    ● ●





































































































    ● ●●




























































































































    ● ●●●

























































































































    ● ●


    ●●





















































































































































    ●●
    ● ●




















































































































    ● ●






















    ●●
































































































    ● ●




































































































    ●●
    ● ●




















































































































    ● ●


































    05/01 06/01 07/01
    Ti
    Examples count
    The number of examples in
    COOKPAD’s spec files
    Tuesday, December 11, 12

    View Slide




















  130. ● ●





































































































    ●●

    ● ●


























































































    ● ●








































































































    ● ●






































































































    ● ●



































    ●●








































    ● ●


















































































    ●●









    ● ●






































































    ● ●






















































    ●●











    ● ● ●













































































    ● ●






























    08/01 09/01
    estamp
    6000
    6500
    7000
    7500

































    ● ●





































































































    ● ●●




























































































































    ● ●●●

























































































































    ● ●


    ●●





















































































































































    ●●
    ● ●




















































































































    ● ●






















    ●●
































































































    ● ●




































































































    ●●
    ● ●




















































































































    ● ●


































    05/01 06/01 07/01
    Ti
    Examples count
    The number of examples in
    COOKPAD’s spec files
    Tuesday, December 11, 12

    View Slide

  131. Fast CI
    Fast developer test
    Accept coincidence of testing
    Requirements:
    Tuesday, December 11, 12

    View Slide

  132. Fast CI
    Fast developer test
    Accept coincidence of testing
    Requirements:
    Scalable test system
    Tuesday, December 11, 12

    View Slide

  133. Solution:
    Tuesday, December 11, 12

    View Slide

  134. Remote Spec
    Distributed CI
    Solution:
    Tuesday, December 11, 12

    View Slide

  135. Mechanism of
    Remote RSpec
    Tuesday, December 11, 12

    View Slide

  136. Three Steps:
    Tuesday, December 11, 12

    View Slide

  137. Three Steps:
    RSync code tree
    Tuesday, December 11, 12

    View Slide

  138. Three Steps:
    RSync code tree
    Prepare middlewares
    Tuesday, December 11, 12

    View Slide

  139. Three Steps:
    RSync code tree
    Prepare middlewares
    Run RSpec remotely
    Tuesday, December 11, 12

    View Slide

  140. Remote RSpec:
    Tuesday, December 11, 12

    View Slide

  141. Remote RSpec:
    Group spec files
    Tuesday, December 11, 12

    View Slide

  142. Remote RSpec:
    Group spec files
    Assign these groups
    to workers
    Tuesday, December 11, 12

    View Slide

  143. Remote RSpec:
    Group spec files
    Assign these groups
    to workers
    Collect results
    Tuesday, December 11, 12

    View Slide

  144. To minimize execution time
    Use running time
    of each spec file
    in the execution log
    Tuesday, December 11, 12

    View Slide

  145. ./spec/models/country_spec.rb:0.239013
    ./spec/integration/pr/event_spec.rb:11.48333
    ./spec/integration/recipes_spec.rb:159.893117
    ./spec/integration/device/info_spec.rb:3.592802
    ./spec/libs/symboliclink_check_spec.rb:2.024438
    ./spec/controllers/advertisement_module_2_spec.rb:27.125839
    ./spec/libs/mail_functions_spec.rb:0.123556
    ./spec/models/generic_recipe_spec.rb:0.716866
    ./spec/libs/ruby_syntax_spec.rb:18.92995
    ./spec/libs/cookpad_logger_spec.rb:0.233645
    ./spec/controllers/device/myfolder_controller_spec.rb:7.037313
    ./spec/integration/trend_keyword_spec.rb:1.224577
    ./spec/models/search_category_recipe_spec.rb:2.82474
    ./spec/controllers/hello_controller_spec.rb:0.846888
    ./spec/controllers/api_controller_spec.rb:0.936871
    ./spec/integration/recipes_hot_notification_spec.rb:27.260698
    ./spec/models/diary_mailer_spec.rb:1.156989
    (snip)
    spec_execution_times.log
    Tuesday, December 11, 12

    View Slide

  146. Condition Execution time
    single rspec too long; unmeasurable
    8-core parallel_tests over 1 hour
    5 workers,
    each have 6-cores
    about 15 min
    Tuesday, December 11, 12

    View Slide

  147. Tuesday, December 11, 12

    View Slide

  148. Tuesday, December 11, 12

    View Slide

  149. $ grep cookpad:spec ci.log
    (11) cookpad:spec --> 0.020000 0.130000 495.010000 (164.655632)
    (07) cookpad:spec --> 0.020000 0.120000 493.490000 (166.317008)
    (03) cookpad:spec --> 0.020000 0.120000 604.060000 (217.101263)
    (12) cookpad:spec --> 0.020000 0.130000 711.730000 (217.371807)
    (10) cookpad:spec --> 0.020000 0.130000 550.990000 (224.883867)
    (08) cookpad:spec --> 0.020000 0.130000 632.330000 (226.134510)
    (04) cookpad:spec --> 0.010000 0.130000 615.030000 (233.176576)
    (06) cookpad:spec --> 0.020000 0.120000 592.900000 (232.820875)
    (01) cookpad:spec --> 0.020000 0.130000 562.510000 (242.002139)
    (13) cookpad:spec --> 0.020000 0.130000 560.200000 (241.834410)
    (09) cookpad:spec --> 0.030000 0.120000 750.040000 (268.914216)
    (00) cookpad:spec --> 0.020000 0.130000 692.820000 (280.803856)
    (05) cookpad:spec --> 0.020000 0.140000 726.640000 (289.625570)
    (02) cookpad:spec --> 0.010000 0.130000 679.850000 (437.498087)
    Tuesday, December 11, 12

    View Slide

  150. Wrong balancing
    Tuesday, December 11, 12

    View Slide

  151. Wrong balancing
    Group before execution
    Tuesday, December 11, 12

    View Slide

  152. Wrong balancing
    Group before execution
    Varying runtime condition
    Tuesday, December 11, 12

    View Slide

  153. Wrong balancing
    Group before execution
    Varying runtime condition
    RSpec implementation
    Tuesday, December 11, 12

    View Slide

  154. Future plan:
    Tuesday, December 11, 12

    View Slide

  155. Future plan:
    Dynamic distribution
    Tuesday, December 11, 12

    View Slide

  156. Future plan:
    Dynamic distribution
    Producer-consumer pattern
    Tuesday, December 11, 12

    View Slide

  157. Future plan:
    Dynamic distribution
    Producer-consumer pattern
    Monkey-patch for RSpec
    Tuesday, December 11, 12

    View Slide

  158. Today’s Summary
    Tuesday, December 11, 12

    View Slide

  159. Idea
    Data Product
    Learn
    Measure
    Build
    Tuesday, December 11, 12

    View Slide

  160. Idea
    Measure
    Ruby
    Rails
    Chanko
    Sara
    RSpec
    Jenkins
    Capistrano
    Unicorn
    Fluentd
    Tuesday, December 11, 12

    View Slide

  161. Service Stability
    Code Quality
    Fast Development
    Chanko
    Tuesday, December 11, 12

    View Slide

  162. Remote Spec
    Distributed CI
    Faster Testing
    Tuesday, December 11, 12

    View Slide

  163. Future plan:
    Dynamic distribution
    Producer-consumer pattern
    Monkey-patch for RSpec
    Tuesday, December 11, 12

    View Slide

  164. Ruby Technologies
    Activating The Development of
    Any Questions or Comments?
    ͷ։ൃΛࢧ͑Δ Ruby ͷٕज़
    Tuesday, December 11, 12

    View Slide