Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ExplorTechToronto20191016.pdf

 ExplorTechToronto20191016.pdf

D400e6d92f9a69b60d9cfdff1dc9a98e?s=128

Kazushige Tominaga

October 16, 2019
Tweet

More Decks by Kazushige Tominaga

Other Decks in Programming

Transcript

  1. Welcome to ExploreTech-Toronto October 16, 2019 @ExploreTechTo

  2. Ania Dileshni Ashley Sean Cinzia Alex Iris ExploreTech Volunteers

  3. Explore and celebrate diversity in the Toronto tech community

  4. None
  5. Tonight’s Agenda: 7:00 pm - Tommy: What I’ve been doing

    to Improve Developer Productivity 7:30 pm – Break 7:40 pm - Mary: Be Brave and Start Small: Lessons in Running an App on GKE 8:00 pm - Chat & mingle 8:30 pm - Doors close
  6. What I’ve been doing to improve Developer Productivity ExplorerTechToronto 16th

    October 2019
  7. AGENDA • What I talk today? • Case Studies •

    Company 1 • CASE1: TDD introduction • Company 2 • CASE2: Workflow Automation • CASE3: Workflow Improvement1 • CASE4: Pay back Technical Depts • Company 3 • CASE5: Workflow Improvement2 • What’s next • Conclusion
  8. What I talk today?

  9. What I talk today? • Bullets of experience I’ve ever

    had
  10. What I talk today? • Bullets of experience I’ve ever

    had • Not about just a specific tool introduction or something
  11. What I talk today? • Bullets of experience I’ve ever

    had • Not about just a specific tool introduction or something • Whatever I did to improve team(my) performance
  12. What I talk today? • Bullets of experience I’ve ever

    had • Not about just a specific tool introduction or something • Whatever I did to improve team(my) performance • As a Software Developer!!(Not DevOps)
  13. Case Study

  14. Company 1 • The number of engineers • ~ 20

    • Product details • PHP ~5.4 • Their own framework • SaaS
  15. CASE1: TDD Introduction

  16. CASE1: TDD Introduction • Situation

  17. CASE1: TDD Introduction • Situation • 10 years old PHP

    product
  18. CASE1: TDD Introduction • Situation • 10 years old PHP

    product • No test suites
  19. CASE1: TDD Introduction • Situation • 10 years old PHP

    product • No test suites • Development Efficiency got slower eventually • A lot of bug whenever we change something • No one knows entire system spec
  20. CASE1: TDD Introduction • Situation • 10 years old PHP

    product • No test suites • Development Efficiency got slower eventually • A lot of bug whenever we change something • No one knows entire system spec Decided to introduce TDD workflow
  21. CASE1: TDD Introduction • TDD? • Test Driven Development •

    Development Process to write automated(unit) tests • It enables to refactor the existing features, add new features easily and safely
  22. CASE1: TDD Introduction • Technical Difficulty to introduce TDD

  23. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test
  24. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test • Define config as CONST 001/config.php 002/config.php
  25. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test • Exit when something happens
  26. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test • These make difficult to execute test suites as a single process
  27. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test • These make difficult to execute test suites as a single process • What I can do to overcome the problem?
  28. CASE1: TDD Introduction • Technical Difficulty to introduce TDD •

    Inappropriate design for unit test • These make difficult to execute test suites as a single process • What I can do to overcome the problem? Execute php as an another process within test process
  29. CASE1: TDD Introduction

  30. CASE1: TDD Introduction • Execute php as an another process

    within test process
  31. CASE1: TDD Introduction • Execute php as an another process

    within test process
  32. CASE1: TDD Introduction • Execute php as an another process

    within test process
  33. CASE1: TDD Introduction • Execute php as an another process

    within test process
  34. CASE1: TDD Introduction • Execute php as an another process

    within test process
  35. CASE1: TDD Introduction • Execute php as an another process

    within test process • Internally executes php-cgi and returns HttpResponse Object as a result Command and response example
  36. CASE1: TDD Introduction • Functional Tester • https://github.com/kazu9su/functional-tester • https://packagist.org/packages/kazu9su/functional-

    tester
  37. CASE1: TDD Introduction • Summary • It’s difficult to keep

    productivity without tests. • It’s better to introduce it as soon as possible • It’s not impossible to introduce it after spending several years without tests • But of course it becomes difficult eventually
  38. Company 2 • The number of engineers • ~ 15

    • Product details • Ruby 2.4 • Ruby on Rails 4.2 • Has test suites & Continuous Integration workflow♥
  39. CASE2: Workflow Automation

  40. CASE2: Workflow Automation • Situation

  41. CASE2: Workflow Automation • Situation • Distributed monolith of RoR

  42. CASE2: Workflow Automation • Situation • Distributed monolith of RoR

    • Shared rails-engine as a gem • models • services
  43. CASE2: Workflow Automation • Situation • Distributed monolith of RoR

    • Shared rails-engine as a gem • models • services • We’d like to automate the deploy flow of the gem
  44. CASE2: Workflow Automation • Situation • Gem?

  45. CASE2: Workflow Automation • Situation • Gem? • The software

    package of ruby • A gem can download via RubyGems which is the package manager of gems • Easy to manage the software version and dependency
  46. • Situation • Application Architecture Image CASE2: Workflow Automation Search

    Admin Transaction Api iOS Android Backend Ruby on Rails Gem
  47. CASE2: Workflow Automation • Situation • Consideration to deploy common

    module gem
  48. CASE2: Workflow Automation • Situation • Consideration to deploy common

    module gem • decide a tag level • MAJOR/MINOR/PATCH • e.g v1.1.1 • upload it onto GitHub • Referred as a gem from each application
  49. CASE2: Workflow Automation

  50. CASE2: Workflow Automation • Step 1: Create a template

  51. CASE2: Workflow Automation • Step 1: Create a template

  52. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ

    Merge Pull Request(Trigger)
  53. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ

    Merge Pull Request(Trigger) ᶄ Create a tag
  54. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ

    Merge Pull Request(Trigger) ᶄ Create a tag ᶅ Notify to Slack
  55. CASE2: Workflow Automation • Summary • Making templates is useful

    to automate the workflow • If you type same commands every time, it’s the time to make that workflow automate
  56. CASE3: Workflow Improvement1

  57. CASE3: Workflow Improvement1 • Situation • Test execution on Circle

    CI
  58. CASE3: Workflow Improvement1 • Situation • Test execution on Circle

    CI • Circle CI 2
  59. CASE3: Workflow Improvement1 • Situation • Test execution on Circle

    CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long
  60. CASE3: Workflow Improvement1 • Situation • Test execution on Circle

    CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long • very very long……..
  61. CASE3: Workflow Improvement1 • Situation • Test execution on Circle

    CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long • very very long…….. • We should parallelize test suites
  62. CASE3: Workflow Improvement1

  63. CASE3: Workflow Improvement1 • Specify the parallelism on config

  64. CASE3: Workflow Improvement1 • Use glob • Output every file

    specified by `*`
  65. CASE3: Workflow Improvement1 • Pipe files to split • Divide

    files by the number of parallelism e.g) Parallelism: 2
  66. CASE3: Workflow Improvement1 • Exclude dependency from tests

  67. CASE3: Workflow Improvement1 • Exclude dependency from tests • e.g)

    Make sure calling Timecop.return after every test suite • Specify commands on rails_helper.rb or something
  68. CASE3: Workflow Improvement1 • Compare the result by the number

    of parallelism
  69. CASE3: Workflow Improvement1 • Compare the result by the number

    of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52
  70. CASE3: Workflow Improvement1 • Compare the result by the number

    of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52 Significant difference
  71. CASE3: Workflow Improvement1 • Compare the result by the number

    of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52 Slightly difference
  72. CASE3: Workflow Improvement1 • Compare the result by the number

    of parallelism • The test time depends on the worst case • After dividing test suites, investigate and improve the worst test suite is the best way to improve the time
  73. CASE3: Workflow Improvement1 • Summary • Circle CI parallelizing feature

    is easy to use • 10 minutes is a threshold as standard • Testing time becomes longer than that, it’s time to consider parallelizing test suites
  74. CASE4: Pay back Technical Debts

  75. CASE4: Pay back Technical Debts • Technical Debts?

  76. CASE4: Pay back Technical Debts • Technical Debts? • Like

    `interest` of monetary debts, what makes it harder to implement changes later on
  77. CASE4: Pay back Technical Debts • Technical Debts? • Like

    `interest` of monetary debts, what makes it harder to implement changes later on • Intentionally or not, it could be implemented by developers
  78. CASE4: Pay back Technical Debts • Situation • Cache Ruby

    Object instance itself
  79. CASE4: Pay back Technical Debts • Situation • Cache Ruby

    Object instance itself
  80. CASE4: Pay back Technical Debts • Situation • Cache Ruby

    Object instance itself • Implicit serializing
  81. CASE4: Pay back Technical Debts • Situation • Cache Ruby

    Object instance itself • Implicit serializing • Implementation depends on the internal API which could not handle by ourselves
  82. CASE4: Pay back Technical Debts • Situation • Cache Ruby

    Object instance itself • Implicit serializing • Implementation depends on the internal API which could not handle by ourselves • It depends on implementation of standard libraries(frame work)
  83. CASE4: Pay back Technical Debts • Why this affects on

    Productivity?
  84. CASE4: Pay back Technical Debts • Why this affects on

    Productivity? • Cache can be shared within several applications
  85. CASE4: Pay back Technical Debts • Why this affects on

    Productivity? • Cache can be shared within several applications • Cache can be shared only with rails applications
  86. CASE4: Pay back Technical Debts • Why this affects on

    Productivity? • Cache can be shared within several applications • Cache can be shared only with rails applications • We cannot establish consistency if rollback would happen • The API implementation could be different from each Ruby version for instance.
  87. CASE4: Pay back Technical Debts • What I can do?

  88. CASE4: Pay back Technical Debts • What I can do?

    • Update the data format which has backward compatibility • e.g) JSON
  89. CASE4: Pay back Technical Debts • What I can do?

    • Update the data format which has backward compatibility • e.g) JSON
  90. CASE4: Pay back Technical Debts • Is it easy to

    replace the data format on prod?
  91. CASE4: Pay back Technical Debts • Is it easy to

    replace the data format on prod? • I can’t say yes
  92. CASE4: Pay back Technical Debts • Is it easy to

    replace the data format on prod? • I can’t say yes • Use feature toggles
  93. CASE4: Pay back Technical Debts • Feature Toggles?

  94. CASE4: Pay back Technical Debts • Feature Toggles? • https://martinfowler.com/articles/feature-toggles.html

    • "Feature Toggling" is a set of patterns which can help a team to deliver new functionality to users rapidly but safely • It enables to return true by the arbitrary setting • Ship the future safely and turn off the feature after deploying if something goes wrong
  95. CASE4: Pay back Technical Debts • Example

  96. CASE4: Pay back Technical Debts • Example

  97. CASE4: Pay back Technical Debts • Example

  98. CASE4: Pay back Technical Debts • Example

  99. CASE4: Pay back Technical Debts • Example • Simply use

    if sentence • The condition to return true depends on implementation
  100. CASE4: Pay back Technical Debts • Summary • Technical debts

    could affect productivity • Continuous Delivery Techniques are useful to pay back technical depts safely
  101. Company 3 • The number of engineers • ~ 5

    • Product details • Ruby 2.4 • RoR 4.2 • No CI
  102. CASE5: Workflow improvement2

  103. CASE5: Workflow Improvement2 • Situation

  104. CASE5: Workflow Improvement2 • Situation • Deploy manually

  105. CASE5: Workflow Improvement2 • Situation • Deploy manually • Communicate

    each other in person
  106. CASE5: Workflow Improvement2 • Situation • Deploy manually • Communicate

    each other in person • Sometimes it causes miscommunication problems between engineers and testers • What is deployed on staging now?
  107. CASE5: Workflow Improvement2

  108. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize

    what happens by using CircleCI workflow
  109. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize

    what happens by using CircleCI workflow ᶃ Merge Pull Request(Trigger) ᶄ Deploy ᶅ Notify to Slack
  110. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize

    what happens by using CircleCI workflow
  111. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize

    what happens by using CircleCI workflow • Automatically send notification to slack after complete a workflow
  112. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize

    what happens by using CircleCI workflow • If need `Approval` step in workflow, just set some parameters as setting
  113. CASE5: Workflow Improvement2 • Summary • We can use CircleCI

    just as CD(Continuous Delivery) workflow tool • More details • Start deploy automation via CircleCI workflow • https://medium.com/@kazu9su/start-deploy- automation-via-circleci-workflow-3bc0431b5eee
  114. What I’ve done until now

  115. What’s next?

  116. Tommy (Kazushige Tominaga) RewardOps (from 30th Sep 2019) Software Developer

    @tooooooooomy @kazu9su Came to Toronto this January from Japan
  117. None
  118. • The number of engineers • ~ 15 • Product

    details • Ruby 2.4 • RoR 4.2 • Has test suites & CI♥ • Has DevOps team
  119. None
  120. Conclusion

  121. Conclusion • There are bunch of stuff to improve Our

    productivity • I keep working to improve myself and my team! • You can do this too! • References • https://github.com/kazu9su/functional-tester • https://martinfowler.com/articles/feature-toggles.html • https://medium.com/@kazu9su/start-deploy- automation-via-circleci-workflow-3bc0431b5eee
  122. Thanks for listening!