ExplorTechToronto20191016.pdf

Transcript

1. Welcome to ExploreTech-Toronto October 16, 2019 @ExploreTechTo
2. Ania Dileshni Ashley Sean Cinzia Alex Iris ExploreTech Volunteers
3. Explore and celebrate diversity in the Toronto tech community
5. Tonight’s Agenda: 7:00 pm - Tommy: What I’ve been doing to Improve Developer Productivity 7:30 pm – Break 7:40 pm - Mary: Be Brave and Start Small: Lessons in Running an App on GKE 8:00 pm - Chat & mingle 8:30 pm - Doors close
6. What I’ve been doing to improve Developer Productivity ExplorerTechToronto 16th October 2019
7. AGENDA • What I talk today? • Case Studies • Company 1 • CASE1: TDD introduction • Company 2 • CASE2: Workflow Automation • CASE3: Workflow Improvement1 • CASE4: Pay back Technical Depts • Company 3 • CASE5: Workflow Improvement2 • What’s next • Conclusion
8. What I talk today?
9. What I talk today? • Bullets of experience I’ve ever had
10. What I talk today? • Bullets of experience I’ve ever had • Not about just a specific tool introduction or something
11. What I talk today? • Bullets of experience I’ve ever had • Not about just a specific tool introduction or something • Whatever I did to improve team(my) performance
12. What I talk today? • Bullets of experience I’ve ever had • Not about just a specific tool introduction or something • Whatever I did to improve team(my) performance • As a Software Developer!!(Not DevOps)
13. Case Study
14. Company 1 • The number of engineers • ~ 20 • Product details • PHP ~5.4 • Their own framework • SaaS
15. CASE1: TDD Introduction
16. CASE1: TDD Introduction • Situation
17. CASE1: TDD Introduction • Situation • 10 years old PHP product
18. CASE1: TDD Introduction • Situation • 10 years old PHP product • No test suites
19. CASE1: TDD Introduction • Situation • 10 years old PHP product • No test suites • Development Efficiency got slower eventually • A lot of bug whenever we change something • No one knows entire system spec
20. CASE1: TDD Introduction • Situation • 10 years old PHP product • No test suites • Development Efficiency got slower eventually • A lot of bug whenever we change something • No one knows entire system spec Decided to introduce TDD workflow
21. CASE1: TDD Introduction • TDD? • Test Driven Development • Development Process to write automated(unit) tests • It enables to refactor the existing features, add new features easily and safely
22. CASE1: TDD Introduction • Technical Difficulty to introduce TDD
23. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test
24. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test • Define config as CONST 001/config.php 002/config.php
25. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test • Exit when something happens
26. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test • These make difficult to execute test suites as a single process
27. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test • These make difficult to execute test suites as a single process • What I can do to overcome the problem?
28. CASE1: TDD Introduction • Technical Difficulty to introduce TDD • Inappropriate design for unit test • These make difficult to execute test suites as a single process • What I can do to overcome the problem? Execute php as an another process within test process
29. CASE1: TDD Introduction
30. CASE1: TDD Introduction • Execute php as an another process within test process
31. CASE1: TDD Introduction • Execute php as an another process within test process
32. CASE1: TDD Introduction • Execute php as an another process within test process
33. CASE1: TDD Introduction • Execute php as an another process within test process
34. CASE1: TDD Introduction • Execute php as an another process within test process
35. CASE1: TDD Introduction • Execute php as an another process within test process • Internally executes php-cgi and returns HttpResponse Object as a result Command and response example
36. CASE1: TDD Introduction • Functional Tester • https://github.com/kazu9su/functional-tester • https://packagist.org/packages/kazu9su/functional- tester
37. CASE1: TDD Introduction • Summary • It’s difficult to keep productivity without tests. • It’s better to introduce it as soon as possible • It’s not impossible to introduce it after spending several years without tests • But of course it becomes difficult eventually
38. Company 2 • The number of engineers • ~ 15 • Product details • Ruby 2.4 • Ruby on Rails 4.2 • Has test suites & Continuous Integration workflow♥
39. CASE2: Workflow Automation
40. CASE2: Workflow Automation • Situation
41. CASE2: Workflow Automation • Situation • Distributed monolith of RoR
42. CASE2: Workflow Automation • Situation • Distributed monolith of RoR • Shared rails-engine as a gem • models • services
43. CASE2: Workflow Automation • Situation • Distributed monolith of RoR • Shared rails-engine as a gem • models • services • We’d like to automate the deploy flow of the gem
44. CASE2: Workflow Automation • Situation • Gem?
45. CASE2: Workflow Automation • Situation • Gem? • The software package of ruby • A gem can download via RubyGems which is the package manager of gems • Easy to manage the software version and dependency
46. • Situation • Application Architecture Image CASE2: Workflow Automation Search Admin Transaction Api iOS Android Backend Ruby on Rails Gem
47. CASE2: Workflow Automation • Situation • Consideration to deploy common module gem
48. CASE2: Workflow Automation • Situation • Consideration to deploy common module gem • decide a tag level • MAJOR/MINOR/PATCH • e.g v1.1.1 • upload it onto GitHub • Referred as a gem from each application
49. CASE2: Workflow Automation
50. CASE2: Workflow Automation • Step 1: Create a template
51. CASE2: Workflow Automation • Step 1: Create a template
52. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ Merge Pull Request(Trigger)
53. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ Merge Pull Request(Trigger) ᶄ Create a tag
54. CASE2: Workflow Automation • Step 2: Automating a workflow ᶃ Merge Pull Request(Trigger) ᶄ Create a tag ᶅ Notify to Slack
55. CASE2: Workflow Automation • Summary • Making templates is useful to automate the workflow • If you type same commands every time, it’s the time to make that workflow automate
56. CASE3: Workflow Improvement1
57. CASE3: Workflow Improvement1 • Situation • Test execution on Circle CI
58. CASE3: Workflow Improvement1 • Situation • Test execution on Circle CI • Circle CI 2
59. CASE3: Workflow Improvement1 • Situation • Test execution on Circle CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long
60. CASE3: Workflow Improvement1 • Situation • Test execution on Circle CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long • very very long……..
61. CASE3: Workflow Improvement1 • Situation • Test execution on Circle CI • Circle CI 2 • The testing time of the longest test suites took over 30 min to complete the entire test suites • It’s very long • very very long…….. • We should parallelize test suites
62. CASE3: Workflow Improvement1
63. CASE3: Workflow Improvement1 • Specify the parallelism on config
64. CASE3: Workflow Improvement1 • Use glob • Output every file specified by `*`
65. CASE3: Workflow Improvement1 • Pipe files to split • Divide files by the number of parallelism e.g) Parallelism: 2
66. CASE3: Workflow Improvement1 • Exclude dependency from tests
67. CASE3: Workflow Improvement1 • Exclude dependency from tests • e.g) Make sure calling Timecop.return after every test suite • Specify commands on rails_helper.rb or something
68. CASE3: Workflow Improvement1 • Compare the result by the number of parallelism
69. CASE3: Workflow Improvement1 • Compare the result by the number of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52
70. CASE3: Workflow Improvement1 • Compare the result by the number of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52 Significant difference
71. CASE3: Workflow Improvement1 • Compare the result by the number of parallelism • 1… over 30min • 2… 22:21 • 3… 22:00 • 4… 16:31 • 6… 15:52 Slightly difference
72. CASE3: Workflow Improvement1 • Compare the result by the number of parallelism • The test time depends on the worst case • After dividing test suites, investigate and improve the worst test suite is the best way to improve the time
73. CASE3: Workflow Improvement1 • Summary • Circle CI parallelizing feature is easy to use • 10 minutes is a threshold as standard • Testing time becomes longer than that, it’s time to consider parallelizing test suites
74. CASE4: Pay back Technical Debts
75. CASE4: Pay back Technical Debts • Technical Debts?
76. CASE4: Pay back Technical Debts • Technical Debts? • Like `interest` of monetary debts, what makes it harder to implement changes later on
77. CASE4: Pay back Technical Debts • Technical Debts? • Like `interest` of monetary debts, what makes it harder to implement changes later on • Intentionally or not, it could be implemented by developers
78. CASE4: Pay back Technical Debts • Situation • Cache Ruby Object instance itself
79. CASE4: Pay back Technical Debts • Situation • Cache Ruby Object instance itself
80. CASE4: Pay back Technical Debts • Situation • Cache Ruby Object instance itself • Implicit serializing
81. CASE4: Pay back Technical Debts • Situation • Cache Ruby Object instance itself • Implicit serializing • Implementation depends on the internal API which could not handle by ourselves
82. CASE4: Pay back Technical Debts • Situation • Cache Ruby Object instance itself • Implicit serializing • Implementation depends on the internal API which could not handle by ourselves • It depends on implementation of standard libraries(frame work)
83. CASE4: Pay back Technical Debts • Why this affects on Productivity?
84. CASE4: Pay back Technical Debts • Why this affects on Productivity? • Cache can be shared within several applications
85. CASE4: Pay back Technical Debts • Why this affects on Productivity? • Cache can be shared within several applications • Cache can be shared only with rails applications
86. CASE4: Pay back Technical Debts • Why this affects on Productivity? • Cache can be shared within several applications • Cache can be shared only with rails applications • We cannot establish consistency if rollback would happen • The API implementation could be different from each Ruby version for instance.
87. CASE4: Pay back Technical Debts • What I can do?
88. CASE4: Pay back Technical Debts • What I can do? • Update the data format which has backward compatibility • e.g) JSON
89. CASE4: Pay back Technical Debts • What I can do? • Update the data format which has backward compatibility • e.g) JSON
90. CASE4: Pay back Technical Debts • Is it easy to replace the data format on prod?
91. CASE4: Pay back Technical Debts • Is it easy to replace the data format on prod? • I can’t say yes
92. CASE4: Pay back Technical Debts • Is it easy to replace the data format on prod? • I can’t say yes • Use feature toggles
93. CASE4: Pay back Technical Debts • Feature Toggles?
94. CASE4: Pay back Technical Debts • Feature Toggles? • https://martinfowler.com/articles/feature-toggles.html • "Feature Toggling" is a set of patterns which can help a team to deliver new functionality to users rapidly but safely • It enables to return true by the arbitrary setting • Ship the future safely and turn off the feature after deploying if something goes wrong
95. CASE4: Pay back Technical Debts • Example
96. CASE4: Pay back Technical Debts • Example
97. CASE4: Pay back Technical Debts • Example
98. CASE4: Pay back Technical Debts • Example
99. CASE4: Pay back Technical Debts • Example • Simply use if sentence • The condition to return true depends on implementation
100. CASE4: Pay back Technical Debts • Summary • Technical debts could affect productivity • Continuous Delivery Techniques are useful to pay back technical depts safely
101. Company 3 • The number of engineers • ~ 5 • Product details • Ruby 2.4 • RoR 4.2 • No CI
102. CASE5: Workflow improvement2
103. CASE5: Workflow Improvement2 • Situation
104. CASE5: Workflow Improvement2 • Situation • Deploy manually
105. CASE5: Workflow Improvement2 • Situation • Deploy manually • Communicate each other in person
106. CASE5: Workflow Improvement2 • Situation • Deploy manually • Communicate each other in person • Sometimes it causes miscommunication problems between engineers and testers • What is deployed on staging now?
107. CASE5: Workflow Improvement2
108. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize what happens by using CircleCI workflow
109. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize what happens by using CircleCI workflow ᶃ Merge Pull Request(Trigger) ᶄ Deploy ᶅ Notify to Slack
110. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize what happens by using CircleCI workflow
111. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize what happens by using CircleCI workflow • Automatically send notification to slack after complete a workflow
112. CASE5: Workflow Improvement2 • Automate the deploy workflow and visualize what happens by using CircleCI workflow • If need `Approval` step in workflow, just set some parameters as setting
113. CASE5: Workflow Improvement2 • Summary • We can use CircleCI just as CD(Continuous Delivery) workflow tool • More details • Start deploy automation via CircleCI workflow • https://medium.com/@kazu9su/start-deploy- automation-via-circleci-workflow-3bc0431b5eee
114. What I’ve done until now
115. What’s next?
116. Tommy (Kazushige Tominaga) RewardOps (from 30th Sep 2019) Software Developer @tooooooooomy @kazu9su Came to Toronto this January from Japan
118. • The number of engineers • ~ 15 • Product details • Ruby 2.4 • RoR 4.2 • Has test suites & CI♥ • Has DevOps team
120. Conclusion
121. Conclusion • There are bunch of stuff to improve Our productivity • I keep working to improve myself and my team! • You can do this too! • References • https://github.com/kazu9su/functional-tester • https://martinfowler.com/articles/feature-toggles.html • https://medium.com/@kazu9su/start-deploy- automation-via-circleci-workflow-3bc0431b5eee
122. Thanks for listening!