Shift Left - Find Bugs as Early as Possible

Shift Left is development paradigm centered on the fact that problems are cheaper to fix the early they are found. In web development world where fixes can be deployed instantly, you do not want to wait for the nightly build to pass or for the manual regression tests. Let the machined do what they are really good at. You want to have a system that will give you comprehensive status about the code you’ve just committed. Ideally in less than 5 minutes (the average developer attention span). Despite the fact that PHP lacks the comprehensive quality tooling compared to the compiled languages, this talk will show you how to make it up and do even better with custom designed tools.

You’ll learn how to create robust build system for PHP that will give you the edge in this DevOps world to deploy faster and with great quality. It will save you a bunch of time and enable your team to focus on more creative and challenging tasks. The talk is technical, focusing on topics such as CI, Phing, linters, HHVM, static code quality, unit tests, api tests. There will also be practical tips and tricks to building your own custom tools for fast, static analysis of PHP code for problems such as SQL Injections, incidental DB table locks.

This talk is from BGPHP conference http://bgphp.org presented on 26/09/2015

EmanuilSlavov.com, @EmanuilSlavov

Transcript

1. SHIFT LEFT FIND BUGS AS EARLY AS POSSIBLE @EmanuilSlavov emo@komfo.com
2. All organizations face problems
3. The Cost of Bugs
4. The price to fix a bug Planning Development Testing Release € €€ €€€ €€€€
5. Humans
6. Speed
7. DETECT PROBLEMS EARLY!
8. “Never send a human to do a machine’s job.”
9. Swiss Cheese Defense
10. All checks should run after every commit.
11. All checks should complete in 5 minutes.
12. Fast checks run first.
13. Black Box application fully operational Unit Tests in memory execution only Static Analysis code parsing; no execution whatsoever
14. Linter
15. Linter
17. php -l api/models/mobile_push_model.php PHP Parse error: api/models/mobile_push_model.php on line 61 Errors parsing api/models/mobile_push_model.php
18. HHVM
19. HHVM Linter
21. UnknownObjectMethod in file: api/models/mobile_push_model.php, line: 55, problem entry: $pusher->reallyUnsubscribeDevice ($params['user_id'], $params['device_id'], $actions)
22. Analytics mode not supported after HHVM 3.5!
23. PHPMD
24. HHVM PHPMD Linter
25. CYCLOMATIC COMPLEXITY function testPrint() { echo('Hello World'); } Complexity: 1 function testPrint($parameter) { if($parameter) { echo('Hello World'); } } Complexity: 2
26. In theory method complexity should be less and than 10. PHP is dynamic, loosely typed language, so keep it less than 15.
27. 12 Fatalities $1,2 Billion Settlement
28. ”The throttle angle function scored [complexity] over 100 (unmaintainable)” Michael Barr
29. Also keep method size less than 100 lines (ideally less than 50).
30. CUSTOM CODE ANALYSIS
31. HHVM PHPMD PHP Reaper Linter
32. Detect SQL Injection Detection in ADOdb
33. $dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”) $dbConn->GetRow(“SELECT * FROM users WHERE id = ?”, array(‘$user_id’))
34. Those errors can be caught with static code analysis.
35. There was no such tool. So we developed one.
36. github.com/emanuil/php-reaper
38. Detect improper way to do DB transactions in ADOdb
39. $this->db->StartTrans(); $this->db->doStuff(); $this->db->CompleteTrans(); try { } catch(Exception $exception) { $this->db->FailTrans(); $this->db->CompleteTrans(); }
40. UNIT TESTS
41. HHVM PHPMD PHP Reaper Unit Linter
42. [Unit Tests Demo]
43. Execute in memory No external dependencies Easy to test edge cases Write tests on the lowest level possible Fast
44. 100% test coverage is not a guarantee against bugs!
45. API TESTS
46. HHVM PHPMD PHP Reaper Unit API Linter
47. Test the whole deployed system Can turn them in load/performance tests Exercise end to end logic Advantages
48. Unreliable Can’t pinpoint the problem accurately Slow Disadvantages
49. We had more than 600 API test with 3 hours of execution time. Which was a big problem.
50. Before 600 API tests After 600 API tests
51. 3 hours 3 minutes
52. If you want to be high performing organization you need to solve the problem with slow tests.
53. Here is how we did it
54. Dedicated test environment Stub all external dependencies Run parallel, solve concurrency issues Design tests to be fully independent
55. UI TESTS
56. HHVM PHPMD PHP Reaper Unit API UI Linter
57. The only true ‘customer’ tests UI is talking to the right API Can turn them in automated security tests 3rd party attacking proxy - Burp, Zap Advantages
58. Too slow for on commit feedback Consider having dedicated JS tests Too fragile Disadvantages
59. SPEED MATTERS
60. HHVM PHPMD PHP Reaper Unit API UI Linter 45 seconds 15 seconds 3 min 12 min Less than 5 minutes (run on every commit) Too slow
61. Encourages small, fast releases Easy to pinpoint where the problem is No human effort is waisted
62. “If you can’t fix what’s broken, you’ll go insane.”
63. How to start
64. Setup a CI job Add basic checks on every commit All complete in less than 5 minutes Constantly add new checks
65. When you locate a problem, think how you can detect it automatically the next time.
66. QUESTIONS
67. EmanuilSlavov.com @EmanuilSlavov