Shift Left - Find Bugs as Early as Possible

Shift Left is development paradigm centered on the fact that problems are cheaper to fix the early they are found. In web development world where fixes can be deployed instantly, you do not want to wait for the nightly build to pass or for the manual regression tests. Let the machined do what they are really good at. You want to have a system that will give you comprehensive status about the code you’ve just committed. Ideally in less than 5 minutes (the average developer attention span). Despite the fact that PHP lacks the comprehensive quality tooling compared to the compiled languages, this talk will show you how to make it up and do even better with custom designed tools.

You’ll learn how to create robust build system for PHP that will give you the edge in this DevOps world to deploy faster and with great quality. It will save you a bunch of time and enable your team to focus on more creative and challenging tasks. The talk is technical, focusing on topics such as CI, Phing, linters, HHVM, static code quality, unit tests, api tests. There will also be practical tips and tricks to building your own custom tools for fast, static analysis of PHP code for problems such as SQL Injections, incidental DB table locks.

This talk is from BGPHP conference http://bgphp.org presented on 26/09/2015

EmanuilSlavov.com, @EmanuilSlavov

Transcript

1. SHIFT LEFT
   FIND BUGS AS EARLY AS POSSIBLE
   @EmanuilSlavov
   emo@komfo.com
   

   View Slide

2. All organizations
   face problems
   

   View Slide

3. The Cost of Bugs
   

   View Slide

4. The price to fix a bug
   Planning Development Testing Release
   €
   €€
   €€€
   €€€€
   

   View Slide

5. Humans
   

   View Slide

6. Speed
   

   View Slide

7. DETECT
   PROBLEMS
   EARLY!
   

   View Slide

8. “Never send a human to
   do a machine’s job.”
   

   View Slide

9. Swiss Cheese Defense
   

   View Slide

10. All checks should run after
    every commit.
    

    View Slide

11. All checks should complete in
    5 minutes.
    

    View Slide

12. Fast checks run first.
    

    View Slide

13. Black Box
    application fully operational
    Unit Tests
    in memory execution only
    Static Analysis
    code parsing; no execution whatsoever
    

    View Slide

14. Linter
    

    View Slide

15. Linter
    

    View Slide

16. View Slide

17. php -l api/models/mobile_push_model.php
    PHP Parse error: api/models/mobile_push_model.php on line 61
    Errors parsing api/models/mobile_push_model.php
    

    View Slide

18. HHVM
    

    View Slide

19. HHVM
    Linter
    

    View Slide

20. View Slide

21. UnknownObjectMethod in file:
    api/models/mobile_push_model.php, line: 55, problem entry:
    $pusher->reallyUnsubscribeDevice
    ($params['user_id'], $params['device_id'], $actions)
    

    View Slide

22. Analytics mode not supported
    after HHVM 3.5!
    

    View Slide

23. PHPMD
    

    View Slide

24. HHVM PHPMD
    Linter
    

    View Slide

25. CYCLOMATIC COMPLEXITY
    function testPrint() {
    echo('Hello World');
    }
    Complexity: 1
    function testPrint($parameter) {
    if($parameter) {
    echo('Hello World');
    }
    }
    Complexity: 2
    

    View Slide

26. In theory method complexity should be
    less and than 10.
    PHP is dynamic, loosely typed language,
    so keep it less than 15.
    

    View Slide

27. 12 Fatalities
    $1,2 Billion Settlement
    

    View Slide

28. ”The throttle angle function scored
    [complexity] over 100 (unmaintainable)”
    Michael Barr
    

    View Slide

29. Also keep method size less than
    100 lines (ideally less than 50).
    

    View Slide

30. CUSTOM CODE ANALYSIS
    

    View Slide

31. HHVM PHPMD PHP Reaper
    Linter
    

    View Slide

32. Detect SQL Injection Detection
    in ADOdb
    

    View Slide

33. $dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”)
    $dbConn->GetRow(“SELECT * FROM users WHERE id = ?”, array(‘$user_id’))
    

    View Slide

34. Those errors can be caught with
    static code analysis.
    

    View Slide

35. There was no such tool.
    So we developed one.
    

    View Slide

36. github.com/emanuil/php-reaper
    

    View Slide

37. View Slide

38. Detect improper way to do
    DB transactions in ADOdb
    

    View Slide

39. $this->db->StartTrans();
    $this->db->doStuff();
    $this->db->CompleteTrans();
    try {
    } catch(Exception $exception) {
    $this->db->FailTrans();
    $this->db->CompleteTrans();
    }
    

    View Slide

40. UNIT TESTS
    

    View Slide

41. HHVM PHPMD PHP Reaper Unit
    Linter
    

    View Slide

42. [Unit Tests Demo]
    

    View Slide

43. Execute in memory
    No external dependencies
    Easy to test edge cases
    Write tests on the lowest level possible
    Fast
    

    View Slide

44. 100% test coverage
    is not a guarantee
    against bugs!
    

    View Slide

45. API TESTS
    

    View Slide

46. HHVM PHPMD PHP Reaper Unit API
    Linter
    

    View Slide

47. Test the whole deployed system
    Can turn them in load/performance tests
    Exercise end to end logic
    Advantages
    

    View Slide

48. Unreliable
    Can’t pinpoint the problem accurately
    Slow
    Disadvantages
    

    View Slide

49. We had more than 600 API test with
    3 hours of execution time.
    Which was a big problem.
    

    View Slide

50. Before
    600 API tests
    After
    600 API tests
    

    View Slide

51. 3
    hours
    3
    minutes
    

    View Slide

52. If you want to be high performing
    organization you need to solve
    the problem with slow tests.
    

    View Slide

53. Here is how we did it
    

    View Slide

54. Dedicated test environment
    Stub all external dependencies
    Run parallel, solve concurrency issues
    Design tests to be fully independent
    

    View Slide

55. UI TESTS
    

    View Slide

56. HHVM PHPMD PHP Reaper Unit API UI
    Linter
    

    View Slide

57. The only true ‘customer’ tests
    UI is talking to the right API
    Can turn them in automated security tests
    3rd party attacking proxy - Burp, Zap
    Advantages
    

    View Slide

58. Too slow for on commit feedback
    Consider having dedicated JS tests
    Too fragile
    Disadvantages
    

    View Slide

59. SPEED
    MATTERS
    

    View Slide

60. HHVM PHPMD PHP Reaper Unit API UI
    Linter
    45 seconds 15 seconds 3 min 12 min
    Less than 5 minutes
    (run on every commit)
    Too slow
    

    View Slide

61. Encourages small, fast releases
    Easy to pinpoint where the problem is
    No human effort is waisted
    

    View Slide

62. “If you can’t fix what’s broken,
    you’ll go insane.”
    

    View Slide

63. How to start
    

    View Slide

64. Setup a CI job
    Add basic checks on every commit
    All complete in less than 5 minutes
    Constantly add new checks
    

    View Slide

65. When you locate a problem,
    think how you can detect it
    automatically the next time.
    

    View Slide

66. QUESTIONS
    

    View Slide

67. EmanuilSlavov.com
    @EmanuilSlavov
    

    View Slide