Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shift Left - Find Bugs as Early as Possible

emanuil
September 26, 2015

Shift Left - Find Bugs as Early as Possible

Shift Left is development paradigm centered on the fact that problems are cheaper to fix the early they are found. In web development world where fixes can be deployed instantly, you do not want to wait for the nightly build to pass or for the manual regression tests. Let the machined do what they are really good at. You want to have a system that will give you comprehensive status about the code you’ve just committed. Ideally in less than 5 minutes (the average developer attention span). Despite the fact that PHP lacks the comprehensive quality tooling compared to the compiled languages, this talk will show you how to make it up and do even better with custom designed tools.

You’ll learn how to create robust build system for PHP that will give you the edge in this DevOps world to deploy faster and with great quality. It will save you a bunch of time and enable your team to focus on more creative and challenging tasks. The talk is technical, focusing on topics such as CI, Phing, linters, HHVM, static code quality, unit tests, api tests. There will also be practical tips and tricks to building your own custom tools for fast, static analysis of PHP code for problems such as SQL Injections, incidental DB table locks.

This talk is from BGPHP conference http://bgphp.org presented on 26/09/2015

EmanuilSlavov.com, @EmanuilSlavov

emanuil

September 26, 2015
Tweet

More Decks by emanuil

Other Decks in Programming

Transcript

  1. SHIFT LEFT
    FIND BUGS AS EARLY AS POSSIBLE
    @EmanuilSlavov
    [email protected]

    View full-size slide

  2. All organizations
    face problems

    View full-size slide

  3. The Cost of Bugs

    View full-size slide

  4. The price to fix a bug
    Planning Development Testing Release

    €€
    €€€
    €€€€

    View full-size slide

  5. DETECT
    PROBLEMS
    EARLY!

    View full-size slide

  6. “Never send a human to
    do a machine’s job.”

    View full-size slide

  7. Swiss Cheese Defense

    View full-size slide

  8. All checks should run after
    every commit.

    View full-size slide

  9. All checks should complete in
    5 minutes.

    View full-size slide

  10. Fast checks run first.

    View full-size slide

  11. Black Box
    application fully operational
    Unit Tests
    in memory execution only
    Static Analysis
    code parsing; no execution whatsoever

    View full-size slide

  12. php -l api/models/mobile_push_model.php
    PHP Parse error: api/models/mobile_push_model.php on line 61
    Errors parsing api/models/mobile_push_model.php

    View full-size slide

  13. UnknownObjectMethod in file:
    api/models/mobile_push_model.php, line: 55, problem entry:
    $pusher->reallyUnsubscribeDevice
    ($params['user_id'], $params['device_id'], $actions)

    View full-size slide

  14. Analytics mode not supported
    after HHVM 3.5!

    View full-size slide

  15. HHVM PHPMD
    Linter

    View full-size slide

  16. CYCLOMATIC COMPLEXITY
    function testPrint() {
    echo('Hello World');
    }
    Complexity: 1
    function testPrint($parameter) {
    if($parameter) {
    echo('Hello World');
    }
    }
    Complexity: 2

    View full-size slide

  17. In theory method complexity should be
    less and than 10.
    PHP is dynamic, loosely typed language,
    so keep it less than 15.

    View full-size slide

  18. 12 Fatalities
    $1,2 Billion Settlement

    View full-size slide

  19. ”The throttle angle function scored
    [complexity] over 100 (unmaintainable)”
    Michael Barr

    View full-size slide

  20. Also keep method size less than
    100 lines (ideally less than 50).

    View full-size slide

  21. CUSTOM CODE ANALYSIS

    View full-size slide

  22. HHVM PHPMD PHP Reaper
    Linter

    View full-size slide

  23. Detect SQL Injection Detection
    in ADOdb

    View full-size slide

  24. $dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”)
    $dbConn->GetRow(“SELECT * FROM users WHERE id = ?”, array(‘$user_id’))

    View full-size slide

  25. Those errors can be caught with
    static code analysis.

    View full-size slide

  26. There was no such tool.
    So we developed one.

    View full-size slide

  27. github.com/emanuil/php-reaper

    View full-size slide

  28. Detect improper way to do
    DB transactions in ADOdb

    View full-size slide

  29. $this->db->StartTrans();
    $this->db->doStuff();
    $this->db->CompleteTrans();
    try {
    } catch(Exception $exception) {
    $this->db->FailTrans();
    $this->db->CompleteTrans();
    }

    View full-size slide

  30. HHVM PHPMD PHP Reaper Unit
    Linter

    View full-size slide

  31. [Unit Tests Demo]

    View full-size slide

  32. Execute in memory
    No external dependencies
    Easy to test edge cases
    Write tests on the lowest level possible
    Fast

    View full-size slide

  33. 100% test coverage
    is not a guarantee
    against bugs!

    View full-size slide

  34. HHVM PHPMD PHP Reaper Unit API
    Linter

    View full-size slide

  35. Test the whole deployed system
    Can turn them in load/performance tests
    Exercise end to end logic
    Advantages

    View full-size slide

  36. Unreliable
    Can’t pinpoint the problem accurately
    Slow
    Disadvantages

    View full-size slide

  37. We had more than 600 API test with
    3 hours of execution time.
    Which was a big problem.

    View full-size slide

  38. Before
    600 API tests
    After
    600 API tests

    View full-size slide

  39. 3
    hours
    3
    minutes

    View full-size slide

  40. If you want to be high performing
    organization you need to solve
    the problem with slow tests.

    View full-size slide

  41. Here is how we did it

    View full-size slide

  42. Dedicated test environment
    Stub all external dependencies
    Run parallel, solve concurrency issues
    Design tests to be fully independent

    View full-size slide

  43. HHVM PHPMD PHP Reaper Unit API UI
    Linter

    View full-size slide

  44. The only true ‘customer’ tests
    UI is talking to the right API
    Can turn them in automated security tests
    3rd party attacking proxy - Burp, Zap
    Advantages

    View full-size slide

  45. Too slow for on commit feedback
    Consider having dedicated JS tests
    Too fragile
    Disadvantages

    View full-size slide

  46. SPEED
    MATTERS

    View full-size slide

  47. HHVM PHPMD PHP Reaper Unit API UI
    Linter
    45 seconds 15 seconds 3 min 12 min
    Less than 5 minutes
    (run on every commit)
    Too slow

    View full-size slide

  48. Encourages small, fast releases
    Easy to pinpoint where the problem is
    No human effort is waisted

    View full-size slide

  49. “If you can’t fix what’s broken,
    you’ll go insane.”

    View full-size slide

  50. How to start

    View full-size slide

  51. Setup a CI job
    Add basic checks on every commit
    All complete in less than 5 minutes
    Constantly add new checks

    View full-size slide

  52. When you locate a problem,
    think how you can detect it
    automatically the next time.

    View full-size slide

  53. EmanuilSlavov.com
    @EmanuilSlavov

    View full-size slide