Shift Left - Find Bugs as Early as Possible

1f686da361195e15bb4e478397a4fc8f?s=47 emanuil
September 26, 2015

Shift Left - Find Bugs as Early as Possible

Shift Left is development paradigm centered on the fact that problems are cheaper to fix the early they are found. In web development world where fixes can be deployed instantly, you do not want to wait for the nightly build to pass or for the manual regression tests. Let the machined do what they are really good at. You want to have a system that will give you comprehensive status about the code you’ve just committed. Ideally in less than 5 minutes (the average developer attention span). Despite the fact that PHP lacks the comprehensive quality tooling compared to the compiled languages, this talk will show you how to make it up and do even better with custom designed tools.

You’ll learn how to create robust build system for PHP that will give you the edge in this DevOps world to deploy faster and with great quality. It will save you a bunch of time and enable your team to focus on more creative and challenging tasks. The talk is technical, focusing on topics such as CI, Phing, linters, HHVM, static code quality, unit tests, api tests. There will also be practical tips and tricks to building your own custom tools for fast, static analysis of PHP code for problems such as SQL Injections, incidental DB table locks.

This talk is from BGPHP conference presented on 26/09/2015, @EmanuilSlavov



September 26, 2015



  2. All organizations face problems

  3. The Cost of Bugs

  4. The price to fix a bug Planning Development Testing Release

    € €€ €€€ €€€€
  5. Humans

  6. Speed


  8. “Never send a human to do a machine’s job.”

  9. Swiss Cheese Defense

  10. All checks should run after every commit.

  11. All checks should complete in 5 minutes.

  12. Fast checks run first.

  13. Black Box application fully operational Unit Tests in memory execution

    only Static Analysis code parsing; no execution whatsoever
  14. Linter

  15. Linter

  16. None
  17. php -l api/models/mobile_push_model.php PHP Parse error: api/models/mobile_push_model.php on line 61

    Errors parsing api/models/mobile_push_model.php
  18. HHVM

  19. HHVM Linter

  20. None
  21. UnknownObjectMethod in file: api/models/mobile_push_model.php, line: 55, problem entry: $pusher->reallyUnsubscribeDevice ($params['user_id'],

    $params['device_id'], $actions)
  22. Analytics mode not supported after HHVM 3.5!

  23. PHPMD

  24. HHVM PHPMD Linter

  25. CYCLOMATIC COMPLEXITY function testPrint() { echo('Hello World'); } Complexity: 1

    function testPrint($parameter) { if($parameter) { echo('Hello World'); } } Complexity: 2
  26. In theory method complexity should be less and than 10.

    PHP is dynamic, loosely typed language, so keep it less than 15.
  27. 12 Fatalities $1,2 Billion Settlement

  28. ”The throttle angle function scored [complexity] over 100 (unmaintainable)” Michael

  29. Also keep method size less than 100 lines (ideally less

    than 50).

  31. HHVM PHPMD PHP Reaper Linter

  32. Detect SQL Injection Detection in ADOdb

  33. $dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”) $dbConn->GetRow(“SELECT *

    FROM users WHERE id = ?”, array(‘$user_id’))
  34. Those errors can be caught with static code analysis.

  35. There was no such tool. So we developed one.


  37. None
  38. Detect improper way to do DB transactions in ADOdb

  39. $this->db->StartTrans(); $this->db->doStuff(); $this->db->CompleteTrans(); try { } catch(Exception $exception) { $this->db->FailTrans();

    $this->db->CompleteTrans(); }

  41. HHVM PHPMD PHP Reaper Unit Linter

  42. [Unit Tests Demo]

  43. Execute in memory No external dependencies Easy to test edge

    cases Write tests on the lowest level possible Fast
  44. 100% test coverage is not a guarantee against bugs!


  46. HHVM PHPMD PHP Reaper Unit API Linter

  47. Test the whole deployed system Can turn them in load/performance

    tests Exercise end to end logic Advantages
  48. Unreliable Can’t pinpoint the problem accurately Slow Disadvantages

  49. We had more than 600 API test with 3 hours

    of execution time. Which was a big problem.
  50. Before 600 API tests After 600 API tests

  51. 3 hours 3 minutes

  52. If you want to be high performing organization you need

    to solve the problem with slow tests.
  53. Here is how we did it

  54. Dedicated test environment Stub all external dependencies Run parallel, solve

    concurrency issues Design tests to be fully independent
  55. UI TESTS

  56. HHVM PHPMD PHP Reaper Unit API UI Linter

  57. The only true ‘customer’ tests UI is talking to the

    right API Can turn them in automated security tests 3rd party attacking proxy - Burp, Zap Advantages
  58. Too slow for on commit feedback Consider having dedicated JS

    tests Too fragile Disadvantages

  60. HHVM PHPMD PHP Reaper Unit API UI Linter 45 seconds

    15 seconds 3 min 12 min Less than 5 minutes (run on every commit) Too slow
  61. Encourages small, fast releases Easy to pinpoint where the problem

    is No human effort is waisted
  62. “If you can’t fix what’s broken, you’ll go insane.”

  63. How to start

  64. Setup a CI job Add basic checks on every commit

    All complete in less than 5 minutes Constantly add new checks
  65. When you locate a problem, think how you can detect

    it automatically the next time.

  67. @EmanuilSlavov