Automation Testing of Legacy Applications

Everyone loves to work on a greenfield project. You can choose language, architecture, design from scratch, all the new, shiny stuff. The reality is that software engineers at some point need to work on or support legacy applications. It’s alive because it has some business value. The code is so messed up (“I just need to add one more ‘if’ to this method”), and the whole thing is held together by spit and baling wire. There are no automated tests safety net. Every change requires manual tests and a prayer to the software gods.

This talk will show you how to start small and work your way up to the point where you reach a confident state. It will show you how to optimize for the team happiness. It will affect topics such as, unit testing, acceptance tests, static code analysis, continuous integration, architecture for testability. The talk is inspired by real life experience, working on three legacy projects in the span of more than five years.

Transcript

1. AUTOMATION TESTING
   LEGACY APPLICATIONS
   @EmanuilSlavov
   OF
   

   View Slide

2. Your application is legacy,
   if you don’t have automated tests.
   

   View Slide

3. GREENFIELD PROJECT
   

   View Slide

4. BROWNFIELD PROJECT
   BROWNFIELD PROJECT
   

   View Slide

5. WHY INVEST IN LEGACY
   SYSTEM?
   TEAM HAPPINESS
   

   View Slide

6. LOW TEAM MORALE
   

   View Slide

7. Fragile Software
   Slow Feedback
   Stupid Errors
   Repetitive Work
   

   View Slide

8. Quality software is team effort.
   

   View Slide

9. SHIFT LEFT
   

   View Slide

10. WHAT TO DO ABOUT IT
    

    View Slide

11. Start with basic acceptance tests
    

    View Slide

12. Functionality that makes money
    Must have functionality - compliance, security
    Repeating Manual Tests - Save Time
    Pareto Principle - 80/20
    

    View Slide

13. Do not test through the UI.
    (if possible)
    

    View Slide

14. View Slide

15. result = RestClient.post(
    REGISTER_URL,
    user_details.to_json,
    {:content_type => :json}
    )
    

    View Slide

16. 800 test x 10 seconds = 2h 13min
    This saved us:
    

    View Slide

17. Limit external dependencies calls.
    

    View Slide

18. Need to Call
    External System
    Comes from
    automated
    test?
    Talk to the
    real system
    No
    Fake the
    response
    Yes
    

    View Slide

19. Test should create the data they need.
    

    View Slide

20. Scenario: Client admin should not be able to
    view master’s agencies
    Given а master user
    And master creates agency
    And a client admin
    When client admin views master's agency
    Then client admin should get an error
    

    View Slide

21. Set test data via API or DB.
    

    View Slide

22. Poll for results from API/DB operations.
    

    View Slide

23. sleeping(1).seconds.between_tries.failing_after(10).tries do
    result = some_operation
    raise 'No Data' if result == []
    end
    

    View Slide

24. Run a test 20 times consecutively.
    Commit only if the test does not fail.
    

    View Slide

25. for i in {1..20}; do your_test; done
    

    View Slide

26. Make async tasks sync
    

    View Slide

27. CODE CHANGES
    

    View Slide

28. First Order of Business:
    Remove Unused Code
    

    View Slide

29. KNIGHT CAPITAL
    MELTDOWN
    LOST $440 MILLION
    IN 45 MINUTES
    

    View Slide

30. Second Order of Business:
    Stop The Rot
    

    View Slide

31. View Slide

32. CONTINUOUS
    INTEGRATION
    

    View Slide

33. View Slide

34. Run on every commit
    Fast
    Hook one by one all the checks
    Run longer tests periodically
    

    View Slide

35. Developers need to receive feedback
    about their new code within 5 minutes.
    

    View Slide

36. WHAT CHECKS TO
    RUN ON COMMIT?
    

    View Slide

37. The PHP Case
    

    View Slide

38. View Slide

39. php -l api/models/mobile_push_model.php
    PHP Parse error: api/models/mobile_push_model.php on line 61
    Errors parsing api/models/mobile_push_model.php
    Linter
    

    View Slide

40. View Slide

41. UnknownObjectMethod in file:
    api/models/mobile_push_model.php, line: 55, problem entry:
    $pusher->reallyUnsubscribeDevice
    ($params['user_id'], $params['device_id'], $actions)
    HHVM
    

    View Slide

42. STATIC CODE QUALITY
    

    View Slide

43. CYCLOMATIC COMPLEXITY
    function testPrint() {
    echo('Hello World');
    }
    Complexity: 1
    function testPrint($parameter) {
    if($parameter) {
    echo('Hello World');
    }
    }
    Complexity: 2
    

    View Slide

44. Method complexity
    should be less than 10.
    

    View Slide

45. 12 Fatalities
    $1,2 Billion Settlement
    

    View Slide

46. ”The throttle angle function scored
    [complexity] over 100 (unmaintainable)”
    Michael Barr
    

    View Slide

47. Complexity 82
    Complexity 10
    Constantly refactor
    to decrease complexity
    

    View Slide

48. Method size should be less than
    100 lines (ideally less than 50).
    

    View Slide

49. Improve the code - then lower the
    threshold on commit check.
    Then repeat.
    

    View Slide

50. FIGHT LEGACY CODE
    WRITE UNIT TESTS
    

    View Slide

51. Written by Developers
    Fast, Independent
    Test Technical Aspects
    Cooperation between QA & Developers
    

    View Slide

52. [Demo]
    

    View Slide

53. 100% test coverage is not sufficient!
    

    View Slide

54. View Slide

55. SECURITY TESTS
    

    View Slide

56. SQL Injection Detection
    (PHP and ADOdb)
    $dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”)
    $dbConn->GetRow(“SELECT * FROM users WHERE id = ?”, array(‘$user_id’))
    

    View Slide

57. Those errors can be caught with code
    analysis.
    

    View Slide

58. There was no such tool.
    So we developed one.
    

    View Slide

59. github.com/emanuil/php-reaper
    

    View Slide

60. View Slide

61. MONITORING
    

    View Slide

62. Your second line of defense.
    

    View Slide

63. Show a lot with TV and Raspberry Pi.
    

    View Slide

64. View Slide

65. Live Graphs + Deploys
    

    View Slide

66. CONCLUSION
    

    View Slide

67. Аutomatе the most important functionalities
    Continuously improve static code quality
    Write unit tests for changed/new code
    Expand checks on commit
    Enable monitoring
    

    View Slide

68. View Slide

69. @EmanuilSlavov
    EmanuilSlavov.com
    

    View Slide