Practical Static Analysis for Continuous Application Security

Static code analysis tools that attempt determine what code does without actually running the code provide an excellent opportunity to perform lightweight security checks as part of the software development lifecycle. Unfortunately, building generic static analysis tools, especially for security, is a costly, time-consuming effort. As a result very few tools exist and commercial tools are very expensive - if they even support your programming language.

The good news is building targeted static analysis tools for your own environment with rules specific to your needs is much easier! Since static analysis tools can be run at any point in the software development lifecycle, even simple tools enable powerful security assurance when added to continuous integration. This talk will go through straight-forward options for static analysis, from grep to writing rules for existing tools through writing static analysis tools from scratch.

Transcript

1. Practical Static Analysis
   for
   Continuous Application Security
   Justin Collins
   @presidentbeef
   AppSecUSA 2016
   

   View Slide

2. @presidentbeef
   Definitions
   2
   

   View Slide

3. @presidentbeef
   Continuous Security
   Automated, always-on processes for
   detecting potential security vulnerabilities
   3
   

   View Slide

4. @presidentbeef
   Static Analysis
   Information determined about a program
   without actually running the code
   4
   

   View Slide

5. @presidentbeef
   Practical Static Analysis
   Things you can reasonably implement after
   this talk
   5
   

   View Slide

6. @presidentbeef
   Why Static Analysis?
   Fast
   Run anytime
   Easy to automate
   Points directly at suspect code
   6
   

   View Slide

7. @presidentbeef
   Background
   7
   

   View Slide

8. @presidentbeef
   Static analysis security tool
   for Ruby on Rails
   8
   

   View Slide

9. @presidentbeef
   Tool Cycle
   Run Tool Wait Interpret Results Fix Issues
   Repeat
   9
   

   View Slide

10. @presidentbeef
    SADB
    10
    

    View Slide

11. @presidentbeef 11
    

    View Slide

12. @presidentbeef
    Getting Started
    12
    

    View Slide

13. @presidentbeef
    1 - Identify a Problem
    Repeated incidents with the same root cause
    Opt-in security, e.g. in APIs
    Unsafe calls/settings no one should use, ever
    13
    

    View Slide

14. @presidentbeef
    2 - Identify a Solution
    Write a safer library
    Make security opt-out
    Detect unsafe calls/settings/API usage
    14
    

    View Slide

15. @presidentbeef
    3 - Enforce the Solution
    Write tests
    Use static analysis
    Use dynamic analysis
    15
    

    View Slide

16. @presidentbeef
    4 - Automate Enforcement
    Part of continuous integration
    Part of code review
    Standalone, continuously-running process
    Local scripts/hooks
    16
    

    View Slide

17. @presidentbeef
    Automation Strategies
    17
    

    View Slide

18. @presidentbeef
    Continuous Integration
    18
    

    View Slide

19. @presidentbeef
    Code Review
    19
    

    View Slide

20. @presidentbeef
    Deployment Gate
    20
    

    View Slide

21. @presidentbeef
    Separate Process
    21
    

    View Slide

22. @presidentbeef
    Local Tests/Git Hook
    22
    

    View Slide

23. @presidentbeef
    Scenario
    23
    

    View Slide

24. @presidentbeef
    1 - Identify a Problem
    get_survey(survey_id)
    24
    

    View Slide

25. @presidentbeef
    2 - Identify a Solution
    get_survey(survey_id, user_id)
    25
    

    View Slide

26. @presidentbeef
    3 - Enforce the Solution
    Use static analysis to check for use
    26
    

    View Slide

27. @presidentbeef
    Statically Analyzing
    27
    

    View Slide

28. @presidentbeef
    Regular Expressions
    • grep
    • ack
    • ag
    • …or build your own
    28
    

    View Slide

29. @presidentbeef
    grep/ack
    grep -R "get_survey([^,)]\+)" *
    ack "get_survey\([^,)]+\)" --type=python
    29
    

    View Slide

30. @presidentbeef
    Changed
    Files
    grep
    file1
    file2
    file3
    ...
    Desired Flow
    30
    

    View Slide

31. @presidentbeef
    Bash
    git checkout $@ --quiet
    files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2)
    grep "get_survey([^,)]\+)" $files
    31
    

    View Slide

32. @presidentbeef
    git diff --name-status
    A README.md
    M lib/blah.py
    M lib/a.rb
    M lib/version.rb
    D test/new_stuff.yml
    32
    

    View Slide

33. @presidentbeef
    Bash
    git checkout $@ --quiet
    files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2)
    grep "get_survey([^,)]\+)" $files
    33
    

    View Slide

34. @presidentbeef
    Bash
    bash check_stuff.sh 9c2ca25
    34
    

    View Slide

35. @presidentbeef
    Changed
    Files
    Rules
    file1 - warning X
    file2 - warning Y
    file3 - warning X
    ...
    Multiple Rules
    35
    

    View Slide

36. @presidentbeef
    Create a Rule
    class CheckSurvey < Rule
    def run(file_name, code)
    if code =~ /get_survey\([^,)]+\)/
    warn file_name, "get_survey without user ID"
    end
    end
    end
    36
    

    View Slide

37. @presidentbeef
    Base Rule Class
    class Rule
    @rules = []
    @warnings = []
    def self.inherited klass
    @rules << klass
    end
    def self.run_rules files
    files.each do |name|
    code = File.read(name)
    @rules.each do |r|
    r.new.run(name, code)
    end
    end
    end
    def self.warnings
    @warnings
    end
    def warn file_name, msg
    Rule.warnings << [file_name, msg]
    end
    end
    37
    

    View Slide

38. @presidentbeef
    Code to Run It
    system "git checkout #{ARGV[0]}"
    files = `git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2`
    Rule.run_rules(files.split)
    Rule.warnings.each do |file_name, message|
    puts "#{message} in #{file_name}"
    end
    if Rule.warnings.any?
    exit 1
    end
    38
    

    View Slide

39. @presidentbeef
    Ruby
    ruby check_stuff.rb 9c2ca25
    39
    

    View Slide

40. @presidentbeef
    False Positives
    # Remember not to use get_survey(survey_id)!
    function get_survey(node) {...}
    40
    

    View Slide

41. @presidentbeef
    False Negatives
    get_survey(input.split(",")[0])
    41
    

    View Slide

42. @presidentbeef
    Abstract Syntax Trees
    42
    

    View Slide

43. @presidentbeef
    Compilation vs. Static Analysis
    Lexical
    Analysis
    Parse into
    Abstract Syntax Tree
    Input Program Text
    Convert to
    Intermediate Form(s)
    Optimize
    Output Compiled Code
    Analyze!
    Output Finding Report
    Semantic Analysis
    43
    

    View Slide

44. @presidentbeef
    get_survey(survey_id)
    call
    args
    local
    nil "get_survey"
    "survey_id"
    44
    

    View Slide

45. @presidentbeef
    S-Expressions
    (call nil "get_survey" (local "survey_id"))
    get_survey(survey_id)
    45
    

    View Slide

46. @presidentbeef
    Ruby (RubyParser)
    s(:call, nil, :get_survey, s(:call, nil, :survey_id))
    RubyParser.new.parse("get_survey(survey_id)")
    46
    

    View Slide

47. @presidentbeef
    Python (Astroid)
    Module()
    body = [
    Discard()
    value =
    CallFunc()
    func =
    Name(get_survey)
    args = [
    Name(survey_id)
    ]
    starargs =
    kwargs =
    ]
    AstroidBuilder().string_build("get_survey(survey_id)")
    47
    

    View Slide

48. @presidentbeef
    JavaScript (Esprima)
    {
    "type": "Program",
    "body": [
    {
    "type": "ExpressionStatement",
    "expression": {
    "type": "CallExpression",
    "callee": {
    "type": "Identifier",
    "name": "get_survey"
    },
    "arguments": [
    {
    "type": "Identifier",
    "name": "survey_id"
    }
    ]
    }
    }
    ],
    "sourceType": "script"
    }
    esprima.parse("get_survey(survey_id)")
    48
    

    View Slide

49. @presidentbeef
    Existing Tools
    49
    

    View Slide

50. @presidentbeef
    Bandit
    50
    https://github.com/openstack/bandit
    

    View Slide

51. @presidentbeef
    Bandit Custom Rule
    import bandit
    from bandit.core import test_properties as test
    @test.checks('Call')
    @test.test_id('B350')
    def unsafe_get_survey(context):
    if (context.call_function_name_qual == 'get_survey' and
    context.call_args_count < 2):
    return bandit.Issue(
    severity=bandit.HIGH,
    confidence=bandit.HIGH,
    text="Use of get_survey without user ID."
    )
    51
    

    View Slide

52. @presidentbeef
    Bandit Custom Warning
    52
    

    View Slide

53. @presidentbeef
    brakeman.org
    53
    

    View Slide

54. @presidentbeef
    Brakeman Custom Check
    require 'brakeman/checks/base_check'
    class Brakeman::CheckGetSurvey < Brakeman::BaseCheck
    Brakeman::Checks.add self
    Brakeman::WarningCodes::Codes[:get_survey] = 2001
    @description = "Finds get_survey calls without a user_id"
    def run_check
    @tracker.find_call(target: false, method: :get_survey).each do |result|
    next if duplicate? result
    add_result result
    if result[:call].second_arg.nil?
    warn :result => result,
    :warning_type => "Direct Object Reference",
    :warning_code => :get_survey,
    :message => "Use of get_survey without user_id",
    :confidence => CONFIDENCE[:high]
    end
    end
    end
    end
    54
    

    View Slide

55. @presidentbeef
    Brakeman Custom Check
    brakeman --add-checks-path custom_checks/
    55
    

    View Slide

56. @presidentbeef
    Brakeman Custom Warning
    56
    

    View Slide

57. @presidentbeef
    Custom Tools
    57
    

    View Slide

58. @presidentbeef
    Esprima
    esprima.org
    58
    

    View Slide

59. @presidentbeef
    JavaScript (Esprima)
    {
    "type": "Program",
    "body": [
    {
    "type": "ExpressionStatement",
    "expression": {
    "type": "CallExpression",
    "callee": {
    "type": "Identifier",
    "name": "get_survey"
    },
    "arguments": [
    {
    "type": "Identifier",
    "name": "survey_id"
    }
    ]
    }
    }
    ],
    "sourceType": "script"
    }
    esprima.parse("get_survey(survey_id)")
    59
    

    View Slide

60. @presidentbeef
    Walking Esprima AST
    var check_get_survey = function(ast) {
    if(ast.type == "CallExpression" &&
    ast.callee.type == "Identifier" &&
    ast.callee.name == "get_survey" &&
    ast.arguments.length == 1) {
    console.log("get_survey called without user_id at line ",
    ast.loc.start)
    }
    }
    var walk = function(ast) {
    if(Array.isArray(ast))
    {
    ast.forEach(walk);
    }
    else if(ast.type) {
    check_get_survey(ast)
    for (key in ast) {
    walk(ast[key])
    }
    }
    }
    var esprima = require('esprima');
    walk(esprima.parse('get_survey(survey_id)', { loc: true }))
    60
    

    View Slide

61. @presidentbeef
    RubyParser
    github.com/seattlerb/ruby_parser
    61
    

    View Slide

62. @presidentbeef
    Ruby (RubyParser)
    s(:call, nil, :get_survey, s(:call, nil, :survey_id))
    RubyParser.new.parse("get_survey(survey_id")
    62
    

    View Slide

63. @presidentbeef
    Walking RubyParser AST
    require 'ruby_parser'
    require 'sexp_processor'
    class FindGetSurvey < SexpInterpreter
    def process_call exp
    if exp[1].nil? and
    exp[2] == :get_survey and
    exp[4].nil?
    puts "get_survey called without user_id at line #{exp.line}"
    end
    end
    end
    ast = RubyParser.new.parse "get_survey(survey_id)"
    FindGetSurvey.new.process ast
    63
    

    View Slide

64. @presidentbeef
    Summary
    Start small: identify single issue to solve
    Tailor solution to your environment
    Automate enforcement
    64
    

    View Slide

65. @presidentbeef
    Thank you
    65
    @PresidentBeef / presidentbeef.com
    @Brakeman / brakeman.org
    @BrakemanPro / brakemanpro.com
    

    View Slide