Continuous Security with Practical Static Analysis

Slide 1

Slide 1 text

@presidentbeef Continuous Security with Practical Static Analysis Justin Collins @presidentbeef ^ tweet me questions CodeMash 2016

Slide 2

Slide 2 text

@presidentbeef Definitions 2

Slide 3

Slide 3 text

@presidentbeef Continuous Security Automated, always-on processes for detecting potential security vulnerabilities 3

Slide 4

Slide 4 text

@presidentbeef Static Analysis Information determined about a program without actually running the code 4

Slide 5

Slide 5 text

@presidentbeef Practical Static Analysis Things you can reasonably implement after this talk 5

Slide 6

Slide 6 text

@presidentbeef Why Static Analysis? Fast Run anytime Easy to automate Points directly at suspect code 6

Slide 7

Slide 7 text

@presidentbeef Background 7

Slide 8

Slide 8 text

@presidentbeef Static analysis security tool for Ruby on Rails 8

Slide 9

Slide 9 text

@presidentbeef Tool Cycle Run Tool Wait Interpret Results Fix Issues Repeat 9

Slide 10

Slide 10 text

@presidentbeef SADB 10

Slide 11

Slide 11 text

@presidentbeef 11

Slide 12

Slide 12 text

@presidentbeef Getting Started 12

Slide 13

Slide 13 text

@presidentbeef 1 - Identify a Problem Repeated incidents with the same root cause Opt-in security, e.g. in APIs Unsafe calls/settings no one should use, ever 13

Slide 14

Slide 14 text

@presidentbeef 2 - Identify a Solution Write a safer library Make security opt-out Detect unsafe calls/settings/API usage 14

Slide 15

Slide 15 text

@presidentbeef 3 - Enforce the Solution Write tests Use static analysis Use dynamic analysis 15

Slide 16

Slide 16 text

@presidentbeef 4 - Automate Enforcement Part of continuous integration Part of code review Standalone, continuously-running process Local scripts/hooks 16

Slide 17

Slide 17 text

@presidentbeef Automation Strategies 17

Slide 18

Slide 18 text

@presidentbeef Continuous Integration 18

Slide 19

Slide 19 text

@presidentbeef Code Review 19

Slide 20

Slide 20 text

@presidentbeef Deployment Gate 20

Slide 21

Slide 21 text

@presidentbeef Separate Process 21

Slide 22

Slide 22 text

@presidentbeef Local Script/Git Hook 22

Slide 23

Slide 23 text

@presidentbeef Scenario 23

Slide 24

Slide 24 text

@presidentbeef 1 - Identify a Problem get_survey(survey_id) 24

Slide 25

Slide 25 text

@presidentbeef 2 - Identify a Solution get_survey(survey_id, user_id) 25

Slide 26

Slide 26 text

@presidentbeef 3 - Enforce the Solution Use static analysis to check for use 26

Slide 27

Slide 27 text

@presidentbeef Statically Analyzing 27

Slide 28

Slide 28 text

@presidentbeef Regular Expressions ● grep ● ack ● ag ● …or build your own 28

Slide 29

Slide 29 text

@presidentbeef grep/ack grep -R "get_survey([^,)]\+)" * ack "get_survey$[^,)]+$" --type=python 29

Slide 30

Slide 30 text

@presidentbeef Changed Files grep file1 file2 file3 ... Desired Flow 30

Slide 31

Slide 31 text

@presidentbeef Bash git checkout $@ --quiet files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2) grep "get_survey([^,)]\+)" $files 31

Slide 32

Slide 32 text

@presidentbeef Bash bash check_stuff.sh 9c2ca25 32

Slide 33

Slide 33 text

@presidentbeef Changed Files Rules file1 - warning X file2 - warning Y file3 - warning X ... Multiple Rules 33

Slide 34

Slide 34 text

@presidentbeef Create a Rule class CheckSurvey < Rule def run(file_name, code) if code =~ /get_survey$[^,)]+$/ warn file_name, "get_survey without user ID" end end end 34

Slide 35

Slide 35 text

@presidentbeef Base Rule Class class Rule @rules = [] @warnings = [] def self.inherited klass @rules << klass end def self.run_rules files files.each do |name| code = File.read(name) @rules.each do |r| r.new.run(name, code) end end end def self.warnings @warnings end def warn file_name, msg Rule.warnings << [file_name, msg] end end 35

Slide 36

Slide 36 text

@presidentbeef Code to Run It system "git checkout #{ARGV[0]}" files = `git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2` Rule.run_rules(files.split) Rule.warnings.each do |file_name, message| puts "#{message} in #{file_name}" end if Rule.warnings.any? exit 1 end 36

Slide 37

Slide 37 text

@presidentbeef Ruby ruby check_stuff.rb 9c2ca25 37

Slide 38

Slide 38 text

@presidentbeef False Positives # Remember not to use get_survey(survey_id)! function get_survey(node) {...} 38

Slide 39

Slide 39 text

@presidentbeef False Negatives get_survey(input.split(",")[0]) 39

Slide 40

Slide 40 text

@presidentbeef Abstract Syntax Trees 40

Slide 41

Slide 41 text

@presidentbeef Compilation vs. Static Analysis Lexical Analysis Parse into Abstract Syntax Tree Input Program Text Convert to Intermediate Form(s) Optimize Output Compiled Code Analyze! Output Finding Report Semantic Analysis 41

Slide 42

Slide 42 text

@presidentbeef get_survey(survey_id) call args local nil "get_survey" "survey_id" 42

Slide 43

Slide 43 text

@presidentbeef S-Expressions (call nil "get_survey" (local "survey_id")) get_survey(survey_id) 43

Slide 44

Slide 44 text

@presidentbeef Ruby (RubyParser) s(:call, nil, :get_survey, s(:call, nil, :survey_id)) RubyParser.new.parse("get_survey(survey_id") 44

Slide 45

Slide 45 text

@presidentbeef Python (Astroid) Module() body = [ Discard() value = CallFunc() func = Name(get_survey) args = [ Name(survey_id) ] starargs = kwargs = ] AstroidBuilder().string_build("get_survey(survey_id)") 45

Slide 46

Slide 46 text

@presidentbeef JavaScript (Esprima) { "type": "Program", "body": [ { "type": "ExpressionStatement", "expression": { "type": "CallExpression", "callee": { "type": "Identifier", "name": "get_survey" }, "arguments": [ { "type": "Identifier", "name": "survey_id" } ] } } ], "sourceType": "script" } esprima.parse("get_survey(survey_id)") 46

Slide 47

Slide 47 text

@presidentbeef Existing Tools 47

Slide 48

Slide 48 text

@presidentbeef www.pylint.org 48

Slide 49

Slide 49 text

@presidentbeef Pylint Custom Checker import astroid from pylint.interfaces import IAstroidChecker from pylint.checkers import BaseChecker class GetSurveyChecker(BaseChecker): __implements__ = IAstroidChecker name = 'survey_checker' msgs = {'E9312': ('Call to get_survey without user ID', 'get_survey', 'Calls to get_survey must include user ID.') } priority = -1 def visit_callfunc(self, node): if (isinstance(node.func, astroid.Name) and node.func.name == 'get_survey' and len(node.args) == 1): self.add_message('get_survey', node=node) def register(linter): linter.register_checker(GetSurveyChecker(linter)) 49

Slide 50

Slide 50 text

@presidentbeef Pylint Custom Checker pylint --load-plugins custom my_file.py 50

Slide 51

Slide 51 text

@presidentbeef Pylint Custom Warning ************* Module my_file ... E: 12, 4: Call to get_survey without user ID (get_survey) ... 51

Slide 52

Slide 52 text

@presidentbeef brakeman.org 52

Slide 53

Slide 53 text

@presidentbeef Brakeman Custom Check require 'brakeman/checks/base_check' class Brakeman::CheckGetSurvey < Brakeman::BaseCheck Brakeman::Checks.add self Brakeman::WarningCodes::Codes[:get_survey] = 2001 @description = "Finds get_survey calls without a user_id" def run_check @tracker.find_call(target: false, method: :get_survey).each do |result| next if duplicate? result add_result result if result[:call].second_arg.nil? warn :result => result, :warning_type => "Direct Object Reference", :warning_code => :get_survey, :message => "Use of get_survey without user_id", :confidence => CONFIDENCE[:high] end end end end 53

Slide 54

Slide 54 text

@presidentbeef Brakeman Custom Check brakeman --add-checks-path custom_checks/ 54

Slide 55

Slide 55 text

@presidentbeef Brakeman Custom Warning 55

Slide 56

Slide 56 text

@presidentbeef Custom Tools 56

Slide 57

Slide 57 text

@presidentbeef Esprima esprima.org 57

Slide 58

Slide 58 text

Slide 59

Slide 59 text

@presidentbeef Walking Esprima AST var check_get_survey = function(ast) { if(ast.type == "CallExpression" && ast.callee.type == "Identifier" && ast.callee.name == "get_survey" && ast.arguments.length == 1) { console.log("get_survey called without user_id at line ", ast.loc.start) } } var walk = function(ast) { if(Array.isArray(ast)) { ast.forEach(walk); } else if(ast.type) { check_get_survey(ast) for (key in ast) { walk(ast[key]) } } } var esprima = require('esprima'); walk(esprima.parse('get_survey(survey_id)', { loc: true })) 59

Slide 60

Slide 60 text

@presidentbeef RubyParser github.com/seattlerb/ruby_parser 60

Slide 61

Slide 61 text

@presidentbeef Ruby (RubyParser) s(:call, nil, :get_survey, s(:call, nil, :survey_id)) RubyParser.new.parse("get_survey(survey_id") 61

Slide 62

Slide 62 text

@presidentbeef Walking RubyParser AST require 'ruby_parser' require 'sexp_processor' class FindGetSurvey < SexpInterpreter def process_call exp if exp[1].nil? and exp[2] == :get_survey and exp[4].nil? puts "get_survey called without user_id at line #{exp.line}" end end end ast = RubyParser.new.parse "get_survey(survey_id)" FindGetSurvey.new.process ast 62

Slide 63

Slide 63 text

@presidentbeef Summary Start small: identify single issue to solve Tailor solution to your environment Automate enforcement 63

Slide 64

Slide 64 text

@presidentbeef Go Mash Some Code! @PresidentBeef / presidentbeef.com @Brakeman / brakeman.org @BrakemanPro / brakemanpro.com @MakotoTheCat 64