Mashing Up QA and Security - CodeMash 2017

Security is domain specific quality assurance, but developers, testers, and security professionals often don’t work together. When this type of disconnect exists between big groups of people who are very good at their jobs, there is usually a mostly untapped potential for learning. I’ve been exploring this landscape by writing an open source fuzzer aimed at discovering new test cases (not just crashes!) using binary rewriting of managed executables and genetic modification of a test corpus, implemented in F# and using Mono.Cecil. I’ll contrast the fundamentals of each discipline, demonstrate tools used by experts on both sides of the security and QA fence, and challenge the audience to find new ways to mix them up. Expect to see lots of code and leave with ideas for making entire communities better, not just your own team!

Transcript

1. M a s h i n g U p Q A
   a n d S e c u r i t y
   Craig Stuntz
   Improving
   https://speakerdeck.com/craigstuntz
   https://github.com/CraigStuntz/Fizil
   

   View Slide

2. https://www.flickr.com/photos/futureshape/566200801
   

   View Slide

3. View Slide

4. https://what-if.xkcd.com/49/
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. S o f t w a r e
   C o r r e c t n e s s
   
   

   View Slide

9. M a n u a l
   A n a l y s i s
   
   

   View Slide

10. U n d e f i n e d
    B e h a v i o r
    
    

    View Slide

11. I m p l e m e n t i n g
    T h i s S t u f f
    
    

    View Slide

12. View Slide

13. View Slide

14. 20 TB SWF
    files from Google index
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    

    View Slide

15. 1 week
    run time on
    2000 cores
    to find minimal set of
    20000 SWF
    files
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    

    View Slide

16. 3 weeks
    run time on
    2000 cores
    with mutated inputs
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    

    View Slide

17. 㱺 400
    unique crash signatures
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    

    View Slide

18. 㱺 106
    distinct security bugs
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    

    View Slide

19. View Slide

20. View Slide

21. https://www.flickr.com/photos/sloth_rider/392367929
    

    View Slide

22. https://commons.wikimedia.org/wiki/File:ACT_recycling_truck.jpg
    

    View Slide

23. View Slide

24. View Slide

25. Fizil AFL
    Runs on Windows ✅ There’s a fork
    Runs on Unix ❌ ✅
    Fast ❌ ✅
    Bunnies! ❌
    Process models In Process, Out of Process Fork Server, Out of Process
    Instrumentation guided Soon? ✅
    Automatic instrumentation .NET Assemblies Clang, GCC, Python
    Rich suite of fuzzing strategies Getting there! ✅
    Automatically disables crash reporting ✅ ❌
    Rich tooling ❌ ✅
    Proven track record ❌ ✅
    Stable ❌ ✅
    License Apache 2.0 Apache 2.0
    

    View Slide

26. View Slide

27. View Slide

28. View Slide

29. https://unsplash.com/search/bug?photo=emTCWiq2txk
    

    View Slide

30. F u z z i n g
    https://commons.wikimedia.org/wiki/File:Rabbit_american_fuzzy_lop_buck_white.jpg
    

    View Slide

31. { "a" : "bc" }
    

    View Slide

32. A
    B D
    C
    E
    

    View Slide

33. A
    B D
    C
    E
    

    View Slide

34. { "a" : "bc" }
    ✅
    

    View Slide

35. { "a" : "bc" }
    

    View Slide

36. { "a" : "bc" }
    |
    

    View Slide

37. A
    B D
    C
    E
    

    View Slide

38. A
    B D
    C
    E
    

    View Slide

39. | "a" : "bc" }
    ❌
    

    View Slide

40. https://www.flickr.com/photos/29278394@N00/696701369
    

    View Slide

41. I m p o s s i b l e ?
    Or just really, amazingly difficult?
    https://commons.wikimedia.org/wiki/File:Impossible_cube_illusion_angle.svg
    

    View Slide

42. View Slide

43. View Slide

44. https://xkcd.com/1316/
    

    View Slide

45. View Slide

46. View Slide

47. E x p l o r a t o r y
    https://dojo.ministryoftesting.com/lessons/exploratory-testing-an-api
    

    View Slide

48. S e c u r i t y ⊇ Q A ?
    

    View Slide

49. Behavior
    

    View Slide

50. Behavior
    Specification
    

    View Slide

51. Behavior
    Specification
    

    View Slide

52. Behavior
    Specification
    

    View Slide

53. Behavior
    Specification
    

    View Slide

54. Behavior
    Specification
    

    View Slide

55. P e o p l e
    https://www.flickr.com/photos/wocintechchat/25677176162/
    

    View Slide

56. http://amanda.secured.org/in-securities-comic/
    

    View Slide

57. https://www.quora.com/What-qualities-make-a-good-QA-engineer
    —Thomas Peham
    

    View Slide

58. T o o l s
    https://commons.wikimedia.org/wiki/
    File:Tools,_arsenical_copper,_Naxos,_2700%E2%80%932200_BC,_BM,_GR_1969.12-31,_142703.jpg
    

    View Slide

59. View Slide

60. View Slide

61. S i m p l e T e s t i n g
    https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf
    

    View Slide

62. https://laurent22.github.io/so-injections/
    

    View Slide

63. View Slide

64. https://en.wikipedia.org/wiki/File:Row_hammer.svg
    

    View Slide

65. S p e c i f i c a t i o n s
    https://lorinhochstein.wordpress.com/2014/06/04/crossing-the-river-with-tla/
    

    View Slide

66. []
    let testReadDoubleWithExponent() =
    let actual = parseString "10.0e1"
    actual |> shouldEqual (Parsed (JsonNumber "10.0e1"))
    

    View Slide

67. let toHexString (bytes: byte[]) : string =
    //...
    

    View Slide

68. http://d3s.mff.cuni.cz/research/seminar/download/2010-02-23-Tobies-HypervisorVerification.pdf
    

    View Slide

69. Best Known
    Example
    Released
    Informal
    Spec
    Formal
    Spec
    Execute
    QuickCheck
    1999
    “Reversing a list twice should result in the
    same list”
    prop_RevRev xs = reverse (reverse xs) == xs
    where types = xs::[Int]
    Main> quickCheck prop_RevRev
    OK, passed 100 tests.
    

    View Slide

70. Best Known
    Example
    Released
    Informal
    Spec
    Formal
    Spec
    Execute
    QuickCheck
    1999
    “Reversing a list twice should result in the
    same list”
    prop_RevRev xs = reverse (reverse xs) == xs
    where types = xs::[Int]
    Main> quickCheck prop_RevRev
    OK, passed 100 tests.
    

    View Slide

71. Best Known
    Example
    Released
    Informal
    Spec
    Formal
    Spec
    Execute
    QuickCheck
    1999
    “Reversing a list twice should result in the
    same list”
    prop_RevRev xs = reverse (reverse xs) == xs
    where types = xs::[Int]
    Main> quickCheck prop_RevRev
    OK, passed 100 tests.
    AFL
    2007
    “System under test shouldn’t crash no matter what I
    pass to it”
    if (WIFSIGNALED(status) #$ !stop_soon) {
    kill_signal = WTERMSIG(status);
    return FAULT_CRASH;
    }
    ./afl-fuzz -i testcase_dir -o findings_dir -- \
    /path/to/tested/program [&&.program's
    cmdline&&.]
    

    View Slide

72. Best Known
    Example
    Released
    Informal
    Spec
    Formal
    Spec
    Execute
    QuickCheck
    1999
    “Reversing a list twice should result in the
    same list”
    prop_RevRev xs = reverse (reverse xs) == xs
    where types = xs::[Int]
    Main> quickCheck prop_RevRev
    OK, passed 100 tests.
    AFL
    2007
    “System under test shouldn’t crash no matter what I
    pass to it”
    if (WIFSIGNALED(status) #$ !stop_soon) {
    kill_signal = WTERMSIG(status);
    return FAULT_CRASH;
    }
    ./afl-fuzz -i testcase_dir -o findings_dir -- \
    /path/to/tested/program [&&.program's
    cmdline&&.]
    

    View Slide

73. https://www.flickr.com/photos/x1brett/2279939232
    

    View Slide

74. https://medium.com/@betable/tifu-by-using-math-random-f1c308c4fd9d
    

    View Slide

75. Thought Experiment:
    W h a t I f
    A u t o m a t e d T e s t s
    W e r e P e r f e c t ?
    

    View Slide

76. View Slide

77. View Slide

78. W h a t I f S e c u r i t y
    A n a l y s i s T o o l s
    W e r e P e r f e c t ?
    

    View Slide

79. –DNI James Clapper
    “Something like 90 percent of cyber intrusions start with
    phishing… Somebody always falls for it.”
    https://twitter.com/ODNIgov/status/776070411482193920
    

    View Slide

80. https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45366.pdf
    

    View Slide

81. View Slide

82. Manual Testing
    Examples Exploratory testing, Binary analysis
    Effort Very high
    Killer App
    Finding cases where code technically correct but fails at human-
    computer interaction
    Major Disadvantage Often misused
    

    View Slide

83. Dynamic Analysis
    Examples QuickCheck, AFL, sqlmap
    Effort Low
    Killer App More like an app killer, amiright?
    Major Disadvantage Tends to find a few specific (though important!) bugs
    

    View Slide

84. Static Analysis
    Examples FxCop, FindBugs, Coverity, Veracode
    Effort Very low
    Killer App Cheaper than air. Just do it.
    Major Disadvantage Limited to finding a few hundred important kinds of bugs
    

    View Slide

85. Formal Verification / Symbolic Execution
    Examples VCC, TLA+, Cryptol
    Effort High effort but correspondingly high return
    Killer App MiTLS, Hyper-V Memory Manager
    Major Disadvantage Hard to find people with skill set
    

    View Slide

86. Program Synthesis
    Examples Nothing off the shelf, really, but Agda and Z3 help
    Effort PhD-level research
    Killer App Elimination of incidental complexity
    Major Disadvantage Doesn’t really exist in general form
    

    View Slide

87. View Slide

88. View Slide

89. How Amazon Web Services Uses Formal Methods
    “Formal methods are a big success at AWS, helping us
    prevent subtle but serious bugs from reaching
    production, bugs we would not have found through any
    other technique. They have helped us devise aggressive
    optimizations to complex algorithms without sacrificing
    quality.”
    http://research.microsoft.com/en-us/um/people/lamport/tla/amazon.html
    

    View Slide

90. View Slide

91. View Slide

92. “Finding and Understanding Bugs in C Compilers,”
    Yang et al.
    https://www.flux.utah.edu/paper/yang-pldi11
    

    View Slide

93. View Slide

94. ===================================
    Technical "whitepaper" for afl-fuzz
    ===================================
    This document provides a quick overview of the guts of American Fuzzy Lop.
    See README for the general instruction manual; and for a discussion of
    motivations and design goals behind AFL, see historical_notes.txt.
    0) Design statement
    -------------------
    American Fuzzy Lop does its best not to focus on any singular principle of
    operation and not be a proof-of-concept for any specific theory. The tool can
    be thought of as a collection of hacks that have been tested in practice,
    found to be surprisingly effective, and have been implemented in the simplest,
    most robust way I could think of at the time.
    Many of the resulting features are made possible thanks to the availability of
    lightweight instrumentation that served as a foundation for the tool, but this
    mechanism should be thought of merely as a means to an end. The only true
    governing principles are speed, reliability, and ease of use.
    1) Coverage measurements
    ------------------------
    The instrumentation injected into compiled programs captures branch (edge)
    coverage, along with coarse branch-taken hit counts. The code injected at
    branch points is essentially equivalent to:
    cur_location = ;
    shared_mem[cur_location ^ prev_location]++;
    http://lcamtuf.coredump.cx/afl/technical_details.txt
    

    View Slide

95. M e m o r y
    

    View Slide

96. { "a" : "bc" }
    

    View Slide

97. let jsonNetResult =
    try JsonConvert.DeserializeObject(str) |> ignore
    Success
    with
    | :? JsonReaderException as jre -> jre.Message |> Error
    | :? JsonSerializationException as jse -> jse.Message |> Error
    | :? System.FormatException as fe ->
    if fe.Message.StartsWith("Invalid hex character”) // hard coded in Json.NET
    then fe.Message |> Error
    else reraise()
    ⃪ T
    est
    ⬑ Special case error stuff
    

    View Slide

98. use proc = new Process()
    proc.StartInfo.FileName <- executablePath
    inputMethod.BeforeStart proc testCase.Data
    proc.StartInfo.UseShellExecute <- false
    proc.StartInfo.RedirectStandardOutput <- true
    proc.StartInfo.RedirectStandardError <- true
    proc.StartInfo.EnvironmentVariables.Add(SharedMemory.environmentVariableName, sharedMemoryName)
    let output = new System.Text.StringBuilder()
    let err = new System.Text.StringBuilder()
    proc.OutputDataReceived.Add(fun args -> output.Append(args.Data) |> ignore)
    proc.ErrorDataReceived.Add (fun args -> err.Append(args.Data) |> ignore)
    proc.Start() |> ignore
    inputMethod.AfterStart proc testCase.Data
    proc.BeginOutputReadLine()
    proc.BeginErrorReadLine()
    proc.WaitForExit()
    let exitCode = proc.ExitCode
    let crashed = exitCode = WinApi.ClrUnhandledExceptionCode
    ⃪ Set up
    ⃪ Read results
    ⃪ Important bit
    

    View Slide

99. View Slide

100. /// An ordered list of functions to use when starting with a single piece of
     /// example data and producing new examples to try
     let private allStrategies = [
     bitFlip 1
     bitFlip 2
     bitFlip 4
     byteFlip 1
     byteFlip 2
     byteFlip 4
     arith8
     arith16
     arith32
     interest8
     interest16
     ]
     

     View Slide

101. let totalBits = bytes.Length * 8
     let testCases = seq {
     for bit = 0 to totalBits - flipBits do
     let newBytes = Array.copy bytes
     let firstByte = bit / 8
     let firstByteMask, secondByteMask = bitMasks(bit, flipBits)
     let newFirstByte = bytes.[firstByte] ^^^ firstByteMask
     newBytes.[firstByte] <- newFirstByte
     let secondByte = firstByte + 1
     if secondByteMask <> 0uy && secondByte < bytes.Length
     then
     let newSecondByte = bytes.[secondByte] ^^^ secondByteMask
     newBytes.[secondByte] <- newSecondByte
     yield newBytes
     }
     Fuzz one byte →
     ^^^ means xor
     ↓
     

     View Slide

102. https://commons.wikimedia.org/wiki/File:CPT-Recursion-Factorial-Code.svg
     

     View Slide

103. private static void F(string arg)
     {
     Console.WriteLine("f");
     Console.Error.WriteLine("Error!");
     Environment.Exit(1);
     }
     

     View Slide

104. private static void F(string arg)
     {
     instrument.Trace(29875);
     Console.WriteLine("f");
     Console.Error.WriteLine("Error!");
     Environment.Exit(1);
     }
     ← Random number
     

     View Slide

105. private static void F(string arg)
     {
     #if MANUAL_INSTRUMENTATION
     instrument.Trace(29875);
     #endif
     Console.WriteLine("f");
     Console.Error.WriteLine("Error!");
     Environment.Exit(1);
     }
     

     View Slide

106. View Slide

107. View Slide

108. let stringify (ob: obj) : string =
     JsonConvert.SerializeObject(ob)
     

     View Slide

109. let stringify (ob: obj) : string =
     JsonConvert.SerializeObject(ob)
     // Method: System.String\u0020Program::stringify(System.Object)
     .body stringify {
     arg_02_0 [generated]
     arg_07_0 [generated]
     nop()
     arg_02_0 = ldloc(ob)
     arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0)
     ret(arg_07_0)
     }
     

     View Slide

110. let stringify (ob: obj) : string =
     JsonConvert.SerializeObject(ob)
     // Method: System.String\u0020Program::stringify(System.Object)
     .body stringify {
     arg_02_0 [generated]
     arg_07_0 [generated]
     nop()
     arg_02_0 = ldloc(ob)
     arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0)
     ret(arg_07_0)
     }
     // Method: System.String\u0020Program::stringify(System.Object)
     .body stringify {
     arg_05_0 [generated]
     arg_0C_0 [generated]
     arg_11_0 [generated]
     arg_05_0 = ldc.i4(23831)
     call(Instrument::Trace, arg_05_0)
     nop()
     arg_0C_0 = ldloc(ob)
     arg_11_0 = call(JsonConvert::SerializeObject, arg_0C_0)
     ret(arg_11_0)
     }
     

     View Slide

111. View Slide

112. View Slide

113. http://www.json.org/
     

     View Slide

114. https://tools.ietf.org/html/rfc4627
     

     View Slide

115. http://www.ecma-international.org/ecma-262/5.1/#sec-15.12
     

     View Slide

116. http://www.ecma-international.org/publications/standards/Ecma-404.htm
     

     View Slide

117. https://tools.ietf.org/html/rfc7158
     

     View Slide

118. https://tools.ietf.org/html/rfc7159
     

     View Slide

119. https://github.com/nst/STJSON
     

     View Slide

120. https://github.com/CraigStuntz/Fizil/blob/master/StJson/StJsonParser.fs
     

     View Slide

121. Standard Accepts, Json.NET Rejects
     Value
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     Standard Says No limit
     Json.NET MaximumJavascriptIntegerCharacterLength = 380;
     

     View Slide

122. Standard Rejects, Json.NET Accepts
     Value [,,,]
     Standard Says
     A JSON value MUST be an object, array, number, or string, or one
     of
     the following three literal names:
     false null true
     Json.NET [null, null, null, null]
     

     View Slide

123. I m p l e m e n t a t i o n
     D e t a i l s
     

     View Slide

124. let private insertTraceInstruction(ilProcessor: ILProcessor, before: Instruction, state) =
     let compileTimeRandom = state.Random.Next(0, UInt16.MaxValue |> Convert.ToInt32)
     let ldArg = ilProcessor.Create(OpCodes.Ldc_I4, compileTimeRandom)
     let callTrace = ilProcessor.Create(OpCodes.Call, state.Trace)
     ilProcessor.InsertBefore(before, ldArg)
     ilProcessor.InsertAfter (ldArg, callTrace)
     This margin is too narrow to contain a try/finally example, so see:
     https://goo.gl/W4y7JH
     

     View Slide

125. View Slide

126. let private removeStrongName (assemblyDefinition : AssemblyDefinition) =
     let name = assemblyDefinition.Name;
     name.HasPublicKey <- false;
     name.PublicKey <- Array.empty;
     assemblyDefinition.Modules |> Seq.iter (
     fun moduleDefinition ->
     moduleDefinition.Attributes <-
     moduleDefinition.Attributes &&& ~~~ModuleAttributes.StrongNameSigned)
     let aptca = assemblyDefinition.CustomAttributes.FirstOrDefault(
     fun attr -> attr.AttributeType.FullName
     = typeof.FullName)
     assemblyDefinition.CustomAttributes.Remove aptca |> ignore
     assembly.MainModule.AssemblyReferences
     |> Seq.filter (fun reference -> Set.contains reference.Name assembliesToInstrument)
     |> Seq.iter (fun reference ->
     reference.PublicKeyToken <- null
     )
     

     View Slide

127. I n / O u t o f
     P r o c e s s
     

     View Slide

128. View Slide

129. View Slide

130. –ECMA-335, Common Language Infrastructure (CLI),
     Partition I
     “If marked BeforeFieldInit then the type’s initializer
     method is executed at, or sometime before, first access to
     any static field defined for that type.”
     

     View Slide

131. f ( x ) = f ( x )
     t i m e ( f ( x ) ) ! = t i m e ( f ( x ) )
     ✅
     ❌
     

     View Slide

132. U n i c o d e
     Original JSON
     { "a": "bc" }
     ASCII Bytes
     7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
     UTF-8 with Byte Order Mark
     EF BB BF 7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
     UTF-16 BE with BOM
     FE FF 00 7B 00 20 00 22 00 61 00 22 00 20 00 3A 00 20 00 22
     00 62 00 63 00 22 00 20 00 7D
     

     View Slide

133. T h a n k Y o u !
     Presentation Review
     Cassandra Faris
     Chad James
     Damian Synadinos
     Doug Mair
     Tommy Graves
     Source Code Inspiration
     Michał Zalewski
     Nicolas Seriot
     Everyone Who Works on dnSpy
     & Mono.Cecil
     

     View Slide

134. View Slide

135. C r a i g S t u n t z
     @craigstuntz
     Craig.Stuntz@Improving.com
     http://www.craigstuntz.com
     http://www.meetup.com/Papers-We-Love-Columbus/
     https://speakerdeck.com/craigstuntz
     

     View Slide