Mashing Up QA and Security - CodeMash 2017

Security is domain specific quality assurance, but developers, testers, and security professionals often don’t work together. When this type of disconnect exists between big groups of people who are very good at their jobs, there is usually a mostly untapped potential for learning. I’ve been exploring this landscape by writing an open source fuzzer aimed at discovering new test cases (not just crashes!) using binary rewriting of managed executables and genetic modification of a test corpus, implemented in F# and using Mono.Cecil. I’ll contrast the fundamentals of each discipline, demonstrate tools used by experts on both sides of the security and QA fence, and challenge the audience to find new ways to mix them up. Expect to see lots of code and leave with ideas for making entire communities better, not just your own team!

Transcript

1. M a s h i n g U p Q A a n d S e c u r i t y Craig Stuntz Improving https://speakerdeck.com/craigstuntz https://github.com/CraigStuntz/Fizil
2. https://www.flickr.com/photos/futureshape/566200801
4. https://what-if.xkcd.com/49/
8. S o f t w a r e C o r r e c t n e s s <spoilers>
9. M a n u a l A n a l y s i s <spoilers>
10. U n d e f i n e d B e h a v i o r <spoilers>
11. I m p l e m e n t i n g T h i s S t u f f <spoilers>
14. 20 TB SWF files from Google index https://security.googleblog.com/2011/08/fuzzing-at-scale.html
15. 1 week run time on 2000 cores to find minimal set of 20000 SWF files https://security.googleblog.com/2011/08/fuzzing-at-scale.html
16. 3 weeks run time on 2000 cores with mutated inputs https://security.googleblog.com/2011/08/fuzzing-at-scale.html
17. 㱺 400 unique crash signatures https://security.googleblog.com/2011/08/fuzzing-at-scale.html
18. 㱺 106 distinct security bugs https://security.googleblog.com/2011/08/fuzzing-at-scale.html
21. https://www.flickr.com/photos/sloth_rider/392367929
22. https://commons.wikimedia.org/wiki/File:ACT_recycling_truck.jpg
25. Fizil AFL Runs on Windows ✅ There’s a fork Runs on Unix ❌ ✅ Fast ❌ ✅ Bunnies! ❌ Process models In Process, Out of Process Fork Server, Out of Process Instrumentation guided Soon? ✅ Automatic instrumentation .NET Assemblies Clang, GCC, Python Rich suite of fuzzing strategies Getting there! ✅ Automatically disables crash reporting ✅ ❌ Rich tooling ❌ ✅ Proven track record ❌ ✅ Stable ❌ ✅ License Apache 2.0 Apache 2.0
29. https://unsplash.com/search/bug?photo=emTCWiq2txk
30. F u z z i n g https://commons.wikimedia.org/wiki/File:Rabbit_american_fuzzy_lop_buck_white.jpg
31. { "a" : "bc" }
32. A B D C E
33. A B D C E
34. { "a" : "bc" } ✅
35. { "a" : "bc" }
36. { "a" : "bc" } |
37. A B D C E
38. A B D C E
39. | "a" : "bc" } ❌
40. https://www.flickr.com/photos/29278394@N00/696701369
41. I m p o s s i b l e ? Or just really, amazingly difficult? https://commons.wikimedia.org/wiki/File:Impossible_cube_illusion_angle.svg
44. https://xkcd.com/1316/
47. E x p l o r a t o r y https://dojo.ministryoftesting.com/lessons/exploratory-testing-an-api
48. S e c u r i t y ⊇ Q A ?
49. Behavior
50. Behavior Specification
51. Behavior Specification
52. Behavior Specification
53. Behavior Specification
54. Behavior Specification
55. P e o p l e https://www.flickr.com/photos/wocintechchat/25677176162/
56. http://amanda.secured.org/in-securities-comic/
57. https://www.quora.com/What-qualities-make-a-good-QA-engineer —Thomas Peham
58. T o o l s https://commons.wikimedia.org/wiki/ File:Tools,_arsenical_copper,_Naxos,_2700%E2%80%932200_BC,_BM,_GR_1969.12-31,_142703.jpg
61. S i m p l e T e s t i n g https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf
62. https://laurent22.github.io/so-injections/
64. https://en.wikipedia.org/wiki/File:Row_hammer.svg
65. S p e c i f i c a t i o n s https://lorinhochstein.wordpress.com/2014/06/04/crossing-the-river-with-tla/
66. [<Test>] let testReadDoubleWithExponent() = let actual = parseString "10.0e1" actual |> shouldEqual (Parsed (JsonNumber "10.0e1"))
67. let toHexString (bytes: byte[]) : string = //...
68. http://d3s.mff.cuni.cz/research/seminar/download/2010-02-23-Tobies-HypervisorVerification.pdf
69. Best Known Example Released Informal Spec Formal Spec Execute QuickCheck 1999 “Reversing a list twice should result in the same list” prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Int] Main> quickCheck prop_RevRev OK, passed 100 tests.
70. Best Known Example Released Informal Spec Formal Spec Execute QuickCheck 1999 “Reversing a list twice should result in the same list” prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Int] Main> quickCheck prop_RevRev OK, passed 100 tests.
71. Best Known Example Released Informal Spec Formal Spec Execute QuickCheck 1999 “Reversing a list twice should result in the same list” prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Int] Main> quickCheck prop_RevRev OK, passed 100 tests. AFL 2007 “System under test shouldn’t crash no matter what I pass to it” if (WIFSIGNALED(status) #$ !stop_soon) { kill_signal = WTERMSIG(status); return FAULT_CRASH; } ./afl-fuzz -i testcase_dir -o findings_dir -- \ /path/to/tested/program [&&.program's cmdline&&.]
72. Best Known Example Released Informal Spec Formal Spec Execute QuickCheck 1999 “Reversing a list twice should result in the same list” prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Int] Main> quickCheck prop_RevRev OK, passed 100 tests. AFL 2007 “System under test shouldn’t crash no matter what I pass to it” if (WIFSIGNALED(status) #$ !stop_soon) { kill_signal = WTERMSIG(status); return FAULT_CRASH; } ./afl-fuzz -i testcase_dir -o findings_dir -- \ /path/to/tested/program [&&.program's cmdline&&.]
73. https://www.flickr.com/photos/x1brett/2279939232
74. https://medium.com/@betable/tifu-by-using-math-random-f1c308c4fd9d
75. Thought Experiment: W h a t I f A u t o m a t e d T e s t s W e r e P e r f e c t ?
78. W h a t I f S e c u r i t y A n a l y s i s T o o l s W e r e P e r f e c t ?
79. –DNI James Clapper “Something like 90 percent of cyber intrusions start with phishing… Somebody always falls for it.” https://twitter.com/ODNIgov/status/776070411482193920
80. https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45366.pdf
82. Manual Testing Examples Exploratory testing, Binary analysis Effort Very high Killer App Finding cases where code technically correct but fails at human- computer interaction Major Disadvantage Often misused
83. Dynamic Analysis Examples QuickCheck, AFL, sqlmap Effort Low Killer App More like an app killer, amiright? Major Disadvantage Tends to find a few specific (though important!) bugs
84. Static Analysis Examples FxCop, FindBugs, Coverity, Veracode Effort Very low Killer App Cheaper than air. Just do it. Major Disadvantage Limited to finding a few hundred important kinds of bugs
85. Formal Verification / Symbolic Execution Examples VCC, TLA+, Cryptol Effort High effort but correspondingly high return Killer App MiTLS, Hyper-V Memory Manager Major Disadvantage Hard to find people with skill set
86. Program Synthesis Examples Nothing off the shelf, really, but Agda and Z3 help Effort PhD-level research Killer App Elimination of incidental complexity Major Disadvantage Doesn’t really exist in general form
89. How Amazon Web Services Uses Formal Methods “Formal methods are a big success at AWS, helping us prevent subtle but serious bugs from reaching production, bugs we would not have found through any other technique. They have helped us devise aggressive optimizations to complex algorithms without sacrificing quality.” http://research.microsoft.com/en-us/um/people/lamport/tla/amazon.html
92. “Finding and Understanding Bugs in C Compilers,” Yang et al. https://www.flux.utah.edu/paper/yang-pldi11
94. =================================== Technical "whitepaper" for afl-fuzz =================================== This document provides a quick overview of the guts of American Fuzzy Lop. See README for the general instruction manual; and for a discussion of motivations and design goals behind AFL, see historical_notes.txt. 0) Design statement ------------------- American Fuzzy Lop does its best not to focus on any singular principle of operation and not be a proof-of-concept for any specific theory. The tool can be thought of as a collection of hacks that have been tested in practice, found to be surprisingly effective, and have been implemented in the simplest, most robust way I could think of at the time. Many of the resulting features are made possible thanks to the availability of lightweight instrumentation that served as a foundation for the tool, but this mechanism should be thought of merely as a means to an end. The only true governing principles are speed, reliability, and ease of use. 1) Coverage measurements ------------------------ The instrumentation injected into compiled programs captures branch (edge) coverage, along with coarse branch-taken hit counts. The code injected at branch points is essentially equivalent to: cur_location = <COMPILE_TIME_RANDOM>; shared_mem[cur_location ^ prev_location]++; http://lcamtuf.coredump.cx/afl/technical_details.txt
95. M e m o r y
96. { "a" : "bc" }
97. let jsonNetResult = try JsonConvert.DeserializeObject<obj>(str) |> ignore Success with | :? JsonReaderException as jre -> jre.Message |> Error | :? JsonSerializationException as jse -> jse.Message |> Error | :? System.FormatException as fe -> if fe.Message.StartsWith("Invalid hex character”) // hard coded in Json.NET then fe.Message |> Error else reraise() ⃪ T est ⬑ Special case error stuff
98. use proc = new Process() proc.StartInfo.FileName <- executablePath inputMethod.BeforeStart proc testCase.Data proc.StartInfo.UseShellExecute <- false proc.StartInfo.RedirectStandardOutput <- true proc.StartInfo.RedirectStandardError <- true proc.StartInfo.EnvironmentVariables.Add(SharedMemory.environmentVariableName, sharedMemoryName) let output = new System.Text.StringBuilder() let err = new System.Text.StringBuilder() proc.OutputDataReceived.Add(fun args -> output.Append(args.Data) |> ignore) proc.ErrorDataReceived.Add (fun args -> err.Append(args.Data) |> ignore) proc.Start() |> ignore inputMethod.AfterStart proc testCase.Data proc.BeginOutputReadLine() proc.BeginErrorReadLine() proc.WaitForExit() let exitCode = proc.ExitCode let crashed = exitCode = WinApi.ClrUnhandledExceptionCode ⃪ Set up ⃪ Read results ⃪ Important bit
100. /// An ordered list of functions to use when starting with a single piece of /// example data and producing new examples to try let private allStrategies = [ bitFlip 1 bitFlip 2 bitFlip 4 byteFlip 1 byteFlip 2 byteFlip 4 arith8 arith16 arith32 interest8 interest16 ]
101. let totalBits = bytes.Length * 8 let testCases = seq { for bit = 0 to totalBits - flipBits do let newBytes = Array.copy bytes let firstByte = bit / 8 let firstByteMask, secondByteMask = bitMasks(bit, flipBits) let newFirstByte = bytes.[firstByte] ^^^ firstByteMask newBytes.[firstByte] <- newFirstByte let secondByte = firstByte + 1 if secondByteMask <> 0uy && secondByte < bytes.Length then let newSecondByte = bytes.[secondByte] ^^^ secondByteMask newBytes.[secondByte] <- newSecondByte yield newBytes } Fuzz one byte → ^^^ means xor ↓
102. https://commons.wikimedia.org/wiki/File:CPT-Recursion-Factorial-Code.svg
103. private static void F(string arg) { Console.WriteLine("f"); Console.Error.WriteLine("Error!"); Environment.Exit(1); }
104. private static void F(string arg) { instrument.Trace(29875); Console.WriteLine("f"); Console.Error.WriteLine("Error!"); Environment.Exit(1); } ← Random number
105. private static void F(string arg) { #if MANUAL_INSTRUMENTATION instrument.Trace(29875); #endif Console.WriteLine("f"); Console.Error.WriteLine("Error!"); Environment.Exit(1); }
108. let stringify (ob: obj) : string = JsonConvert.SerializeObject(ob)
109. let stringify (ob: obj) : string = JsonConvert.SerializeObject(ob) // Method: System.String\u0020Program::stringify(System.Object) .body stringify { arg_02_0 [generated] arg_07_0 [generated] nop() arg_02_0 = ldloc(ob) arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0) ret(arg_07_0) }
110. let stringify (ob: obj) : string = JsonConvert.SerializeObject(ob) // Method: System.String\u0020Program::stringify(System.Object) .body stringify { arg_02_0 [generated] arg_07_0 [generated] nop() arg_02_0 = ldloc(ob) arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0) ret(arg_07_0) } // Method: System.String\u0020Program::stringify(System.Object) .body stringify { arg_05_0 [generated] arg_0C_0 [generated] arg_11_0 [generated] arg_05_0 = ldc.i4(23831) call(Instrument::Trace, arg_05_0) nop() arg_0C_0 = ldloc(ob) arg_11_0 = call(JsonConvert::SerializeObject, arg_0C_0) ret(arg_11_0) }
113. http://www.json.org/
114. https://tools.ietf.org/html/rfc4627
115. http://www.ecma-international.org/ecma-262/5.1/#sec-15.12
116. http://www.ecma-international.org/publications/standards/Ecma-404.htm
117. https://tools.ietf.org/html/rfc7158
118. https://tools.ietf.org/html/rfc7159
119. https://github.com/nst/STJSON
120. https://github.com/CraigStuntz/Fizil/blob/master/StJson/StJsonParser.fs
121. Standard Accepts, Json.NET Rejects Value 88888888888888888888888888888888888888888888888888 88888888888888888888888888888888888888888888888888 88888888888888888888888888888888888888888888888888 88888888888888888888888888888888888888888888888888 88888888888888888888888888888888888888888888888888 Standard Says No limit Json.NET MaximumJavascriptIntegerCharacterLength = 380;
122. Standard Rejects, Json.NET Accepts Value [,,,] Standard Says A JSON value MUST be an object, array, number, or string, or one of the following three literal names: false null true Json.NET [null, null, null, null]
123. I m p l e m e n t a t i o n D e t a i l s
124. let private insertTraceInstruction(ilProcessor: ILProcessor, before: Instruction, state) = let compileTimeRandom = state.Random.Next(0, UInt16.MaxValue |> Convert.ToInt32) let ldArg = ilProcessor.Create(OpCodes.Ldc_I4, compileTimeRandom) let callTrace = ilProcessor.Create(OpCodes.Call, state.Trace) ilProcessor.InsertBefore(before, ldArg) ilProcessor.InsertAfter (ldArg, callTrace) This margin is too narrow to contain a try/finally example, so see: https://goo.gl/W4y7JH
126. let private removeStrongName (assemblyDefinition : AssemblyDefinition) = let name = assemblyDefinition.Name; name.HasPublicKey <- false; name.PublicKey <- Array.empty; assemblyDefinition.Modules |> Seq.iter ( fun moduleDefinition -> moduleDefinition.Attributes <- moduleDefinition.Attributes &&& ~~~ModuleAttributes.StrongNameSigned) let aptca = assemblyDefinition.CustomAttributes.FirstOrDefault( fun attr -> attr.AttributeType.FullName = typeof<System.Security.AllowPartiallyTrustedCallersAttribute>.FullName) assemblyDefinition.CustomAttributes.Remove aptca |> ignore assembly.MainModule.AssemblyReferences |> Seq.filter (fun reference -> Set.contains reference.Name assembliesToInstrument) |> Seq.iter (fun reference -> reference.PublicKeyToken <- null )
127. I n / O u t o f P r o c e s s
130. –ECMA-335, Common Language Infrastructure (CLI), Partition I “If marked BeforeFieldInit then the type’s initializer method is executed at, or sometime before, first access to any static field defined for that type.”
131. f ( x ) = f ( x ) t i m e ( f ( x ) ) ! = t i m e ( f ( x ) ) ✅ ❌
132. U n i c o d e Original JSON { "a": "bc" } ASCII Bytes 7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D UTF-8 with Byte Order Mark EF BB BF 7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D UTF-16 BE with BOM FE FF 00 7B 00 20 00 22 00 61 00 22 00 20 00 3A 00 20 00 22 00 62 00 63 00 22 00 20 00 7D
133. T h a n k Y o u ! Presentation Review Cassandra Faris Chad James Damian Synadinos Doug Mair Tommy Graves Source Code Inspiration Michał Zalewski Nicolas Seriot Everyone Who Works on dnSpy & Mono.Cecil
135. C r a i g S t u n t z @craigstuntz Craig.Stuntz@Improving.com http://www.craigstuntz.com http://www.meetup.com/Papers-We-Love-Columbus/ https://speakerdeck.com/craigstuntz