Mashing Up QA and Security - Preroll Quotes

The preroll deck of thought-provoking quotes I showed before my CodeMash 2017 talk, Mashing Up QA and Security

Transcript

1. Mashing Up QA and Security …will start at 8:30 a.m. –John D. Cook “This brings up a third reason why testing [random number generators] is subtle: any test we write will fail occasionally. If a test never fails, then it demonstrates a predictable attribute of our random number sequence. So not only can our tests fail from time to time, they should fail from time to time!”
2. Mashing Up QA and Security …will start at 8:30 a.m. –Tavis Ormandy “We want to make very generic fuzz testing tools that can apply to lots of software. We’ve been calling this “Making software dumber”, as opposed to making fuzzers smarter.”
3. Mashing Up QA and Security …will start at 8:30 a.m. –Karen N. Johnson “I recall asking what the worst-case scenario was if the software didn’t perform correctly. The answer ‘patient death’ made me open my eyes wide and think hard about accepting the work.”
4. Mashing Up QA and Security …will start at 8:30 a.m. “Yet, despite the crippling and obvious limitations of fuzzing and the virtues of symbolic execution, there is one jarring discord: I’m fairly certain that probably around 70% of all remote code execution vulnerabilities disclosed in the past few years trace back to fairly "dumb" fuzzing tools, with the pattern showing little change over time. The remaining 30% is attributable almost exclusively to manual work…” –Michał Zalewski
5. Mashing Up QA and Security …will start at 8:30 a.m. “Another myth spread by security researchers is that the planet Earth contains more than six programmers who can correctly use security labels and information flow control. This belief requires one to assume that, even though the most popular variable names are “thing” and “thing2,” programmers will magically become disciplined software architects when confronted with a Dungeons-and-Dragons- style type system that requires variables to be annotated with rich biographical data and a list of vulnerabilities to output sinks. ” –James Mickens
6. https://twitter.com/whitequark/status/815008102814810112
7. Mashing Up QA and Security …will start at 8:30 a.m. –Kate Thompson “It’s easier to get something right if it’s short.”
8. Mashing Up QA and Security …will start at 8:30 a.m. –D. J. Bernstein “The conventional wisdom: ‘We’ll never build a serious software system without security holes.’ Why not? ‘It’s impossible.’ Or: ‘Maybe it’s possible, but it’s much too expensive.’ “The conventional wisdom: ‘We’ll never build a tunnel from England to France. Why not? ‘It’s impossible.’ Or: ‘Maybe it’s possible, but it’s much too expensive.’ Engineer’s reaction: ‘How expensive is it? How big a tunnel can we build? How can we reduce the costs?’”
9. Mashing Up QA and Security …will start at 8:30 a.m. –Moxie Marlinspike “Computers are no longer for computer people. We can’t just teach everyone to be like us.”
10. Mashing Up QA and Security …will start at 8:30 a.m. –Alex Stamos “Most large companies now have Big Data capabilities only GOOG/FB/YHOO/MSFT/GS had 5 years ago. Do security products leverage? Nope!”
11. Mashing Up QA and Security …will start at 8:30 a.m. – Regehr, Yang et al. “As of early 2011, the under-development version of CompCert is the only compiler we have tested for which Csmith cannot find wrong- code errors. This is not for lack of trying: we have devoted about six CPU-years to the task. The apparent unbreakability of CompCert supports a strong argument that developing compiler optimizations within a proof framework, where safety checks are explicit and machine-checked, has tangible benefits for compiler users.” https://www.flux.utah.edu/paper/yang-pldi11