Footguns and factorisation: how to make users of your cryptographic library successful

Video of this talk.

Abstract

Cryptography forms the backbone of how we securely use information online, but most developers don’t have more than a surface level understanding of cryptography.

Shannon's maxim states that “one ought to design systems under the assumption that the enemy will immediately gain full familiarity with them”. Open source makes this feasible for cryptography, with open source cryptographic libraries handling a huge proportion of information on the internet in flight and at rest.

Developers place a lot of trust in the authors of these libraries to get the cryptography engineering right.

But when basic usability issues result in developers using the libraries incorrectly, that trust and painstaking cryptography engineering can be for naught. Worse still, developers often believe they have used the libraries to build something that is secure. But that belief is often mistaken — their use of these libraries is actually insecure.

In this talk, attendees will learn:

1. What research says about how the usability of cryptographic libraries impacts the ability of users to deliver code that handles data securely
2. What common usability traps open source cryptography projects fall into
3. How authors, maintainers, and communities around open source cryptographic library can make their users successful

References

Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C., Cappos, J., & Zhuang, Y. (2014, December). It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305).

Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP) (pp. 289-305). IEEE.

Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (pp. 3070-3084).

Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7).

Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the usability of cryptographic APIs In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE.

Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve security warnings, too: On the effect of integrated security advice on cryptographic API misuse. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281).

Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

Further reading

API Blindspots: Why Experienced Developers Write Vulnerable Code

Blindspots in Python and Java APIs Result in Vulnerable Code

What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings

I Do and I Understand. Not Yet True for Security APIs. So Sad

How Usable are Rust Cryptography APIs?

You Really Shouldn’t Roll Your Own Crypto: An Empirical Study of Vulnerabilities in Cryptographic Libraries

Transcript

1. Footguns and factorisation How to make users of your cryptographic library successful. Lindsay Holmwood @auxesis
3. In 2020, over 300,000 patient records — including detailed consult notes — were leaked and used to extort Vastaamo patients.
4. At around 4 pm, Jere checked Snapchat. An email notification popped up on his screen. His hands began to shake. The subject line included his full name, his social security number, and the name of a clinic where he’d gotten mental health treatment as a teenager: Vastaamo. They Told Their Therapists Everything. Hackers Leaked It All William Ralston, Wired
6. Vastaamo’s system violated one of the “first principles of cybersecurity”: It didn’t anonymize the records. It didn’t even encrypt them. The only thing protecting patients’ confessions and confidences were a couple of firewalls and a server login screen. They Told Their Therapists Everything. Hackers Leaked It All William Ralston, Wired
7. “I would never do this”
8. 20% of submitted code was insecure, but the developers strongly believed it was secure.
9. It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. Mark Twain
10. Human Factors in Secure Software Development: There is a lot of academic research into cryptography usability and developer behaviour.
11. Let’s take a look at what the research says about: 1. What are the usability traps in cryptography libraries 2. What you can do to help your users avoid them
12. Before we begin…
13. We’re talking about cryptography, not cryptocurrency
14. We’re talking about developers, not end users
15. We’re looking at peer reviewed research, much of it replicated
16. Developers don’t think about security Published 2014
17. Developers shown 6 code examples that contained vulnerabilities covering: ◦ TLS setup ◦ Time-Of-Check-To-Time-Of-Use ◦ Brute force exhaustion ◦ Buffer overflows ◦ XSS ◦ SQL injection The experiment
18. Developers grouped, prompted before each example: The experiment Question Group What is the user input to this program? Control, Priming What happens when this code executes? Control, Priming Could a developer experience unexpected results when running such code? Priming What could be examples of these unexpected results and where do they appear in the code? Priming This code has a vulnerability. Can you pinpoint the problem? Explicit
19. The result? Developers who were primed identified security problems at almost twice the rate of un-primed developers.
20. “security is not part of the heuristics used by developers in their daily programming tasks“ Developers mostly focus on functionality and performance. What it means
21. What it means We think that developers should learn about security, and apply what they learn when they need it. This is not how humans actually think.
22. “we recommend software that interface with the developer (IDEs, text editors, compilers, etc) prime developers on the spot when they need it: while coding” What it means Oliviera et al. (2014)
23. TL;DR, devs don’t think about security, priming increases the likelihood of discovering security bugs
24. Stack Overflow provides functional yet insecure code examples Published 2016
25. Android devs provided with: ◦ a skeleton app ◦ 4 tasks in random order: 1. Secure Networking 2. Secure Storage 3. Inter-Component Communication 4. Least Permissions ◦ 20-30 minutes to complete each task ◦ An exit survey Split into 4 groups, could only refer to: ◦ Stack Overflow ◦ Official documentation ◦ Books ◦ All of the above The experiment
26. The results? 1. Functional results 2. Secure results
27. Functional results Devs assigned Stack Overflow and books performed best. Devs assigned official docs performed worst. Resources Success Rate Stack Overflow 67.3% Book 66.1% Official docs 40.4% Free choice 51.8%
28. Developers self-assessed their solutions to be secure when they mostly were not. Secure results Task Success Rate Confidence Least Permissions 87% 22% Secure Networking 38% 79% Inter Component Comms 38% 70% Secure Storage 38% 68%
29. Android developers who use Stack Overflow get functional code quicker, but are more likely to produce insecure code. What this means
30. Devs are unlikely to stop using Stack Overflow, because they get functional code. ⬇ What this means Developers mostly focus on functionality and performance.
31. “it is critical to develop documentation and resources that combine the usefulness of forums like Stack Overflow with the security awareness of books or official API documents” What this means Acar et al. (2016)
32. What you can do 1. Regularly spend time looking at Stack Overflow questions about your library 2. Identify gaps in documentation being filled by Stack Overflow 3. Fill the gaps 4. Post links in follow up comments on accepted Stack Overflow answers
33. TL;DR, Stack Overflow answers provide functional, insecure code; developers are bad at assessing whether that code is secure
34. ✨ side quest ✨ Published 2021
35. “Our results show that insecure code ends up in the top results and is clicked on more often.” “There is at least a 22.8% chance that one out of the top three Google Search results [for Stack Overflow] leads to insecure code”
37. “Participants that used our modified search engine to look for help online submitted more secure and functional results, with statistical significance”
38. TL;DR, Novel solution to the problem that requires zero behaviour change from developers; can someone in the audience action this?
39. Most “cryptography bugs” come from the use of libraries, not the libraries themselves Published 2014
40. Analysis of 269 “Cryptographic Issues” (CWE310) in CVE database from Jan 2011 to May 2014 The study Categorised vulnerabilities by: Impact ◦ Plaintext disclosure ◦ Manipulator-In-The-Middle ◦ Brute-force ◦ Side-channel Layer ◦ Primitive (like AES, DH) ◦ Protocol (library or abstraction) ◦ Application (everything else)
41. The result? were in the app layer } 100% of plaintext disclosures 86% of MITM 79% of brute-force 0% of side-channel
42. ◦ Expose interfaces in your libraries that are misuse resistant. What you can do
43. 83% of cryptographic bugs come from improper uses of cryptographic libraries. What this means
44. Features and documentation are the #1 requirement for devs to use your library securely Published 2017
45. ◦ Python devs assigned five tasks with stub code: 1. Symmetric: key generation 2. Symmetric: encryption/decryption 3. Asymmetric: key generation 4. Asymmetric: encryption/decryption 5. Symmetric: certificate validation ◦ An exit interview — “is your code secure?” Assigned one of five Python crypto libs 1. cryptography.io🤞 2. Keyczar🤞 3. PyNaCl🤞 4. M2Crypto 5. PyCrypto Asked to only use included docs. The experiment
46. Modified Jupyter Notebook to: ◦ Snapshot the code ◦ Detect and store copy/paste events The experiment
47. The results? 1. Functional results 2. Secure results
48. Wide variation of task success across libraries. Asymmetric tasks less successful. Functional results
49. Copy-paste code 3× more likely to be functional on symmetric tasks. Functional results
50. Python experience, security background, or library experience did not change success. Functional results
51. Asymmetric tasks 3× more likely to produce a secure solution. Secure results
52. Keyczar usage 25× more likely to be secure. But only 10% of submitted Keyczar solutions were functional. Secure results
53. 20% of submitted code was insecure, but the developers strongly believed it was secure. Secure results
54. Libraries with simplified interfaces produced more secure results. And yet, security success rate <80%, even on simplified libraries. What it means
55. What it means Peripheral features (like secure key storage & password-based key generation) make-or-break security. You can’t rely on the dev to identify danger, nor find secure alternatives.
56. You can never have too many code examples. If devs can’t find working code examples in your docs, they will go to Stack Overflow and find a footgun. What it means
57. What you can do 1. Create and maintain an overabundance of secure code examples for your library 2. Identify inputs and dependencies your users implicitly rely on, and either: ◦ Provide secure wrappers for them ◦ Document them exhaustively
58. TL;DR, Misuse resistance helps but it ain’t enough; devs crave code examples — you need to provide them
59. Put runtime warnings into your API for potentially insecure usage Published 2018
60. Python devs assigned: ◦ Three tasks with stub code: 1. Symmetric: key generation 2. Symmetric: encryption 3. Key storage ◦ Control or patch version of PyCrypto An exit interview — “is your code secure?” Patched PyCrypto? Print to stdout if created object is an insecure cryptography feature. The experiment
61. The results Condition Functional success Secure success Control 90% 27% Patch 86% 51% Devs shown warnings produced secure solutions at nearly twice the rate. Devs self-assessed their solutions as more secure when given the warning. Devs rated control + patched PyCrypto equally usable.
62. The results Devs who wrote code that triggered a warning were 15× more likely to convert it to a secure solution
63. Warnings need these 5 things 1. Title message 2. Colour 3. Source code location 4. Link to external resources 5. Message classification
64. You don’t have to change your API’s shape to increase secure usage. Emitting runtime warnings significantly improves secure usage. What it means
65. “we recommend software that interface with the developer (IDEs, text editors, compilers, etc) prime developers on the spot when they need it: while coding” ⬇ ⬇ ⬇
66. What you can do 1. Make your library emit warnings when you detect potentially insecure usage 2. Provide an obvious mechanism to silence warnings
67. TL;DR, Use runtime warnings to prime developers about insecure usage during dev
68. Takeaways
69. Developers don’t prioritise security (you have to do the hard work for them)
70. You can help developers prioritise by providing: ◦ Targeted warnings in your APIs ◦ Frictionless, misuse-resistant APIs that work and are secure ◦ Copious secure code examples in your documentation
71. Most security bugs happen on the periphery of your cryptography library, inside your users’ applications
72. Your challenge: do the hard work to make it easy for your users
73. Thank you! 🤔 What questions do you have? Like the talk? Let @auxesis know on Twitter. Slides at cipherstash.com/lindsay Come help us build next gen cryptography at CipherStash!
74. References Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C., Cappos, J., & Zhuang, Y. (2014, December). It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305). [PDF] Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP) (pp. 289-305). IEEE. [PDF] Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (pp. 3070-3084). [PDF] Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7). [PDF] Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the usability of cryptographic apis. In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE. [PDF] Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve security warnings, too: On the effect of integrated security advice on cryptographic {API} misuse. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281). [PDF] Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13). [PDF]
75. Further reading ◦ API Blindspots: Why Experienced Developers Write Vulnerable Code [PDF] ◦ Blindspots in Python and Java APIs Result in Vulnerable Code [PDF] ◦ What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings [PDF] ◦ I Do and I Understand. Not Yet True for Security APIs. So Sad [PDF] ◦ How Usable are Rust Cryptography APIs? [PDF] ◦ You Really Shouldn’t Roll Your Own Crypto: An Empirical Study of Vulnerabilities in Cryptographic Libraries [PDF]
76. Appendix
77. Developers who were primed identified security problems at almost twice the rate of un-primed developers. Appendix It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and How Priming Can Illuminate Developer’s Blind Spot
78. Some types of vulnerabilities were more familiar to developers than others. Appendix It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and How Priming Can Illuminate Developer’s Blind Spot
79. Stack Overflow provides functional, insecure code Published 2016
80. A study in two parts 1. Survey developers about what information sources they use 2. Study developers when given security challenges with different information sources
81. 1. The Survey ◦ most devs use search engines and Stack Overflow ◦ a lot also consult the official API documentation ◦ a few use books
82. 2. The Study Android devs assigned to one of four study groups: 1. Stack Overflow only 2. Official documentation only 3. Books only 4. Free choice
83. Android devs provided with: ◦ a skeleton app ◦ 4 tasks in random order ◦ 20-30 minutes to complete each task ◦ An exit interview 2. The Study
84. 2. The Study The four tasks: 1. Secure networking 2. Secure storage 3. Inter-Component Communication 4. Least permissions
85. Results? 1. Function 2. Security
86. Results: function "Our results demonstrate that the assigned resource condition had a notable impact on participants’ ability to complete the tasks functionally correctly"
87. Results: function ◦ Stack Overflow and book participants performed best ◦ Official doc participants performed worst Condition Success Rate SO 67.3% Book 66.1% Official Docs 40.4% Free 51.8%
88. Results: function Self-assessment binning: ◦ Confident == strongly agree/agree ◦ Not Confident == strongly disagree/disagree/neutral Task Confidence Least Permissions 81.1% Secure Networking 20.1% ICC 40.7% Secure Storage 53.7%
89. Results: security “Our results suggest that choice of resources has the opposite effect on security than it did on functionality”
90. Results: security ◦ Official Docs and Book participants performed best ◦ Stack Overflow participants performed worst Condition Success Rate SO 51.4% Book 73.0% Official Docs 85.7% Free 65.5%
91. Results: security Self-assessment binning: ◦ Confident == strongly agree/agree ◦ Not Confident == strongly disagree/disagree/neutral Task Confidence Least Permissions 22.2% Secure Networking 79.6% ICC 70.4% Secure Storage 68.5%
92. What this means ◦ “using Stack Overflow helps Android developers to arrive at functional solutions more quickly than with other resources” ◦ “Because Stack Overflow contains many insecure answers, Android developers who rely on this resource are likely to create less secure code.” ◦ “developers are unlikely to give up using resources that help them quickly address their immediate problems”
93. What this means ◦ “it is critical to develop documentation and resources that combine the usefulness of forums like Stack Overflow with the security awareness of books or official API documents” ◦ “Stack Overflow could add a mechanism for explicitly rating the security of provided answers and weighting those rated secure more heavily in search results and thread ordering”
94. Most “cryptography bugs” come from the use of libraries, not the libraries themselves Published 2014
95. Analysis of 269 “Cryptographic Issues” (CWE310) in CVE database from Jan 2011 to May 2014 The study Categorised vulnerabilities by: Impact ◦ Plaintext disclosure ◦ Manipulator-In-The-Middle ◦ Brute-force ◦ Side-channel Layer ◦ Primitive (like AES, DH) ◦ Protocol (library or abstraction) ◦ Application (everything else)
96. 100% of plaintext disclosures were in the app layer 86% of MITM were in the app layer 79% of brute-force were in the app layer 0% of side-channel were in the app layer The result?