Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Footguns and factorisation: how to make users of your cryptographic library successful

Footguns and factorisation: how to make users of your cryptographic library successful

Video of this talk.

Abstract

Cryptography forms the backbone of how we securely use information online, but most developers don’t have more than a surface level understanding of cryptography.

Shannon's maxim states that “one ought to design systems under the assumption that the enemy will immediately gain full familiarity with them”. Open source makes this feasible for cryptography, with open source cryptographic libraries handling a huge proportion of information on the internet in flight and at rest.

Developers place a lot of trust in the authors of these libraries to get the cryptography engineering right.

But when basic usability issues result in developers using the libraries incorrectly, that trust and painstaking cryptography engineering can be for naught. Worse still, developers often believe they have used the libraries to build something that is secure. But that belief is often mistaken — their use of these libraries is actually insecure.

In this talk, attendees will learn:

1. What research says about how the usability of cryptographic libraries impacts the ability of users to deliver code that handles data securely
2. What common usability traps open source cryptography projects fall into
3. How authors, maintainers, and communities around open source cryptographic library can make their users successful

References

Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C., Cappos, J., & Zhuang, Y. (2014, December). It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305).

Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP) (pp. 289-305). IEEE.

Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (pp. 3070-3084).

Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7).

Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the usability of cryptographic APIs In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE.

Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve security warnings, too: On the effect of integrated security advice on cryptographic API misuse. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281).

Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

Further reading

API Blindspots: Why Experienced Developers Write Vulnerable Code

Blindspots in Python and Java APIs Result in Vulnerable Code

What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings

I Do and I Understand. Not Yet True for Security APIs. So Sad

How Usable are Rust Cryptography APIs?

You Really Shouldn’t Roll Your Own Crypto: An Empirical Study of Vulnerabilities in Cryptographic Libraries

Fad1e9ed293fc5b3ec7d4abdffeb636f?s=128

Lindsay Holmwood

January 16, 2022
Tweet

More Decks by Lindsay Holmwood

Other Decks in Technology

Transcript

  1. Footguns and factorisation How to make users of your cryptographic

    library successful. Lindsay Holmwood @auxesis
  2. None
  3. In 2020, over 300,000 patient records — including detailed consult

    notes — were leaked and used to extort Vastaamo patients.
  4. At around 4 pm, Jere checked Snapchat. An email notification

    popped up on his screen. His hands began to shake. The subject line included his full name, his social security number, and the name of a clinic where he’d gotten mental health treatment as a teenager: Vastaamo. They Told Their Therapists Everything. Hackers Leaked It All William Ralston, Wired
  5. None
  6. Vastaamo’s system violated one of the “first principles of cybersecurity”:

    It didn’t anonymize the records. It didn’t even encrypt them. The only thing protecting patients’ confessions and confidences were a couple of firewalls and a server login screen. They Told Their Therapists Everything. Hackers Leaked It All William Ralston, Wired
  7. “I would never do this”

  8. 20% of submitted code was insecure, but the developers strongly

    believed it was secure.
  9. It ain’t what you don’t know that gets you into

    trouble. It’s what you know for sure that just ain’t so. Mark Twain
  10. Human Factors in Secure Software Development: There is a lot

    of academic research into cryptography usability and developer behaviour.
  11. Let’s take a look at what the research says about:

    1. What are the usability traps in cryptography libraries 2. What you can do to help your users avoid them
  12. Before we begin…

  13. We’re talking about cryptography, not cryptocurrency

  14. We’re talking about developers, not end users

  15. We’re looking at peer reviewed research, much of it replicated

  16. Developers don’t think about security Published 2014

  17. Developers shown 6 code examples that contained vulnerabilities covering: ◦

    TLS setup ◦ Time-Of-Check-To-Time-Of-Use ◦ Brute force exhaustion ◦ Buffer overflows ◦ XSS ◦ SQL injection The experiment
  18. Developers grouped, prompted before each example: The experiment Question Group

    What is the user input to this program? Control, Priming What happens when this code executes? Control, Priming Could a developer experience unexpected results when running such code? Priming What could be examples of these unexpected results and where do they appear in the code? Priming This code has a vulnerability. Can you pinpoint the problem? Explicit
  19. The result? Developers who were primed identified security problems at

    almost twice the rate of un-primed developers.
  20. “security is not part of the heuristics used by developers

    in their daily programming tasks“ Developers mostly focus on functionality and performance. What it means
  21. What it means We think that developers should learn about

    security, and apply what they learn when they need it. This is not how humans actually think.
  22. “we recommend software that interface with the developer (IDEs, text

    editors, compilers, etc) prime developers on the spot when they need it: while coding” What it means Oliviera et al. (2014)
  23. TL;DR, devs don’t think about security, priming increases the likelihood

    of discovering security bugs
  24. Stack Overflow provides functional yet insecure code examples Published 2016

  25. Android devs provided with: ◦ a skeleton app ◦ 4

    tasks in random order: 1. Secure Networking 2. Secure Storage 3. Inter-Component Communication 4. Least Permissions ◦ 20-30 minutes to complete each task ◦ An exit survey Split into 4 groups, could only refer to: ◦ Stack Overflow ◦ Official documentation ◦ Books ◦ All of the above The experiment
  26. The results? 1. Functional results 2. Secure results

  27. Functional results Devs assigned Stack Overflow and books performed best.

    Devs assigned official docs performed worst. Resources Success Rate Stack Overflow 67.3% Book 66.1% Official docs 40.4% Free choice 51.8%
  28. Developers self-assessed their solutions to be secure when they mostly

    were not. Secure results Task Success Rate Confidence Least Permissions 87% 22% Secure Networking 38% 79% Inter Component Comms 38% 70% Secure Storage 38% 68%
  29. Android developers who use Stack Overflow get functional code quicker,

    but are more likely to produce insecure code. What this means
  30. Devs are unlikely to stop using Stack Overflow, because they

    get functional code. ⬇ What this means Developers mostly focus on functionality and performance.
  31. “it is critical to develop documentation and resources that combine

    the usefulness of forums like Stack Overflow with the security awareness of books or official API documents” What this means Acar et al. (2016)
  32. What you can do 1. Regularly spend time looking at

    Stack Overflow questions about your library 2. Identify gaps in documentation being filled by Stack Overflow 3. Fill the gaps 4. Post links in follow up comments on accepted Stack Overflow answers
  33. TL;DR, Stack Overflow answers provide functional, insecure code; developers are

    bad at assessing whether that code is secure
  34. ✨ side quest ✨ Published 2021

  35. “Our results show that insecure code ends up in the

    top results and is clicked on more often.” “There is at least a 22.8% chance that one out of the top three Google Search results [for Stack Overflow] leads to insecure code”
  36. None
  37. “Participants that used our modified search engine to look for

    help online submitted more secure and functional results, with statistical significance”
  38. TL;DR, Novel solution to the problem that requires zero behaviour

    change from developers; can someone in the audience action this?
  39. Most “cryptography bugs” come from the use of libraries, not

    the libraries themselves Published 2014
  40. Analysis of 269 “Cryptographic Issues” (CWE310) in CVE database from

    Jan 2011 to May 2014 The study Categorised vulnerabilities by: Impact ◦ Plaintext disclosure ◦ Manipulator-In-The-Middle ◦ Brute-force ◦ Side-channel Layer ◦ Primitive (like AES, DH) ◦ Protocol (library or abstraction) ◦ Application (everything else)
  41. The result? were in the app layer } 100% of

    plaintext disclosures 86% of MITM 79% of brute-force 0% of side-channel
  42. ◦ Expose interfaces in your libraries that are misuse resistant.

    What you can do
  43. 83% of cryptographic bugs come from improper uses of cryptographic

    libraries. What this means
  44. Features and documentation are the #1 requirement for devs to

    use your library securely Published 2017
  45. ◦ Python devs assigned five tasks with stub code: 1.

    Symmetric: key generation 2. Symmetric: encryption/decryption 3. Asymmetric: key generation 4. Asymmetric: encryption/decryption 5. Symmetric: certificate validation ◦ An exit interview — “is your code secure?” Assigned one of five Python crypto libs 1. cryptography.io🤞 2. Keyczar🤞 3. PyNaCl🤞 4. M2Crypto 5. PyCrypto Asked to only use included docs. The experiment
  46. Modified Jupyter Notebook to: ◦ Snapshot the code ◦ Detect

    and store copy/paste events The experiment
  47. The results? 1. Functional results 2. Secure results

  48. Wide variation of task success across libraries. Asymmetric tasks less

    successful. Functional results
  49. Copy-paste code 3× more likely to be functional on symmetric

    tasks. Functional results
  50. Python experience, security background, or library experience did not change

    success. Functional results
  51. Asymmetric tasks 3× more likely to produce a secure solution.

    Secure results
  52. Keyczar usage 25× more likely to be secure. But only

    10% of submitted Keyczar solutions were functional. Secure results
  53. 20% of submitted code was insecure, but the developers strongly

    believed it was secure. Secure results
  54. Libraries with simplified interfaces produced more secure results. And yet,

    security success rate <80%, even on simplified libraries. What it means
  55. What it means Peripheral features (like secure key storage &

    password-based key generation) make-or-break security. You can’t rely on the dev to identify danger, nor find secure alternatives.
  56. You can never have too many code examples. If devs

    can’t find working code examples in your docs, they will go to Stack Overflow and find a footgun. What it means
  57. What you can do 1. Create and maintain an overabundance

    of secure code examples for your library 2. Identify inputs and dependencies your users implicitly rely on, and either: ◦ Provide secure wrappers for them ◦ Document them exhaustively
  58. TL;DR, Misuse resistance helps but it ain’t enough; devs crave

    code examples — you need to provide them
  59. Put runtime warnings into your API for potentially insecure usage

    Published 2018
  60. Python devs assigned: ◦ Three tasks with stub code: 1.

    Symmetric: key generation 2. Symmetric: encryption 3. Key storage ◦ Control or patch version of PyCrypto An exit interview — “is your code secure?” Patched PyCrypto? Print to stdout if created object is an insecure cryptography feature. The experiment
  61. The results Condition Functional success Secure success Control 90% 27%

    Patch 86% 51% Devs shown warnings produced secure solutions at nearly twice the rate. Devs self-assessed their solutions as more secure when given the warning. Devs rated control + patched PyCrypto equally usable.
  62. The results Devs who wrote code that triggered a warning

    were 15× more likely to convert it to a secure solution
  63. Warnings need these 5 things 1. Title message 2. Colour

    3. Source code location 4. Link to external resources 5. Message classification
  64. You don’t have to change your API’s shape to increase

    secure usage. Emitting runtime warnings significantly improves secure usage. What it means
  65. “we recommend software that interface with the developer (IDEs, text

    editors, compilers, etc) prime developers on the spot when they need it: while coding” ⬇ ⬇ ⬇
  66. What you can do 1. Make your library emit warnings

    when you detect potentially insecure usage 2. Provide an obvious mechanism to silence warnings
  67. TL;DR, Use runtime warnings to prime developers about insecure usage

    during dev
  68. Takeaways

  69. Developers don’t prioritise security (you have to do the hard

    work for them)
  70. You can help developers prioritise by providing: ◦ Targeted warnings

    in your APIs ◦ Frictionless, misuse-resistant APIs that work and are secure ◦ Copious secure code examples in your documentation
  71. Most security bugs happen on the periphery of your cryptography

    library, inside your users’ applications
  72. Your challenge: do the hard work to make it easy

    for your users
  73. Thank you! 🤔 What questions do you have? Like the

    talk? Let @auxesis know on Twitter. Slides at cipherstash.com/lindsay Come help us build next gen cryptography at CipherStash!
  74. References Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C.,

    Cappos, J., & Zhuang, Y. (2014, December). It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305). [PDF] Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP) (pp. 289-305). IEEE. [PDF] Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (pp. 3070-3084). [PDF] Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7). [PDF] Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the usability of cryptographic apis. In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE. [PDF] Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve security warnings, too: On the effect of integrated security advice on cryptographic {API} misuse. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281). [PDF] Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13). [PDF]
  75. Further reading ◦ API Blindspots: Why Experienced Developers Write Vulnerable

    Code [PDF] ◦ Blindspots in Python and Java APIs Result in Vulnerable Code [PDF] ◦ What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings [PDF] ◦ I Do and I Understand. Not Yet True for Security APIs. So Sad [PDF] ◦ How Usable are Rust Cryptography APIs? [PDF] ◦ You Really Shouldn’t Roll Your Own Crypto: An Empirical Study of Vulnerabilities in Cryptographic Libraries [PDF]
  76. Appendix

  77. Developers who were primed identified security problems at almost twice

    the rate of un-primed developers. Appendix It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and How Priming Can Illuminate Developer’s Blind Spot
  78. Some types of vulnerabilities were more familiar to developers than

    others. Appendix It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and How Priming Can Illuminate Developer’s Blind Spot
  79. Stack Overflow provides functional, insecure code Published 2016

  80. A study in two parts 1. Survey developers about what

    information sources they use 2. Study developers when given security challenges with different information sources
  81. 1. The Survey ◦ most devs use search engines and

    Stack Overflow ◦ a lot also consult the official API documentation ◦ a few use books
  82. 2. The Study Android devs assigned to one of four

    study groups: 1. Stack Overflow only 2. Official documentation only 3. Books only 4. Free choice
  83. Android devs provided with: ◦ a skeleton app ◦ 4

    tasks in random order ◦ 20-30 minutes to complete each task ◦ An exit interview 2. The Study
  84. 2. The Study The four tasks: 1. Secure networking 2.

    Secure storage 3. Inter-Component Communication 4. Least permissions
  85. Results? 1. Function 2. Security

  86. Results: function "Our results demonstrate that the assigned resource condition

    had a notable impact on participants’ ability to complete the tasks functionally correctly"
  87. Results: function ◦ Stack Overflow and book participants performed best

    ◦ Official doc participants performed worst Condition Success Rate SO 67.3% Book 66.1% Official Docs 40.4% Free 51.8%
  88. Results: function Self-assessment binning: ◦ Confident == strongly agree/agree ◦

    Not Confident == strongly disagree/disagree/neutral Task Confidence Least Permissions 81.1% Secure Networking 20.1% ICC 40.7% Secure Storage 53.7%
  89. Results: security “Our results suggest that choice of resources has

    the opposite effect on security than it did on functionality”
  90. Results: security ◦ Official Docs and Book participants performed best

    ◦ Stack Overflow participants performed worst Condition Success Rate SO 51.4% Book 73.0% Official Docs 85.7% Free 65.5%
  91. Results: security Self-assessment binning: ◦ Confident == strongly agree/agree ◦

    Not Confident == strongly disagree/disagree/neutral Task Confidence Least Permissions 22.2% Secure Networking 79.6% ICC 70.4% Secure Storage 68.5%
  92. What this means ◦ “using Stack Overflow helps Android developers

    to arrive at functional solutions more quickly than with other resources” ◦ “Because Stack Overflow contains many insecure answers, Android developers who rely on this resource are likely to create less secure code.” ◦ “developers are unlikely to give up using resources that help them quickly address their immediate problems”
  93. What this means ◦ “it is critical to develop documentation

    and resources that combine the usefulness of forums like Stack Overflow with the security awareness of books or official API documents” ◦ “Stack Overflow could add a mechanism for explicitly rating the security of provided answers and weighting those rated secure more heavily in search results and thread ordering”
  94. Most “cryptography bugs” come from the use of libraries, not

    the libraries themselves Published 2014
  95. Analysis of 269 “Cryptographic Issues” (CWE310) in CVE database from

    Jan 2011 to May 2014 The study Categorised vulnerabilities by: Impact ◦ Plaintext disclosure ◦ Manipulator-In-The-Middle ◦ Brute-force ◦ Side-channel Layer ◦ Primitive (like AES, DH) ◦ Protocol (library or abstraction) ◦ Application (everything else)
  96. 100% of plaintext disclosures were in the app layer 86%

    of MITM were in the app layer 79% of brute-force were in the app layer 0% of side-channel were in the app layer The result?