Footguns and factorisation: how to make users of your cryptographic library successful

Video of this talk.

Abstract

Cryptography forms the backbone of how we securely use information online, but most developers don’t have more than a surface level understanding of cryptography.

Shannon's maxim states that “one ought to design systems under the assumption that the enemy will immediately gain full familiarity with them”. Open source makes this feasible for cryptography, with open source cryptographic libraries handling a huge proportion of information on the internet in flight and at rest.

Developers place a lot of trust in the authors of these libraries to get the cryptography engineering right.

But when basic usability issues result in developers using the libraries incorrectly, that trust and painstaking cryptography engineering can be for naught. Worse still, developers often believe they have used the libraries to build something that is secure. But that belief is often mistaken — their use of these libraries is actually insecure.

In this talk, attendees will learn:

1. What research says about how the usability of cryptographic libraries impacts the ability of users to deliver code that handles data securely
2. What common usability traps open source cryptography projects fall into
3. How authors, maintainers, and communities around open source cryptographic library can make their users successful

References

Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C., Cappos, J., & Zhuang, Y. (2014, December). It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305).

Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP) (pp. 289-305). IEEE.

Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (pp. 3070-3084).

Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7).

Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the usability of cryptographic APIs In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE.

Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve security warnings, too: On the effect of integrated security advice on cryptographic API misuse. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281).

Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

Further reading

API Blindspots: Why Experienced Developers Write Vulnerable Code

Blindspots in Python and Java APIs Result in Vulnerable Code

What Do We Really Know about How Habituation to Warnings Occurs Over Time? A Longitudinal fMRI Study of Habituation and Polymorphic Warnings

I Do and I Understand. Not Yet True for Security APIs. So Sad

How Usable are Rust Cryptography APIs?

You Really Shouldn’t Roll Your Own Crypto: An Empirical Study of Vulnerabilities in Cryptographic Libraries

Transcript

1. Footguns and
   factorisation
   How to make users of your
   cryptographic library successful.
   Lindsay Holmwood
   @auxesis
   

   View Slide

2. View Slide

3. In 2020, over 300,000 patient records —
   including detailed consult notes — were
   leaked and used to extort Vastaamo patients.
   

   View Slide

4. At around 4 pm, Jere checked Snapchat.
   An email notification popped up on his
   screen. His hands began to shake. The
   subject line included his full name, his
   social security number, and the name of
   a clinic where he’d gotten mental health
   treatment as a teenager: Vastaamo.
   They Told Their Therapists Everything. Hackers Leaked It All
   William Ralston, Wired
   

   View Slide

5. View Slide

6. Vastaamo’s system violated one of the
   “first principles of cybersecurity”: It
   didn’t anonymize the records. It didn’t
   even encrypt them. The only thing
   protecting patients’ confessions and
   confidences were a couple of firewalls
   and a server login screen.
   They Told Their Therapists Everything. Hackers Leaked It All
   William Ralston, Wired
   

   View Slide

7. “I would never do this”
   

   View Slide

8. 20% of submitted code was
   insecure,
   but the developers strongly
   believed it was secure.
   

   View Slide

9. It ain’t what you don’t know
   that gets you into trouble.
   It’s what you know for sure
   that just ain’t so.
   Mark Twain
   

   View Slide

10. Human Factors in Secure Software Development:
    There is a lot of academic
    research into cryptography
    usability and developer
    behaviour.
    

    View Slide

11. Let’s take a look at what the
    research says about:
    1. What are the usability traps in
    cryptography libraries
    2. What you can do to help your
    users avoid them
    

    View Slide

12. Before we begin…
    

    View Slide

13. We’re talking about
    cryptography,
    not cryptocurrency
    

    View Slide

14. We’re talking
    about developers,
    not end users
    

    View Slide

15. We’re looking at peer
    reviewed research,
    much of it replicated
    

    View Slide

16. Developers don’t
    think about security
    Published 2014
    

    View Slide

17. Developers shown 6 code examples that contained
    vulnerabilities covering:
    ○ TLS setup
    ○ Time-Of-Check-To-Time-Of-Use
    ○ Brute force exhaustion
    ○ Buffer overflows
    ○ XSS
    ○ SQL injection
    The experiment
    

    View Slide

18. Developers grouped, prompted before each example:
    The experiment
    Question Group
    What is the user input to this program? Control, Priming
    What happens when this code executes? Control, Priming
    Could a developer experience unexpected results
    when running such code?
    Priming
    What could be examples of these unexpected results
    and where do they appear in the code?
    Priming
    This code has a vulnerability.
    Can you pinpoint the problem?
    Explicit
    

    View Slide

19. The result?
    Developers who were primed
    identified security problems
    at almost twice the rate
    of un-primed developers.
    

    View Slide

20. “security is not part of the
    heuristics used by developers in
    their daily programming tasks“
    Developers mostly focus on
    functionality and performance.
    What it means
    

    View Slide

21. What it means
    We think that developers should
    learn about security, and apply
    what they learn when they need it.
    This is not how humans
    actually think.
    

    View Slide

22. “we recommend software that
    interface with the developer
    (IDEs, text editors, compilers, etc)
    prime developers on the spot
    when they need it: while coding”
    What it means
    Oliviera et al. (2014)
    

    View Slide

23. TL;DR,
    devs don’t think about
    security,
    priming increases the
    likelihood of discovering
    security bugs
    

    View Slide

24. Stack Overflow provides
    functional yet insecure
    code examples
    Published 2016
    

    View Slide

25. Android devs provided with:
    ○ a skeleton app
    ○ 4 tasks in random order:
    1. Secure Networking
    2. Secure Storage
    3. Inter-Component
    Communication
    4. Least Permissions
    ○ 20-30 minutes
    to complete each task
    ○ An exit survey
    Split into 4 groups,
    could only refer to:
    ○ Stack Overflow
    ○ Official documentation
    ○ Books
    ○ All of the above
    The experiment
    

    View Slide

26. The results?
    1. Functional results
    2. Secure results
    

    View Slide

27. Functional results
    Devs assigned Stack Overflow and books performed best.
    Devs assigned official docs performed worst.
    Resources Success Rate
    Stack Overflow 67.3%
    Book 66.1%
    Official docs 40.4%
    Free choice 51.8%
    

    View Slide

28. Developers self-assessed their solutions
    to be secure when they mostly were not.
    Secure results
    Task Success Rate Confidence
    Least Permissions 87% 22%
    Secure Networking 38% 79%
    Inter Component Comms 38% 70%
    Secure Storage 38% 68%
    

    View Slide

29. Android developers who use Stack
    Overflow get functional code
    quicker, but are more likely to
    produce insecure code.
    What this means
    

    View Slide

30. Devs are unlikely to stop using
    Stack Overflow, because they get
    functional code. ⬇
    What this means
    Developers mostly focus on functionality and performance.
    

    View Slide

31. “it is critical to develop
    documentation and resources that
    combine the usefulness of forums
    like Stack Overflow
    with the security awareness of
    books or official API documents”
    What this means
    Acar et al. (2016)
    

    View Slide

32. What you can do
    1. Regularly spend time looking at Stack
    Overflow questions about your library
    2. Identify gaps in documentation being
    filled by Stack Overflow
    3. Fill the gaps
    4. Post links in follow up comments on
    accepted Stack Overflow answers
    

    View Slide

33. TL;DR,
    Stack Overflow answers
    provide functional,
    insecure code;
    developers are bad at
    assessing whether that
    code is secure
    

    View Slide

34. ✨ side quest ✨
    Published 2021
    

    View Slide

35. “Our results show that insecure code
    ends up in the top results and is
    clicked on more often.”
    “There is at least a 22.8% chance
    that one out of the top three Google
    Search results [for Stack Overflow]
    leads to insecure code”
    

    View Slide

36. View Slide

37. “Participants that used our modified
    search engine to look for help online
    submitted more secure and
    functional results, with statistical
    significance”
    

    View Slide

38. TL;DR,
    Novel solution to the problem
    that requires zero behaviour
    change from developers;
    can someone in the
    audience action this?
    

    View Slide

39. Most “cryptography bugs”
    come from the use of libraries,
    not the libraries themselves
    Published 2014
    

    View Slide

40. Analysis of 269 “Cryptographic Issues” (CWE310)
    in CVE database from Jan 2011 to May 2014
    The study
    Categorised vulnerabilities by:
    Impact
    ○ Plaintext disclosure
    ○ Manipulator-In-The-Middle
    ○ Brute-force
    ○ Side-channel
    Layer
    ○ Primitive
    (like AES, DH)
    ○ Protocol
    (library or abstraction)
    ○ Application
    (everything else)
    

    View Slide

41. The result?
    were in the
    app layer
    }
    100% of plaintext
    disclosures
    86% of MITM
    79% of brute-force
    0% of side-channel
    

    View Slide

42. ○ Expose interfaces in your libraries
    that are misuse resistant.
    What you can do
    

    View Slide

43. 83% of cryptographic bugs
    come from improper uses of
    cryptographic libraries.
    What this means
    

    View Slide

44. Features and documentation
    are the #1 requirement for devs
    to use your library securely
    Published 2017
    

    View Slide

45. ○ Python devs assigned
    five tasks with stub code:
    1. Symmetric:
    key generation
    2. Symmetric:
    encryption/decryption
    3. Asymmetric:
    key generation
    4. Asymmetric:
    encryption/decryption
    5. Symmetric: certificate
    validation
    ○ An exit interview — “is
    your code secure?”
    Assigned one of five
    Python crypto libs
    1. cryptography.io🤞
    2. Keyczar🤞
    3. PyNaCl🤞
    4. M2Crypto
    5. PyCrypto
    Asked to only use included docs.
    The experiment
    

    View Slide

46. Modified Jupyter Notebook to:
    ○ Snapshot the code
    ○ Detect and store copy/paste events
    The experiment
    

    View Slide

47. The results?
    1. Functional results
    2. Secure results
    

    View Slide

48. Wide variation of task success across libraries.
    Asymmetric tasks less successful.
    Functional results
    

    View Slide

49. Copy-paste code 3× more likely to be functional
    on symmetric tasks.
    Functional results
    

    View Slide

50. Python experience,
    security background, or
    library experience
    did not change success.
    Functional results
    

    View Slide

51. Asymmetric tasks 3× more likely to produce a secure solution.
    Secure results
    

    View Slide

52. Keyczar usage 25× more likely to be secure.
    But only 10% of submitted Keyczar solutions were functional.
    Secure results
    

    View Slide

53. 20% of submitted code was insecure,
    but the developers strongly
    believed it was secure.
    Secure results
    

    View Slide

54. Libraries with simplified
    interfaces produced more
    secure results.
    And yet, security success rate <80%,
    even on simplified libraries.
    What it means
    

    View Slide

55. What it means
    Peripheral features
    (like secure key storage & password-based key generation)
    make-or-break security.
    You can’t rely on the dev
    to identify danger, nor
    find secure alternatives.
    

    View Slide

56. You can never have too many
    code examples.
    If devs can’t find working code
    examples in your docs,
    they will go to Stack Overflow
    and find a footgun.
    What it means
    

    View Slide

57. What you can do
    1. Create and maintain an
    overabundance of secure code
    examples for your library
    2. Identify inputs and dependencies your
    users implicitly rely on, and either:
    ○ Provide secure wrappers for them
    ○ Document them exhaustively
    

    View Slide

58. TL;DR,
    Misuse resistance helps
    but it ain’t enough;
    devs crave code
    examples — you need to
    provide them
    

    View Slide

59. Put runtime warnings
    into your API for
    potentially insecure usage
    Published 2018
    

    View Slide

60. Python devs assigned:
    ○ Three tasks with stub code:
    1. Symmetric:
    key generation
    2. Symmetric:
    encryption
    3. Key storage
    ○ Control or patch version of
    PyCrypto
    An exit interview — “is
    your code secure?”
    Patched PyCrypto?
    Print to stdout
    if created object
    is an insecure
    cryptography
    feature.
    The experiment
    

    View Slide

61. The results
    Condition Functional success Secure success
    Control 90% 27%
    Patch 86% 51%
    Devs shown warnings produced secure solutions
    at nearly twice the rate.
    Devs self-assessed their solutions as more secure
    when given the warning.
    Devs rated control + patched PyCrypto equally usable.
    

    View Slide

62. The results
    Devs who wrote code that
    triggered a warning
    were 15× more likely to
    convert it to a secure solution
    

    View Slide

63. Warnings need these 5 things
    1. Title message
    2. Colour
    3. Source code location
    4. Link to external resources
    5. Message classification
    

    View Slide

64. You don’t have to change
    your API’s shape
    to increase secure usage.
    Emitting runtime warnings significantly
    improves secure usage.
    What it means
    

    View Slide

65. “we recommend software that
    interface with the developer
    (IDEs, text editors, compilers, etc)
    prime developers on the spot
    when they need it: while coding”
    ⬇ ⬇ ⬇
    

    View Slide

66. What you can do
    1. Make your library emit warnings
    when you detect potentially
    insecure usage
    2. Provide an obvious mechanism to
    silence warnings
    

    View Slide

67. TL;DR,
    Use runtime warnings to
    prime developers about
    insecure usage during dev
    

    View Slide

68. Takeaways
    

    View Slide

69. Developers don’t
    prioritise security
    (you have to do the hard work for them)
    

    View Slide

70. You can help developers
    prioritise by providing:
    ○ Targeted warnings in your APIs
    ○ Frictionless, misuse-resistant
    APIs that work and are secure
    ○ Copious secure code examples in
    your documentation
    

    View Slide

71. Most security bugs
    happen on the periphery
    of your cryptography
    library, inside your users’
    applications
    

    View Slide

72. Your challenge:
    do the hard work
    to make it easy
    for your users
    

    View Slide

73. Thank you!
    🤔 What questions do
    you have?
    Like the talk?
    Let @auxesis know on Twitter.
    Slides at cipherstash.com/lindsay
    Come help us build next gen
    cryptography at CipherStash!
    

    View Slide

74. References
    Oliveira, D., Rosenthal, M., Morin, N., Yeh, K. C., Cappos, J., & Zhuang, Y. (2014, December). It's the psychology
    stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots. In
    Proceedings of the 30th Annual Computer Security Applications Conference (pp. 296-305). [PDF]
    Acar, Y., Backes, M., Fahl, S., Kim, D., Mazurek, M. L., & Stransky, C. (2016, May). You get where you're looking
    for: The impact of information sources on code security. In 2016 IEEE Symposium on Security and Privacy (SP)
    (pp. 289-305). IEEE. [PDF]
    Fischer, F., Stachelscheid, Y., & Grossklags, J. (2021, November). The Effect of Google Search on Software
    Security: Unobtrusive Security Interventions via Content Re-ranking. In Proceedings of the 2021 ACM SIGSAC
    Conference on Computer and Communications Security (pp. 3070-3084). [PDF]
    Lazar, D., Chen, H., Wang, X., & Zeldovich, N. (2014, June). Why does cryptographic software fail? A case study
    and open problems. In Proceedings of 5th Asia-Pacific Workshop on Systems (pp. 1-7). [PDF]
    Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017, May). Comparing the
    usability of cryptographic apis. In 2017 IEEE Symposium on Security and Privacy (SP) (pp. 154-171). IEEE. [PDF]
    Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018). Developers deserve
    security warnings, too: On the effect of integrated security advice on cryptographic {API} misuse. In Fourteenth
    Symposium on Usable Privacy and Security ({SOUPS} 2018) (pp. 265-281). [PDF]
    Gorski, P. L., Acar, Y., Lo Iacono, L., & Fahl, S. (2020, April). Listen to Developers! A Participatory Design Study
    on Security Warnings for Cryptographic APIs. In Proceedings of the 2020 CHI Conference on Human Factors in
    Computing Systems (pp. 1-13). [PDF]
    

    View Slide

75. Further reading
    ○ API Blindspots: Why Experienced Developers Write
    Vulnerable Code [PDF]
    ○ Blindspots in Python and Java APIs Result in Vulnerable
    Code [PDF]
    ○ What Do We Really Know about How Habituation to
    Warnings Occurs Over Time? A Longitudinal fMRI Study of
    Habituation and Polymorphic Warnings [PDF]
    ○ I Do and I Understand. Not Yet True for Security APIs. So
    Sad [PDF]
    ○ How Usable are Rust Cryptography APIs? [PDF]
    ○ You Really Shouldn’t Roll Your Own Crypto: An Empirical
    Study of Vulnerabilities in Cryptographic Libraries [PDF]
    

    View Slide

76. Appendix
    

    View Slide

77. Developers who
    were primed
    identified security
    problems
    at almost
    twice the rate
    of un-primed
    developers.
    Appendix
    It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and
    How Priming Can Illuminate Developer’s Blind Spot
    

    View Slide

78. Some types of
    vulnerabilities were
    more familiar to
    developers than
    others.
    Appendix
    It’s the Psychology Stupid: How Heuristics Explain Software Vulnerabilities and
    How Priming Can Illuminate Developer’s Blind Spot
    

    View Slide

79. Stack Overflow provides
    functional, insecure code
    Published 2016
    

    View Slide

80. A study in two parts
    1. Survey developers about what
    information sources they use
    2. Study developers when given
    security challenges with
    different information sources
    

    View Slide

81. 1. The Survey
    ○ most devs use search engines
    and Stack Overflow
    ○ a lot also consult the official API
    documentation
    ○ a few use books
    

    View Slide

82. 2. The Study
    Android devs assigned to one of four study groups:
    1. Stack Overflow only
    2. Official documentation only
    3. Books only
    4. Free choice
    

    View Slide

83. Android devs provided with:
    ○ a skeleton app
    ○ 4 tasks in random order
    ○ 20-30 minutes to complete each task
    ○ An exit interview
    2. The Study
    

    View Slide

84. 2. The Study
    The four tasks:
    1. Secure networking
    2. Secure storage
    3. Inter-Component Communication
    4. Least permissions
    

    View Slide

85. Results?
    1. Function
    2. Security
    

    View Slide

86. Results: function
    "Our results demonstrate that the assigned
    resource condition had a notable impact on
    participants’ ability to complete the tasks
    functionally correctly"
    

    View Slide

87. Results: function
    ○ Stack Overflow and book participants performed best
    ○ Official doc participants performed worst
    Condition Success Rate
    SO 67.3%
    Book 66.1%
    Official Docs 40.4%
    Free 51.8%
    

    View Slide

88. Results: function
    Self-assessment binning:
    ○ Confident == strongly agree/agree
    ○ Not Confident == strongly disagree/disagree/neutral
    Task Confidence
    Least
    Permissions
    81.1%
    Secure
    Networking
    20.1%
    ICC 40.7%
    Secure Storage 53.7%
    

    View Slide

89. Results: security
    “Our results suggest that choice of resources has
    the opposite effect on security than it did on
    functionality”
    

    View Slide

90. Results: security
    ○ Official Docs and Book participants performed best
    ○ Stack Overflow participants performed worst
    Condition Success Rate
    SO 51.4%
    Book 73.0%
    Official Docs 85.7%
    Free 65.5%
    

    View Slide

91. Results: security
    Self-assessment binning:
    ○ Confident == strongly agree/agree
    ○ Not Confident == strongly disagree/disagree/neutral
    Task Confidence
    Least
    Permissions
    22.2%
    Secure
    Networking
    79.6%
    ICC 70.4%
    Secure Storage 68.5%
    

    View Slide

92. What this means
    ○ “using Stack Overflow helps Android developers
    to arrive at functional solutions more quickly than
    with other resources”
    ○ “Because Stack Overflow contains many insecure
    answers, Android developers who rely on this
    resource are likely to create less secure code.”
    ○ “developers are unlikely to give up using
    resources that help them quickly address their
    immediate problems”
    

    View Slide

93. What this means
    ○ “it is critical to develop documentation and
    resources that combine the usefulness of forums
    like Stack Overflow with the security awareness of
    books or official API documents”
    ○ “Stack Overflow could add a mechanism for
    explicitly rating the security of provided answers
    and weighting those rated secure more heavily in
    search results and thread ordering”
    

    View Slide

94. Most “cryptography bugs”
    come from the use of libraries,
    not the libraries themselves
    Published 2014
    

    View Slide

95. Analysis of 269 “Cryptographic Issues” (CWE310)
    in CVE database from Jan 2011 to May 2014
    The study
    Categorised vulnerabilities by:
    Impact
    ○ Plaintext disclosure
    ○ Manipulator-In-The-Middle
    ○ Brute-force
    ○ Side-channel
    Layer
    ○ Primitive
    (like AES, DH)
    ○ Protocol
    (library or abstraction)
    ○ Application
    (everything else)
    

    View Slide

96. 100% of plaintext
    disclosures were in the
    app layer
    86% of MITM were in the
    app layer
    79% of brute-force were
    in the app layer
    0% of side-channel
    were in the app layer
    The result?
    

    View Slide