Tales from Python Core Security (ConFoo Montreal 2017)

https://confoo.ca/en/yul2017/session/tales-from-python-core-security

Transcript

1. Tales from Python Core Security ConFoo Montreal 2017 Christian Heimes christian@python.org / @ChristianHeimes Senior Software Engineer 2017-03-10
2. 2 Who am I • from Hamburg / Germany • Linux user since 1998 • Python developer since 2002 • Python core contributor since 2008
3. 3 Python • PEP 370: Per user site-packages directory • PEP 452: API for Cryptographic Hash Functions v2.0 • PEP 456: Secure and interchangeable hash algorithm • PEP 543: Unified TLS API (Cory Benfield) • maintainer of ssl and hashlib modules • maintainer of Coverity Scan • Python Security Response Team (emeritus) • I put bytes and b'' into Python 2
4. 4 Professional Life • Senior Software Engineer with Red Hat • Security Engineering & Identity Management • FreeIPA IdM • Dogtag PKI • Custodia
5. 5 FreeIPA/IdM KDC LDAP PKI DNS CLI/UI
6. 6 Professionally paranoid security engineer
7. Topics
8. 8 Topics • What does 'secure' mean? Why is it so hard? • Security efforts • Examples of CPython security issues • How to write secure software • from __future__ import ...
9. Hall Of Fame
10. 10 Python Security People • Alex Gaynor • Antoine Pitrou • Ashwini Oruganti • Benjamin Peterson • Cory Benfield • Gregory P. Smith • Hynek Schlawack • lvh • Paul Kehrer • Serhiy Storchaka • Victor Stinner
11. 11 Donald Stufft donate.pypi.io
12. Why is security relevant?
13. What can go wrong?
14. 14 Denial of Service (direct or indirect)
15. 15 bypass of authentication authorization access control
16. 16 data breach information disclosure
17. 17 code execution
18. 18 cryptography
19. Security is hard
20. 20 weakest link causes catastrophic failure
21. 21 untestable almost unprovable
22. 22 demo exploits are hard to write
23. 23 security fix = breaking backwards compatibility
24. 24 Different workflow for security fixes (issue tracker, reviews, testing, QA)
25. 25 crazy attack vectors side channel attacks
26. Python security efforts
27. 27 Motivation Culture Diversity
28. 28 Reviews • PEP process • design reviews • code reviews • commit reviews • Did I mention reviews?
29. 29 Collaboration • Upstream vendor security teams • Ruby and PHP security • Python Cryptography Authority (PyCA) • security people (Ryan Sleevi, DJB)
30. 30 Tools • multiple compilers and platforms • compiler warnings • clang code analysis • fuzzing • Coverity Scan • python-security.readthedocs.io
33. In the beginning...
37. 37 XML <xml> <tag attribute=”value”>text</tag> </xml> <xml> <tag attribute=”value”>text</tag> </xml>
38. 38 XML entities <!DOCTYPE example [ <!ENTITY title "My title" > ]> <xml> <tag attribute=”value”>&title;</tag> </xml> <!DOCTYPE example [ <!ENTITY title "My title" > ]> <xml> <tag attribute=”value”>&title;</tag> </xml>
39. 39 XML entities expansion attack <!DOCTYPE xmlbomb [ <!ENTITY a "1234567890" > <!ENTITY b "&a;&a;&a;&a;&a;&a;&a;&a;"> <!ENTITY c "&b;&b;&b;&b;&b;&b;&b;&b;"> <!ENTITY d "&c;&c;&c;&c;&c;&c;&c;&c;"> ]> <bomb>&d;</bomb> <!DOCTYPE xmlbomb [ <!ENTITY a "1234567890" > <!ENTITY b "&a;&a;&a;&a;&a;&a;&a;&a;"> <!ENTITY c "&b;&b;&b;&b;&b;&b;&b;&b;"> <!ENTITY d "&c;&c;&c;&c;&c;&c;&c;&c;"> ]> <bomb>&d;</bomb>
40. 40 XML network / file access <!DOCTYPE external [ <!ENTITY remote SYSTEM "http://www.python.org/some.xml"> <!ENTITY local SYSTEM "file:///etc/passwd"> ]> <xml> <url>&remote;</file> <file>&local;</file> </xml> <!DOCTYPE external [ <!ENTITY remote SYSTEM "http://www.python.org/some.xml"> <!ENTITY local SYSTEM "file:///etc/passwd"> ]> <xml> <url>&remote;</file> <file>&local;</file> </xml>
41. 41 XML attacks • billion laughs / exponential entity expansion • quadratic blowup entity expansion • DTD & external entity expansion (remote and local) • attribute blowup / attribute hash collision attack • decompression bomb (gzip) • XPath injection attacks • XInclude <xi:include /> • XMLSchema-Import <xs:import /> • XSLT features wie xalan/redirect, xalan/java
42. 42 Defusing XML • defusedxml • defusedexpat YAML has some fancy features, too.
43. Reading and parsing data
44. 44 HTTP RFC822 header content-type: text/html; charset=utf-8 content-length: 47446 x-clacks-overhead: GNU Terry Pratchett <html> <head> ... content-type: text/html; charset=utf-8 content-length: 47446 x-clacks-overhead: GNU Terry Pratchett <html> <head> ...
45. 45 HTTP header parsing sock = create_connection(('host', 80)) f = sock.makefile() for line in f: name, value = line.split(':', 1) ... sock = create_connection(('host', 80)) f = sock.makefile() for line in f: name, value = line.split(':', 1) ...
46. 46 HTTP parsing (fixed) MAX_LENGTH = 1024 sock = create_connection((host, port)) f = sock.makefile() while True: line = f.readline(MAX_LENGTH + 1) if len(line) > MAX_LENGTH: raise ValueError ... MAX_LENGTH = 1024 sock = create_connection((host, port)) f = sock.makefile() while True: line = f.readline(MAX_LENGTH + 1) if len(line) > MAX_LENGTH: raise ValueError ...
47. 47 CVE-2013-1752 • Issue #16037 httplib (Adrien Kunysz) • Issue #16038 ftplib • Issue #16039 imaplib • Issue #16040 nntplib • Issue #16041 poplib • Issue #16042 smtplib
48. 48 Header injection http://127.0.0.1%0d%0aX-injected:%20header %0d%0ax-leftover:%20:12345/foo GET /foo HTTP/1.1 Host: 127.0.0.1 X-injected: header x-leftover: :12345 >>> socket.getaddrinfo( '127.0.0.1\r\n', 12345)[0][4] ('127.0.0.1', 12345) http://127.0.0.1%0d%0aX-injected:%20header %0d%0ax-leftover:%20:12345/foo GET /foo HTTP/1.1 Host: 127.0.0.1 X-injected: header x-leftover: :12345 >>> socket.getaddrinfo( '127.0.0.1\r\n', 12345)[0][4] ('127.0.0.1', 12345)
49. Hash collision attacks PEP 456
50. 50 Python's dict type • dicts are implemented as hash maps • closed hashing • open addressing • average search and insert: O(1) • worst case: O(n)
51. 51 Hash map primer >>> hash('de') 12800076900115529 >>> hash('de') & (8 - 1) 1 >>> hash('de') 12800076900115529 >>> hash('de') & (8 - 1) 1 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 de
52. 52 Hash map primer (2) >>> hash('df') 6672104196504639850 >>> hash('df') & (8 - 1) 2 >>> hash('df') 6672104196504639850 >>> hash('df') & (8 - 1) 2 0 1 2 3 4 5 6 7 de df
53. 53 Hash collision >>> hash('cf') & (8 - 1) 1 >>> hash('bg') & (8 - 1) 1 >>> hash('cf') & (8 - 1) 1 >>> hash('bg') & (8 - 1) 1 0 1 2 3 4 5 6 7 de cf bg df
54. 54 PEP 456 SipHash24 • First attempt by Victor Stinner failed • Alternatives were slow and/or insecure • SipHash24 by Jean-Philippe Aumasson and Daniel J. Bernstein
55. ssl.match_hostname()
56. 56 TLS / SSL • encrypts communication • protects data integrity • prevents reply attacks • authenticate server with X.509 certificate
57. 57 TLS handshake 1) Client and server negotiate connection parameters. • protocol version • SSL 3, TLS 1.0, TLS 1.1, TLS 1.2 • cipher suit • TLS_RSA_WITH_AES_128_CBC_SHA • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 2) Client verifies server cert. 3) Client and server agree on a master secret.
58. 58 Hostnames in X.509 certificates Certificate: Data: Issuer: C=IL, O=StartCom Ltd., ... Validity Not Before: Feb 2 08:35:11 2015 GMT Not After : Feb 1 21:03:59 2017 GMT Subject: C=US, ST=Oregon, L=Beaverton, O=Python Software Foundation, CN=*.python.org X509v3 extensions: X509v3 Subject Alternative Name: DNS:*.python.org, DNS:python.org Certificate: Data: Issuer: C=IL, O=StartCom Ltd., ... Validity Not Before: Feb 2 08:35:11 2015 GMT Not After : Feb 1 21:03:59 2017 GMT Subject: C=US, ST=Oregon, L=Beaverton, O=Python Software Foundation, CN=*.python.org X509v3 extensions: X509v3 Subject Alternative Name: DNS:*.python.org, DNS:python.org
59. 59 Bug #0 No function to verify a hostname • ssl.match_hostname() added in Python 3.2 • backported to 2.7.9
60. 60 Bug #1 (Issue #12000) Use CN when subject alternative name field has no dNSName entries. • fixed in 3.3
61. 61 Bug #2 (CVE-2013-2099) Multiple wildcards cause Denial-of-Service • *.*.*.*.*.example.org • caused by regular expression engine • fixed in 3.3.3
62. 62 Bug #3 (Issue #17997) Use RFC 6125 instead of outdated RFC 2818 • only one left-most wildcard is allowed • *.*.python.org or www.*.python.org are forbidden • fixed in 3.3.3
63. 63 Bug #4 (Issue #17997) Fix wildcard matching for internationalized domain names (IDN) • über.python.org → xn--ber-goa.python.org • fixed in 3.3.3 • Mozilla NSS / Firefox and OpenSSL 1.0.2dev were affected, too.
64. 64 Bug #5 (Issue #18709) NULL bytes inside subjectAltNames general names, e.g. www.python.org\x00evil.com • fixed in 3.4 • Ruby and PHP were also affected
65. 65 Bug #6 (too many CVEs) Python does not check hostname by default • PEP 476 (Alex Gaynor) • fixed for httplib/urllib in 3.4 and 2.7.9 • still not fixed for imap, pop, smtp etc.
66. 66 Bug #7 (issue #17305) Python does not support IDNA 2008 • critical for German domains like straße.de • IDNA 2003: strasse.de • Chrome, Edge, Konqueror • IDNA 2008: xn--strae-oqa.de • Firefox • still not fixed
67. 67 'Руthοn' != 'Python'
68. 68 homoglyphic confusion attack >>> import unicodedata >>> for c in 'Руthοn': ... print(unicodedata.name(c)) ... CYRILLIC CAPITAL LETTER ER CYRILLIC SMALL LETTER U LATIN SMALL LETTER T LATIN SMALL LETTER H GREEK SMALL LETTER OMICRON LATIN SMALL LETTER N >>> import unicodedata >>> for c in 'Руthοn': ... print(unicodedata.name(c)) ... CYRILLIC CAPITAL LETTER ER CYRILLIC SMALL LETTER U LATIN SMALL LETTER T LATIN SMALL LETTER H GREEK SMALL LETTER OMICRON LATIN SMALL LETTER N
69. PBKDF2 Password based key derivation function
71. How to write secure software
72. 72 Think first! • Read, understand and use open standards • Learn from mistakes • Document security claims and trade-offs • Design software for human • Default to secure and easy to use • Defense in depths
73. 73 Does • Use frameworks • Django, Flask, Plone, Pyramid… • sqlalchemy • PyCA cryptography • passlib • Use TLS/SSL (Let's Encrypt, FreeIPA)
74. 74 ssl.CERT_REQUIRED verify_hostname = True
75. 75 Here be dragons • Beware of dangerous Python features • exec(), eval(), import • pickle, marshal • random module • Don't roll your own crypto!
76. 76 pip install safety
77. Sane and secure defaults settings
78. 78 2 PEPs for ssl work-in-progress Supported OpenSSL version Secure default settings
79. 79 PEP 446 by Victor Stinner Make newly created file descriptors non-inheritable
80. 80 PEP 524 by Victor Stinner Make os.urandom() blocking on Linux
81. 81 Random Number Generator • random.random() • /dev/random • /dev/urandom • getrandom() Linux syscall • getentropy() BSD syscall • CryptGenRandom() Work-in-progress: osrandom_engine
82. 82 Python 3.6 • PEP 524 os.urandom() • hashlib • blake2, sha3, scrypt • ssl • OpenSSL 1.1.0 • sessions • get cipher suite • AF_ALG Linux Kernel Crypto
83. 83 SHA-ttered
84. Future
85. 85 Future • TLS 1.3 • Hardware security (Intel MPX, HSM, TPM 2.0) • 2FA • containers • IoT (internet of things) • Quantum Computers
86. Questions? christian@python.org @ChristianHeimes
87. THANK YOU plus.google.com/+RedHat linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHatNews