Codemotion Milan: Computational Randomness

Transcript

1. Computational Randomness: Controlled Chaos in an Organized Machine Amanda Sopkin @amandasopkin Milan | November 29 - 30, 2018
2. Who am I? @amandasopkin
3. Where are we headed? @amandasopkin
4. Why python? @amandasopkin
5. History of Cryptography @amandasopkin
6. 1900 BC: Cryptographic hieroglyphics @amandasopkin
8. 100 BC: Julius Caeser @amandasopkin
9. @amandasopkin
10. Bread and Butter of cryptography
11. 1500s: Vigenère’s system @amandasopkin
12. To encrypt, a table of alphabets can be used, termed a tabula recta, Vigenère square or Vigenère table. It has the alphabet written out 26 times in different rows, each alphabet shifted cyclically to the left compared to the previous alphabet, corresponding to the 26 possible Caesar ciphers. At different points in the encryption process, the cipher uses a different alphabet from one of the rows. The alphabet used at each point depends on a repeating keyword 1500s: Vigenère’s system
14. Kerchoff’s Principle: secrecy of key @amandasopkin
15. 1900s: Hebern’s system @amandasopkin
16. 1900s: Hebern’s system
17. 1900s: Hebern’s system
18. Letter frequencies Can be used to break a cipher
19. Letter frequencies
20. Letter frequencies Frequency of each letter in cipher Sorted from most common to least
21. Letter frequencies Standard english letter frequencies
22. Letter frequencies Frequencies of letters in cipher
23. Letter frequencies Most frequent letters in cipher are S and O
24. Substitute E and T for S and O
25. Spot instances of “tle”
26. Most common letter is now G, which must be a, i, or o
27. Spot oFe and theF and then Lheet
28. Spot sODVe and OK
29. Sub l for D, v for V, R for K
30. Spot enoQRh
31. Spot EesMge and Nount
32. Few more steps...
33. Solved!
34. Back to history... @amandasopkin
35. WW2: Engima Machine @amandasopkin
36. WW2: Engima Machine The rotors rotate at different rates as you type on the keyboard and output appropriate letters of cipher text. In this case the key was the initial setting of the rotors.
38. 1973: IBM’s Luther Algorithm (DES) @amandasopkin
39. Note on block cipher encryption A block cipher is an encryption method that applies a deterministic algorithm along with a symmetric key to encrypt a block of text, rather than encrypting one bit at a time as in stream ciphers. For example, a common block cipher, AES, encrypts 128 bit blocks with a key of predetermined length: 128, 192, or 256 bits.
41. 2000: Advanced Encryption Standard (Rijndael) @amandasopkin
43. 1970s-1990s: Crypto Wars @amandasopkin
45. Key size restrictions @amandasopkin
46. 1996: The end of crypto wars @amandasopkin
47. 1991: PGP (Pretty Good Privacy) @amandasopkin
49. Bread and Butter of cryptography
51. Why is randomness important? @amandasopkin
52. Random private key + Public key = encrypted message @amandasopkin
53. 1. You generate a random key. 2. You use that key to encrypt your data. 3. I send you my public key. 4. You use my public key to encrypt your random key. 5. Send both the encrypted data and the encrypted random key to me. 6. I use my private key to decrypt your random key. 7. I use your random key to decrypt the data. @amandasopkin
54. The security of the private key is important @amandasopkin
55. Lets talk about randomness... @amandasopkin
56. @amandasopkin
57. Randomness Makes processes secure Mathematically/computationally, naturally, philosophically important Difficult to actually achieve @amandasopkin
58. Why do we need randomness today? @amandasopkin
59. 4oio342ip4o24p32o
60. 4fdslf95454
63. Problems with randomness The seed, or starting point The algorithm @amandasopkin
65. 1. Determined that user ids were seeded with restart time 2. Crashed the Hacker News site 3. Predicted restart time 4. Predicted assigned user ids as users logged in 5. Impersonated discovered users @amandasopkin
66. DUAL_EC_DRBG Controversy • 2004: Dual EC PRNG introduced
67. • 08/2007: Shumow and Ferguson present Dual_EC_DRBG flaw at cryptography conference DUAL_EC_DRBG Controversy
68. • 11/2007: Schneier bases article in Wired on their findings DUAL_EC_DRBG Controversy
69. “...would allow NSA to determine the state of the random number generator, and thereby eventually be able to read all data sent over the SSL connection.” DUAL_EC_DRBG Controversy
71. • 09/2013: One of the purposes of Bullrun is described as being "to covertly introduce weaknesses into the encryption standards followed by hardware and software developers around the world." DUAL_EC_DRBG Controversy
72. • NIST recommends removal of the algorithm as a standard DUAL_EC_DRBG Controversy
73. • 2004: Dual EC PRNG introduced • 08/2007: Shumow and Ferguson present Dual_EC_DRBG flaw at cryptography conference • 11/2007: Schneier bases article in Wired on their findings DUAL_EC_DRBG Controversy
74. • 09/2013: One of the purposes of Bullrun is described as being "to covertly introduce weaknesses into the encryption standards followed by hardware and software developers around the world." • 12/2013: Presidential advisory examines encryption standards • 2014: Standard is removed DUAL_EC_DRBG Controversy
75. Years until standard removed... 10!
76. Who did this impact? Microsoft, Google, Apple, McAfee, Docker, IBM, Oracle, Cisco, VMWare, Juniper, HP, Red Hat, Samsung, Toshiba, DELL, Ruckus, F5 Networks, Lenovo, Nokia, the RSA BSAFE libraries for Java and C++ and more....
77. Ok, so you want to create randomness... @amandasopkin
78. An ideal pseudo random number generator should...
79. 1. Pass statistical tests of randomness An ideal pseudo random number generator should... Monobit Distance Poker or Craps Birthday
80. 1. Pass statistical tests of randomness 2. Take a long time before repeating An ideal pseudo random number generator should... Have a long “period”
81. 1. Pass statistical tests of randomness 2. Take a long time before repeating 3. Execute efficiently An ideal pseudo random number generator should... & Quick Low storage
82. 1. Pass statistical tests of randomness 2. Take a long time before repeating 3. Execute efficiently 4. Be repeatable An ideal pseudo random number generator should...
83. 1. Pass statistical tests of randomness 2. Take a long time before repeating 3. Execute efficiently 4. Be repeatable 5. Be portable An ideal pseudo random number generator should... Can be run on any machine or system
84. What are the common ways of generating “randomness”? @amandasopkin
85. Linear congruential generators Linear congruential generators take the form xk = (axk−1 + c) (mod M) where x0 is the seed, the integer M is the largest representable integer, and the period is at most M.
86. Linear congruential generators a = 3 c = 9 m = 16 x0 = 4394 def lcg(): xi = x0 for i in range(10): xi = (a*xi + c)%m print(xi)
87. Linear congruential generators Algorithm: xi = (a*xi + c)%m 7 14 3 2 15 6 11 10 7
88. Towards a better pseudorandom generator @amandasopkin
89. Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin.
90. Mid square method generally Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s Take the middle 4 digits of the result Repeat the sequence
91. Mid square method generally Start with a 4 digit seed 9834
92. Mid square method generally Start with a 4 digit seed Square this value 96707556 9834
93. Mid square method generally Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s 96707556 9834 96707556
94. Mid square method generally Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s Take the middle 4 digits of the result Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s 9834 96707556 96707556 7075
95. Mid square method generally Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s Take the middle 4 digits of the result Repeat the sequence Start with a 4 digit seed Square this value If the result has fewer than 8 digits, add leading 0s 9834 96707556 96707556 7075 50055625
96. Mid square method seed_number = int(input("Please enter a four digit number:\n[####] ")) number = seed_number already_seen = set() counter = 0 while number not in already_seen: counter += 1 already_seen.add(number) number = int(str(number * number).zfill(8)[2:6]) print(f"#{counter}: {number}") print(f"We began with the seed {seed_number}, and" f" we repeated ourselves after {counter} steps" f" with {number}.")
97. Mid square method Please enter a four digit number: [####] 5859 #1: 3278 #2: 7452 #3: 5323 #4: 3343 #5: 1756 #6: 835 #7: 6972 #8: 6087 #9: 515 #10: 2652 ....... #59: 24 #60: 5 #61: 0 #62: 0 We began with the seed 5859, and we repeated ourselves after 62 steps with 0.
98. Issues with mid square method Relatively slow Statistically unsatisfactory Sample of random numbers may be too short
99. Predicting the mid square method Advanced LCG Mid square method
100. Let’s talk cryptography @amandasopkin
101. Most used pseudo random number generator Very long period (the Mersenne prime: 219937 − 1) Not cryptographically secure The Mersenne Twister
102. Predicting the random() module from random import random import matplotlib.pyplot as plt def uni(n, m, a, c, seed): sequence = [] Xn = seed for i in range(n): Xn = ((a*Xn + c) % m) sequence.append(Xn/float(m-1)) return(sequence) x = range(1000) y_1 = uni(1000, 2**32, 11695477, 1, datetime.now().microsecond) y_2 = [random() for i in range(1000)] plt.plot(x, y_1, "o", color="blue") plt.show() plt.plot(x, y_2, "o", color="red") plt.show()
103. Predicting the random() module Advanced LCG Built in Random PRNG
104. Whats wrong with the random module? @amandasopkin
106. Problems with the random module...
107. Problems with the random module...
108. Problems with the random module... ...
109. Introducing...the secrets module! @amandasopkin
110. The Secrets module Is cryptographically secure Includes ready made “batteries” for Users that don’t want to build their own Uses 32 bytes of entropy by default
111. A note on entropy... @amandasopkin
112. @amandasopkin Natural sources of entropy
113. Source code of Secrets module from random import SystemRandom _sysrand = SystemRandom() randbits = _sysrand.getrandbits choice = _sysrand.choice def randbelow(exclusive_upper_bound): return _sysrand._randbelow(exclusive_upper_bound) DEFAULT_ENTROPY = 32 # number of bytes to return by default def token_bytes(nbytes=None): if nbytes is None: nbytes = DEFAULT_ENTROPY return os.urandom(nbytes) def token_hex(nbytes=None): return binascii.hexlify(token_bytes(nbytes)).decode('ascii') def token_urlsafe(nbytes=None): tok = token_bytes(nbytes) return base64.urlsafe_b64encode(tok).rstrip(b'=').decode('ascii')
114. SystemRandom Uses OS as a source of randomness Not available on all systems Does not rely on software states Sequences are not repeatable
115. /dev/random Will block without sufficient entropy Relies on “the kernel entropy pool” Slower than /dev/urandom
116. /dev/urandom Will not block without sufficient entropy Relies on “the kernel entropy pool” Faster than /dev/random Theoretically vulnerable to attack
117. Using the secrets module to get tokens import secrets token1 = secrets.token_hex(16) token2 = secrets.token_hex(10) print(token1) print(token2) d2bdc979d5ecec0dccf67854459c5284 584d93ac921d3c74be9c
118. Using the secrets module for password generation import secrets import string alphabet = string.ascii_letters + string.digits password = ''.join(secrets.choice(alphabet) for i in range(10)) print(password) i3OFMKPr8q
119. The secrets module: not the end all be all. @amandasopkin
120. Popular algorithms • AES • RSA • ECC
121. Popular algorithms • AES: made of three block ciphers: AES-128, AES-192 and AES-256. Each cipher encrypts and decrypts data in blocks of 128 bits using cryptographic keys of 128-, 192- and 256-bits
122. Popular algorithms: AES
123. Popular algorithms • RSA: based on the practical difficulty of the factorization of the product of two large prime numbers, the "factoring problem."
124. Popular algorithms: RSA
125. Popular algorithms • ECC: based on the algebraic structure of elliptic curves over finite fields.
126. Popular algorithms: ECC
127. Python’s “nuclear reactor” of Randomness "...folks really are better off learning to use things like cryptography.io for security sensitive software, so this change is just about harm mitigation given that it's inevitable that a non-trivial proportion of the millions of current and future Python developers won't do that."
128. Cryptography.io
129. For the paranoid...
130. How it works
131. Let’s wrap up... @amandasopkin
132. Is very important for security Difficult to truly achieve Can be simulated Randomness... @amandasopkin
133. Something to say? Amanda Sopkin @amandasopkin
134. Please submit feedback on the app!
135. Thank you! @amandasopkin
136. Sources: • Icons taken from flaticon.com • https://crypto.stackexchange.com/questions/51232/using- 32-hexadecimal-digits-vs-ascii-equivalent-16-character- password • https://dev.to/walker/pseudo-random-numbers-in-python-f rom-arithmetic-to-probability-distributions • Wired Magazine • The Washington Post • NYT • Dilbert