Lies, Damned Lies, and Substrings

Generate all of the substrings of a string—a classic coding problem. But what's its time complexity? In many languages this is a pretty straightforward question, but in Ruby it turns out, it depends.

Follow me into the matrix as I explore copy-on-write optimization, how substrings are created in MRI, and eventually create a custom build of Ruby to try to speed up this classic coding problem. This talk will be a mix of computer science and a deep dive into how Ruby strings work in MRI.

Originally delivered ay RubyConf 2016.

Transcript

1. Lies, Damned Lies, and Substrings HASEEB QURESHI SOFTWARE ENGINEER @
2. Let me tell you a story about a time Ruby lied to me.
3. A coworker and I were arguing about an algorithm. Him Me
4. It started with a classic problem: How to generate all of the substrings of a string?
5. Hello H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello Hello i = 0 j = 3
6. Hello H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello Hello i = 1 j = 4 Each substring is defined by a unique start and end index.
7. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end Quadratically many pairs of indices, therefore the inner loop runs O(n2) many times.
8. Me: This algorithm is O(n2).
9. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end But what about what’s inside the loop?
10. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end How long does it actually take to build a substring?
11. (We’re going to assume fixed-width [ASCII/ UTF-32] strings for simplicity.) (Also, Ruby treats strings less than 24 characters differently, but we can ignore that for large n.)
12. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 Memory e l l 52a0 52a1 52a2 52a3 52a4 52a5 str str2 = str[1..3]
13. Obviously, copying each substring takes linear time. That is, linear in the length of the average substring.
14. O(1)? Log(n)? O(n)? H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello … Which is how long?
15. require_relative 'substrings' def average_substring_ratio(original_string_length) str = 'a' * original_string_length substring_lengths = substrings(str).map(&:length) average_substring_length = substring_lengths.reduce(:+) .fdiv(substring_lengths.count) average_substring_length / original_string_length end (1..150).step(5).each do |count| puts "#{count}: #{average_substring_ratio(count)}" end
16. 1: 1.0 6: 0.4444444444444444 11: 0.3939393939393939 16: 0.375 21: 0.3650793650793651 26: 0.358974358974359 31: 0.3548387096774194 36: 0.35185185185185186 41: 0.34959349593495936 46: 0.34782608695652173 51: 0.34640522875817 56: 0.34523809523809523 61: 0.3442622950819672 66: 0.3434343434343434 71: 0.3427230046948357 76: 0.34210526315789475 81: 0.34156378600823045 86: 0.34108527131782945 91: 0.34065934065934067 96: 0.34027777777777773 101: 0.33993399339933994 106: 0.33962264150943394 111: 0.3393393393393393 116: 0.339080459770115 121: 0.33884297520661155 126: 0.3386243386243386 131: 0.3384223918575064 136: 0.3382352941176471 141: 0.3380614657210402 146: 0.33789954337899547 (You can also prove this mathematically.) Lim n→∞= ⅓n
17. H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello So the average substring grows linearly with the original string.
18. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end Thus, this copy is O(n)
19. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end So this whole thing takes O(n3) time. Colleague:
20. Not so fast. (or slow.)
21. Enter COW (copy-on-write)
22. Copy-on-write is a kind of structural sharing.
23. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 Memory str str2 = str[1..3] str_ptr: 8fe1 length: 3
24. Here’s the proof.
25. require_relative 'display_string' # credit to Pat Shaughnessy debug = Debug.new str = ('a'..'z').to_a.join str2 = str.dup debug.display_string(str) # DEBUG: RString = 0x7f98fb05b090 # DEBUG: ptr = 0x7f98fc0aa970 -> "abcdefghijklmnopqrstuvwxyz" # DEBUG: len = 26 debug.display_string(str2) # DEBUG: RString = 0x7f98fb05afa0 # DEBUG: ptr = 0x7f98fc0aa970 -> "abcdefghijklmnopqrstuvwxyz" # DEBUG: len = 26 Pointer to same string in memory!
26. require_relative 'display_string' # credit to Pat Shaughnessy debug = Debug.new str = ('a'..'z').to_a.join str2 = str[1..-1] debug.display_string(str) # DEBUG: RString = 0x7f98fb05b090 # DEBUG: ptr = 0x7f98fc0aa970 -> "abcdefghijklmnopqrstuvwxyz" # DEBUG: len = 26 debug.display_string(str2) # DEBUG: RString = 0x7f98fb05afa0 # DEBUG: ptr = 0x7f98fc0aa971 -> "bcdefghijklmnopqrstuvwxyz" # DEBUG: len = 25 Still the same string, but now offset by 1.
27. What happens if either string gets mutated?
28. require_relative 'display_string' # credit to Pat Shaughnessy debug = Debug.new str = ('a'..'z').to_a.join str2 = str[1..-1] str[1] = '&' debug.display_string(str) # DEBUG: RString = 0x7fa2a304fbf8 # DEBUG: ptr = 0x7fa2a2f1f170 -> "a&cdefghijklmnopqrstuvwxyz" # DEBUG: len = 26 debug.display_string(str2) # DEBUG: RString = 0x7fa2a304fae0 # DEBUG: ptr = 0x7fa2a2f50b11 -> "bcdefghijklmnopqrstuvwxyz" # DEBUG: len = 25 The write forced a copy to a new string in memory.
29. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 Memory str str2 = str[1..3] str_ptr: 8fe1 length: 3 callbacks: [str2]
30. str[1] = '&'
31. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 Memory str str2 = str[1..3] callbacks: [str2] e l l 52a0 52a1 52a2 52a3 52a4 52a5
32. H & l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 Memory str str2 = str[1..3] e l l 52a0 52a1 52a2 52a3 52a4 52a5
33. So…
34. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end This is a shallow copy, which is actually O(1).
35. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end And this whole thing takes O(n2) time.
37. Case closed.
38. require_relative 'substrings' require 'benchmark' str = 'abcdefgh' * 128 str2 = str * 2 benchmarks = Benchmark.bmbm do |bm| bm.report(str.length) do substrings(str) end bm.report(str2.length) do substrings(str2) end end puts 'Growth: ' + benchmarks[1].real / benchmarks[0].real
39. Rehearsal ---------------------------------------- 1024 0.290000 0.070000 0.360000 ( 0.357953) 2048 2.360000 0.500000 2.860000 ( 2.876344) ------------------------------- total: 3.220000sec user system total real 1024 0.270000 0.070000 0.340000 ( 0.338351) 2048 2.200000 0.400000 2.600000 ( 2.601713) Growth: 7.689380300623611
41. When the input doubles, the time grows by a factor of 8. This algorithm is not quadratic. ( 0.338351) ( 2.601713)
42. wat
43. require 'benchmark' NUM_TIMES = 100_000 str = 'abcde' * 2 ** 10 str2 = str * 2 Benchmark.bmbm do |bm| bm.report(str.length) do NUM_TIMES.times { str[1..-1] } end bm.report(str2.length) do NUM_TIMES.times { str2[1..-1] } end end
44. Rehearsal ----------------------------------------- ... --------------------------------------------------- user system total real 5120 0.020000 0.000000 0.020000 ( 0.021144) 10240 0.020000 0.000000 0.020000 ( 0.020291)
45. That sure looks like copy-on-write optimization…
46. require 'benchmark' NUM_TIMES = 100_000 str = 'abcde' * 2 ** 10 str2 = str * 2 Benchmark.bmbm do |bm| bm.report(str.length) do NUM_TIMES.times { str[1..-2] } end bm.report(str2.length) do NUM_TIMES.times { str2[1..-2] } end end
47. Rehearsal ----------------------------------------- ... --------------------------------------------------- user system total real 5120 0.110000 0.060000 0.170000 ( 0.171367) 10240 0.200000 0.140000 0.340000 ( 0.347153)
49. Only substrings that include the last character are copy-on-write. So turns out:
50. the vast majority of substrings don’t include the last character. H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello And, of course,
51. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end So this on average is linear.
52. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end And this whole thing is O(n3).
53. It was all a lie.
54. WHY HAVE YOU BETRAYED ME RUBY
56. Naturally…
57. Hmm.
59. ¯\_(ϑ)_/¯
60. ᕕ( ᐛ )ᕗ Let’s… … recompile Ruby…?
61. maml004775hquresh:ruby haseeb_qureshi$ make install CC = clang LD = ld LDSHARED = clang -dynamic -bundle CFLAGS = -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno- long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality - Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Werror=implicit-int - Werror=pointer-arith -Werror=write-strings -Werror=declaration-after-statement -Werror=shorten-64- to-32 -Werror=implicit-function-declaration -Werror=division-by-zero -Werror=deprecated- declarations -Werror=extra-tokens -pipe XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden - DRUBY_EXPORT -fPIE CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -I. - I.ext/include/x86_64-darwin15 -I./include -I. -I./enc/unicode/9.0.0 DLDFLAGS = -Wl,-undefined,dynamic_lookup -Wl,-multiply_defined,suppress -fstack-protector -Wl,- u,_objc_msgSend -Wl,-pie -framework CoreFoundation SOLIBS = Apple LLVM version 7.3.0 (clang-703.0.31) Target: x86_64-apple-darwin15.4.0
62. I now have a custom version of in my usr/local/bin ml004775hquresh:bin haseeb_qureshi$ ls -l ... -rwxr-xr-x 1 haseeb_qureshi admin 3.1M Oct 23 00:37 ruby ... ml004775hquresh:bin haseeb_qureshi$ ./ruby -v ruby 2.4.0dev (2016-10-23 trunk 56478) [x86_64-darwin15]
63. require 'benchmark' NUM_TIMES = 100_000 str = 'abcde' * 2 ** 10 str2 = str * 2 Benchmark.bmbm do |bm| bm.report(str.length) do NUM_TIMES.times { str[1..-2] } end bm.report(str2.length) do NUM_TIMES.times { str2[1..-2] } end end Let’s run this benchmark again…
64. Rehearsal ----------------------------------------- ... --------------------------------------------------- user system total real 5120 0.020000 0.000000 0.020000 ( 0.020432) 10240 0.020000 0.000000 0.020000 ( 0.020300) ml004775hquresh:bin haseeb_qureshi$ ./ruby ~/Projects/substrings/benchmark3.rb Boom.
65. Ruby is now doing copy-on-write optimization on all strings!
66. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end And this bad boy, finally, takes O(n2) time.
67. (Applause break)
68. But you have to wonder… why was that the default behavior?
69. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 str \0 In C, strings should end with a null-terminator or null byte. This is how C knows it’s reached the end of a string. Null terminator
70. H e l l o 8fe0 8fe1 8fe2 8fe3 8fe4 8fe5 str \0 If you passed a substring which did not include a NUL into a library written in C, it might keep reading bytes until it found the NUL. Null terminator str2 = str[1..3]
71. Essentially, it ensures any C extensions treat all Ruby strings correctly.
72. So that’s it. We’re finally done. We have an O(n2) algorithm for substrings.
73. Except one thing…
74. Remember where we started? We need to generate all the substrings. Did we actually… generate them?
75. def substrings(str) (0...str.length).each_with_object([]) do |i, subs| (i...str.length).each do |j| subs << str[i..j] end end end puts substrings("Hello") It takes linear time to print a substring, so printing all the substrings will still take O(n3) time.
76. So in what sense is this O(n2)?
77. If you think about it, the whole idea of copy-on-write is laziness. What we’ve created are lazy strings.
78. H, e, l, l, o He, el, ll, lo Hel, ell, llo Hell, ello Hello Instead of making these: str[0..-1] We made these: str[0..3], str[1..4] str[0..2], str[1..3], str[2..4] str[0..1], str[1..2], str[2..3], str[3..4] str[0..0], str[1..1], str[2..2], str[3..3], str[4..4]
79. All we’ve really done is build each pair of indices.
80. The Ruby array that substrings(str) returns does not actually contain the substrings. It’s just a clever, lazy way to express them.
81. It’s lies all the way down.
82. Thanks for listening. You can follow me at @hosseeb Special thanks to Ned Ruggeri, David Runger, and Pat Shaughnessy. You can find the code on Github: Haseeb-Qureshi