Float is Legacy - Speaker Deck

Float is Legacy

by Kenta Murata

Slide 1

Slide 1 text

Float is Legacy Kenta Murata RubyConf 2011 1 Monday, October 10, 11

Slide 2

Slide 2 text

http://www.flickr.com/photos/recompile_net/5951998279/ Kenta Murata CRuby committer bigdecimal maintainer OS X platform maintainer Interested in number system @mrkn 2 Monday, October 10, 11

Slide 3

Slide 3 text

https://twitter.com/#!/shyouhei/status/119802983423807488 3 Monday, October 10, 11

Slide 4

Slide 4 text

https://twitter.com/#!/shyouhei/status/119802983423807488 3 Monday, October 10, 11

Slide 5

Slide 5 text

http://www.flickr.com/photos/recompile_net/5951998279/ Kenta Murata CRuby committer bigdecimal maintainer OS X platform maintainer Interested in number system Ruby Sapporo @mrkn 4 Monday, October 10, 11

Slide 6

Slide 6 text

Sapporo, Japan http://www.flickr.com/photos/muraken/6174655831 5 Monday, October 10, 11

Slide 7

Slide 7 text

Sapporo, Japan http://www.flickr.com/photos/irasally/4708650832/ 6 Monday, October 10, 11

Slide 8

Slide 8 text

The RubyKaigi is ﬁnished. 7 Monday, October 10, 11

Slide 9

Slide 9 text

Regional RubyKaigi is continue. 8 Monday, October 10, 11

Slide 10

Slide 10 text

Sapporo RubyKaigi 04 in the next summer. 9 Monday, October 10, 11

Slide 11

Slide 11 text

Ofﬁcial information will be coming soon. 10 Monday, October 10, 11

Slide 12

Slide 12 text

Acknowledgement Tatsuhiro Ujihisa, @ujm HootSuite Media, Inc. Yoshimasa Niwa, @niw Twitter, Inc. 11 Monday, October 10, 11

Slide 13

Slide 13 text

Float is Legacy Kenta Murata RubyConf 2011 12 Monday, October 10, 11

Slide 14

Slide 14 text

Summary Float requires us the advanced knowledge Most rubyists don’t need Float Rational is enough for us Literal of decimal fraction interpreted as Rational makes us more happy 13 Monday, October 10, 11

Slide 15

Slide 15 text

Float class 14 Monday, October 10, 11

Slide 16

Slide 16 text

What is Float class? A wrapper for C double. Boxing a value of double. Need to allocate an object to generate a new Float. 15 Monday, October 10, 11

Slide 17

Slide 17 text

Do you know C double? Floating point number with double precision. No concrete representation is speciﬁed. Most current platforms employ IEEE754. It is IEEE754 binary64 on these platforms. There are platforms employing other spec. 16 Monday, October 10, 11

Slide 18

Slide 18 text

CRuby and JIS Ruby Not requiring IEEE754. 17 Monday, October 10, 11

Slide 19

Slide 19 text

Floating point numbers 18 Monday, October 10, 11

Slide 20

Slide 20 text

The origin NA = +6 . 022 141 79 ⇥ 10 23 ( ± 0 . 000 000 0030 ⇥ 10 23 ) [1 / mol] h = +6 . 626 069 57 ⇥ 10 34 ( ± 0 . 000 000 0029 ⇥ 10 34 ) [J s] 19 Monday, October 10, 11

Slide 21

Slide 21 text

The origin NA = +6 . 022 141 79 ⇥ 10 23 ( ± 0 . 000 000 0030 ⇥ 10 23 ) [1 / mol] h = +6 . 626 069 57 ⇥ 10 34 ( ± 0 . 000 000 0029 ⇥ 10 34 ) [J s] sign 19 Monday, October 10, 11

Slide 22

Slide 22 text

The origin NA = +6 . 022 141 79 ⇥ 10 23 ( ± 0 . 000 000 0030 ⇥ 10 23 ) [1 / mol] h = +6 . 626 069 57 ⇥ 10 34 ( ± 0 . 000 000 0029 ⇥ 10 34 ) [J s] fraction part sign 19 Monday, October 10, 11

Slide 23

Slide 23 text

The origin NA = +6 . 022 141 79 ⇥ 10 23 ( ± 0 . 000 000 0030 ⇥ 10 23 ) [1 / mol] h = +6 . 626 069 57 ⇥ 10 34 ( ± 0 . 000 000 0029 ⇥ 10 34 ) [J s] exponent part fraction part sign 19 Monday, October 10, 11

Slide 24

Slide 24 text

The origin NA = +6 . 022 141 79 ⇥ 10 23 ( ± 0 . 000 000 0030 ⇥ 10 23 ) [1 / mol] h = +6 . 626 069 57 ⇥ 10 34 ( ± 0 . 000 000 0029 ⇥ 10 34 ) [J s] exponent part: fraction part: sign: s 2 {0, 1} 0  f  Bn 1 e min  e q  e max 20 Monday, October 10, 11

Slide 25

Slide 25 text

Floating point numbers Numbers can be identiﬁed by (s, e, f ). Represent approximation of real numbers. Float types can be described by B, N, q, emin , and emax. B is the base number of the exponent part. N is the number of digits in the fraction part. q is the bias for the exponent part. emax and emin specify the limit of the exponent part. 21 Monday, October 10, 11

Slide 26

Slide 26 text

(s, e, f) = ( 1)s ⇥ f BN ⇥ Be q 22 Monday, October 10, 11

Slide 27

Slide 27 text

e.g. IEEE754 binary64 B = 2 N = 53 q = 1,023 emin = –1,022 emax = +1,023 The maximum positive: 1.797 693 134 862 315 7 ×10+308 The minimum nonzero positive: 2.225 073 858 507 201 4 ×10–308 23 Monday, October 10, 11

Slide 28

Slide 28 text

(s, e, f) = ( 1)s ⇥ f BN ⇥ Be q 24 Monday, October 10, 11

Slide 29

Slide 29 text

e.g. IEEE754 decimal64 B = 10 N = 16 q = 398 emin = –383 emax = +384 The maximum positive: 9.999 999 999 999 999 ×10+384 The minimum nonzero positive: 0.000 000 000 000 001 ×10–383 25 Monday, October 10, 11

Slide 30

Slide 30 text

e.g. IBM’s double precision B = 16 N = 56 q = 64 emin = –64 emax = +63 The maximum positive: 7.237 005 577 332 262 11 ×10+75 The minimum nonzero positive: 5.397 605 346 934 027 89 ×10–79 26 Monday, October 10, 11

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Every ﬂoat is approximation 28 Monday, October 10, 11

Slide 33

Slide 33 text

Every ﬂoat is approximation 0 3/2 –1 28 Monday, October 10, 11

Slide 34

Slide 34 text

Every ﬂoat is approximation 0 3/2 –1 0.0 1.5 –1.0 { { { 28 Monday, October 10, 11

Slide 35

Slide 35 text

Every ﬂoat is approximation 0 3/2 –1 0.0 1.5 –1.0 { { { 28 Monday, October 10, 11

Slide 36

Slide 36 text

Every ﬂoat is approximation 0 3/2 –1 0.0 1.5 –1.0 { { { 28 Monday, October 10, 11

Slide 37

Slide 37 text

Every ﬂoat is approximation We should think: There are no numbers represented exactly. Floating point numbers always include errors. Magnitude of errors depend on B, N, and e. 29 Monday, October 10, 11

Slide 38

Slide 38 text

Why including errors? Unavoidable issue from place-value notation with ﬁnite digits rounding. Very few values can be speciﬁed exactly. We shouldn’t expect that a given value is exact. 30 Monday, October 10, 11

Slide 39

Slide 39 text

How many decimal fractions can be exactly represented in the form of binary fraction? 31 Monday, October 10, 11

Slide 40

Slide 40 text

32 Monday, October 10, 11

Slide 41

Slide 41 text

Decimal form: (0.1234)10 = (1234)10 104 32 Monday, October 10, 11

Slide 42

Slide 42 text

Decimal form: Binary form: (0.10111)2 = (10111)2 25 (0.1234)10 = (1234)10 104 32 Monday, October 10, 11

Slide 43

Slide 43 text

Decimal form: Binary form: (0.10111)2 = (10111)2 25 (0.1234)10 = (1234)10 104 0.b1b2 · · · bn = (b1b2 · · · bn)2 2n 0.d1d2 · · · dm = (d1d2 · · · dm)10 10m 32 Monday, October 10, 11

Slide 44

Slide 44 text

Decimal form: Binary form: (0.10111)2 = (10111)2 25 (0.1234)10 = (1234)10 104 0.b1b2 · · · bn = (b1b2 · · · bn)2 2n 0.d1d2 · · · dm = (d1d2 · · · dm)10 10m 32 Monday, October 10, 11

Slide 45

Slide 45 text

(d1d2 · · · dm)10 10m = (d1d2 · · · dm)10 2m 5m = C 5m 2m 5m = C 2m 33 Monday, October 10, 11

Slide 46

Slide 46 text

1.0 0.5 0.0 5 10 15 20 25 30 0 The ratio of inexact numbers The ratio of exact numbers The number of decimal digits 34 Monday, October 10, 11

Slide 47

Slide 47 text

1.0 0.5 0.0 5 10 15 20 25 30 0 The ratio of inexact numbers The ratio of exact numbers 17 The number of decimal digits 34 Monday, October 10, 11

Slide 48

Slide 48 text

1.0 0.5 0.0 5 10 15 20 25 30 0 The ratio of inexact numbers The ratio of exact numbers IEEE754 binary64 17 The number of decimal digits 34 Monday, October 10, 11

Slide 49

Slide 49 text

Decimal in Binary A N-digit decimal notation is exactly represented in binary notation only if its numerator divisible by 5N. The ratio of N-digit decimal fractions exactly represented as binary fraction is 1 / 5N. In IEEE754 binary64, almost all numbers are inexact. 35 Monday, October 10, 11

Slide 50

Slide 50 text

Floating-point arithmetics add, sub, mul, div, sqrt, ... These operations work with errors. Please read detail description: “What Every Computer Scientist Should Know About Floating-Point Arithmetic” 36 Monday, October 10, 11

Slide 51

Slide 51 text

Decimal fraction of Ruby 37 Monday, October 10, 11

Slide 52

Slide 52 text

What’s the problem? Ruby interprets literals of decimal fraction as Float The following three numbers are Float, so they have errors. 1.0 1.2 0.42e+12 38 Monday, October 10, 11

Slide 53

Slide 53 text

The issues from Float There are many issues about Float reported to redmine.ruby-lang.org They are caused by that Ruby interpretes the literals of decimal fraction as Float, I think. Do you know these issues? 39 Monday, October 10, 11

Slide 54

Slide 54 text

http://redmine.ruby-lang.org/issues/4576 40 Monday, October 10, 11

Slide 55

Slide 55 text

Demonstration 41 Monday, October 10, 11

Slide 56

Slide 56 text

$ ruby -v ruby 1.9.4dev (2011-09-28 trunk 33354) [x86_64-darwin10.8.0] $ irb --simple-prompt >> (1.0 .. 12.7).step(1.3).to_a => [1.0, 2.3, 3.6, 4.9, 6.2, 7.5, 8.8, 10.1, 11.4, 12.700000000000001] >> (1.0 ... 128.4).step(18.2).to_a => [1.0, 19.2, 37.4, 55.599999999999994, 73.8, 92.0, 110.19999999999999, 128.39999999999998] >> (1.0 ... 128.4).step(18.2).to_a.size => 8 >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a => [1, (96/5), (187/5), (278/5), (369/5), (92/1), (551/5)] >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a.size => 7 42 Monday, October 10, 11

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Range#step with Float The ﬁrst case The last value of the array is not equal to the end of the range. The second case Some elements include errors. The array size is one larger than the right size. 46 Monday, October 10, 11

Slide 61

Slide 61 text

Rational with decimal notation Introducing one flag into a Rational object. The flag represents a Rational seems which fraction or decimal. If the flag is true, a Rational is converted decimal string by to_s. 47 Monday, October 10, 11

Slide 62

Slide 62 text

Literal for Rational with decimal notation Simple change for parser. Interpreting literal of decimal fraction without exponent as Rational with decimal notation. Literal of decimal fraction with exponent stays on Float. 48 Monday, October 10, 11

Slide 63

Slide 63 text

Demonstration using the patched Ruby https://github.com/mrkn/ruby/tree/decimal_rational_implementation 49 Monday, October 10, 11

Slide 64

Slide 64 text

$ ruby -v ruby 1.9.4dev (2011-09-28 trunk 33354) [x86_64-darwin10.8.0] $ irb --simple-prompt >> (1.0 .. 12.7).step(1.3).to_a => [1.0, 2.3, 3.6, 4.9, 6.2, 7.5, 8.8, 10.1, 11.4, 12.7] >> (1.0 .. 12.7).step(1.3).map(&:class) => [Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational] >> (1.0 ... 128.4).step(18.2).to_a => [1.0, 19.2, 37.4, 55.6, 73.8, 92.0, 110.2] >> (1.0 ... 128.4).step(18.2).to_a.size => 7 >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a => [1, (96/5), (187/5), (278/5), (369/5), (92/1), (551/5)] >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a.size => 7 50 Monday, October 10, 11

Slide 65

Slide 65 text

$ ruby -v ruby 1.9.4dev (2011-09-28 trunk 33354) [x86_64-darwin10.8.0] $ irb --simple-prompt >> (1.0 .. 12.7).step(1.3).to_a => [1.0, 2.3, 3.6, 4.9, 6.2, 7.5, 8.8, 10.1, 11.4, 12.7] >> (1.0 .. 12.7).step(1.3).map(&:class) => [Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational] >> (1.0 ... 128.4).step(18.2).to_a => [1.0, 19.2, 37.4, 55.6, 73.8, 92.0, 110.2] >> (1.0 ... 128.4).step(18.2).to_a.size => 7 >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a => [1, (96/5), (187/5), (278/5), (369/5), (92/1), (551/5)] >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a.size => 7 The last value of the array is equal to the end of the range. 51 Monday, October 10, 11

Slide 66

Slide 66 text

$ ruby -v ruby 1.9.4dev (2011-09-28 trunk 33354) [x86_64-darwin10.8.0] $ irb --simple-prompt >> (1.0 .. 12.7).step(1.3).to_a => [1.0, 2.3, 3.6, 4.9, 6.2, 7.5, 8.8, 10.1, 11.4, 12.7] >> (1.0 .. 12.7).step(1.3).map(&:class) => [Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational] >> (1.0 ... 128.4).step(18.2).to_a => [1.0, 19.2, 37.4, 55.6, 73.8, 92.0, 110.2] >> (1.0 ... 128.4).step(18.2).to_a.size => 7 >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a => [1, (96/5), (187/5), (278/5), (369/5), (92/1), (551/5)] >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a.size => 7 All elements in the array is Rational rather than Float. 52 Monday, October 10, 11

Slide 67

Slide 67 text

$ ruby -v ruby 1.9.4dev (2011-09-28 trunk 33354) [x86_64-darwin10.8.0] $ irb --simple-prompt >> (1.0 .. 12.7).step(1.3).to_a => [1.0, 2.3, 3.6, 4.9, 6.2, 7.5, 8.8, 10.1, 11.4, 12.7] >> (1.0 .. 12.7).step(1.3).map(&:class) => [Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational, Rational] >> (1.0 ... 128.4).step(18.2).to_a => [1.0, 19.2, 37.4, 55.6, 73.8, 92.0, 110.2] >> (1.0 ... 128.4).step(18.2).to_a.size => 7 >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a => [1, (96/5), (187/5), (278/5), (369/5), (92/1), (551/5)] >> (1 ... 1284.quo(10)).step(182.quo(10)).to_a.size => 7 The result array size is correct. 53 Monday, October 10, 11

Slide 68

Slide 68 text

Benchmarking Comparing Float, Rational, and C double. Experimental environment: MacBook Pro 15in (Mid 2010) Core i7 2.66 GHz Ruby 1.9.4dev (r33300) with gcc-4.2 -O3 C with llvm-gcc -O0 54 Monday, October 10, 11

Slide 69

Slide 69 text

Benchmarking codes Ruby code https://gist.github.com/1253088 C code https://gist.github.com/1253090 55 Monday, October 10, 11

Slide 70

Slide 70 text

0 [s] 0.75 [s] 1.5 [s] 2.25 [s] 3 [s] 1M additions 1M subtractions 1M multiplications Based on ruby-1.9.4dev (r33300) Float Rational C double 0.37 2.16 0.73 2.17 0.70 1.78 0.00777 0.00670 0.00770 56 Monday, October 10, 11

Slide 71

Slide 71 text

0 [s] 0.003 [s] 0.005 [s] 0.008 [s] 0.01 [s] 1M additions 1M subtractions 1M multiplications Based on ruby-1.9.4dev (r33300) Float Rational C double 0.37 2.16 0.73 2.17 0.70 1.78 0.00777 0.00670 0.00770 57 Monday, October 10, 11

Slide 72

Slide 72 text

Benchmarking summary Rational is 2-5 times slower than Float. Float is 2-digit order slower than C double. C is amazingly fast. 58 Monday, October 10, 11

Slide 73

Slide 73 text

If you said Rational is slow, Float isn’t as fast as your expect. 59 Monday, October 10, 11

Slide 74

Slide 74 text

Rational vs Float 60 Monday, October 10, 11

Slide 75

Slide 75 text

Rational vs Float 61 Monday, October 10, 11

Slide 76

Slide 76 text

Rational vs Float Exact computation is required by domains such as ﬁnance. Float is required by scientiﬁc computation. 61 Monday, October 10, 11

Slide 77

Slide 77 text

Rational vs Float Exact computation is required by domains such as ﬁnance. Float is required by scientiﬁc computation. Other aspects indepenend of whether Rational or Float. 61 Monday, October 10, 11

Slide 78

Slide 78 text

Conclusion Float is difﬁcult, troublesome, and not human oriented. Rational is easy to understand, and human oriented. It makes us more happy that Ruby interprets literal of decimal fraction as Rational. 62 Monday, October 10, 11

Slide 79

Slide 79 text

Float is Legacy 63 Monday, October 10, 11