Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why 0.1 + 0.2 != 0.3, or the mysterious world of floating-point numbers.

Why 0.1 + 0.2 != 0.3, or the mysterious world of floating-point numbers.

Yes, running the above expression in PHP, and in a lot of other languages, will confirm that 0.1 + 0.2 does not equal to 0.3.

But this weird behavior is explainable and even justified.

Floating point numbers can open you up a completely new universe.
Provided of course that they are properly used.

This talk will go through a bit of theory, and a lot of concrete examples.

From the Gulf War to the Quake graphics engine, via the Ariane 5 space rocket, we will see the benefits and the potential dangers of using floating-point numbers.

From these examples, we will extract some lessons we can translate into our day to day development practices.

And finally, the talk will cover when to use floating-point numbers, and when to avoid them, like when managing prices or billing, and what are the alternatives in those cases.

Benoit Jacquemont

October 23, 2020
Tweet

More Decks by Benoit Jacquemont

Other Decks in Programming

Transcript

  1. Why
    Why
    0.1 + 0.2 != 0.3
    0.1 + 0.2 != 0.3
    The Mysterious World of Floating Point Numbers
    The Mysterious World of Floating Point Numbers
    Benoit Jacquemont
    Benoit Jacquemont
    @bjacquemont
    @bjacquemont

    View Slide

  2. if (0.1 + 0.2 != 0.3) {
    echo "0.1 + 0.2 is NOT EQUAL to 0.3\n";
    }
    0.1 + 0.2 is NOT EQUAL to 0.3

    View Slide

  3. $sum = 0.1 + 0.2;
    echo "Sum is ".$sum."\n";
    if ($sum != 0.3) {
    echo "$sum is NOT EQUAL to 0.3\n";
    }
    Sum is 0.3
    0.3 is NOT EQUAL to 0.3
    GO HOME
    GO HOME PHP
    PHP,
    ,

    View Slide

  4. Let's try the same in
    JavaScript
    if (0.1 + 0.2 != 0.3) {
    console.log('0.1 + 0.2 is NOT EQUAL to 0.3');
    }
    0.1 + 0.2 is NOT EQUAL to 0.3

    View Slide

  5. Let's continue with
    JavaScript...
    let sum = 0.1 + 0.2;
    console.log("Sum is " + sum);
    if (sum != 0.3) {
    console.log(sum + ' is NOT EQUAL to 0.3');
    }
    Sum is 0.30000000000000004
    0.30000000000000004 is NOT EQUAL to 0.3

    View Slide

  6. Why in PHP
    ?
    $sum = 0.1 + 0.2;
    echo "Sum is ".$sum."\n";
    if ($sum != 0.3) {
    echo "$sum is NOT EQUAL to 0.3\n";
    }
    Sum is 0.3
    0.3 is NOT EQUAL to 0.3

    View Slide

  7. The precision con g option
    No impact on the precision of the computation!
    Round oating point numbers for display
    php.net/manual/en/ini.core.php#ini.precision

    View Slide

  8. Increasing how many digits are
    displayed
    ini_set('precision', 17); // default 14
    $sum = 0.1 + 0.2;
    echo "Sum is ".$sum."\n";
    if ($sum != 0.3) {
    echo "$sum is NOT EQUAL to 0.3\n";
    }
    Sum is 0.30000000000000004
    0.30000000000000004 is NOT EQUAL to 0.3

    View Slide

  9. 0.1 + 0.2
    ==
    0.30000000000000004
    What is going on?

    View Slide

  10. Our computers are
    Our computers are
    binary
    binary
    0
    0 or
    or 1
    1
    How to represent something in
    How to represent something in
    between?
    between?

    View Slide

  11. Representing Real Numbers in
    Scienti c Notation
    0.006458
    6.458 x 10-3
    6 . 458 x 10 -3
    6: most signi cant digit, always non zero
    458: other signi cant digits
    -3: exponent
    3 parts, each representable by integers

    View Slide

  12. Binary Implementation
    0.0110111
    1 . 10111 x 2 -10
    Let's play with the exponent!
    Look ⇑, a oating point!

    View Slide

  13. 64 bits Floating Point Encoding
    sign
    1
    bit
    exponent
    11 bits
    significand
    52 bits

    View Slide

  14. Binary Floating Point to Decimal
    Representation
    000000000110000000000000000000000000000000000000000000000010101
    1.10101 x 23
    110.101
    1 1 0 . 1 0 1
    22= 4 21= 2 20= 1 2-1= ½ 2-2= ¼ 2-3= ⅛
    1.10101 x 22 = 4 + 2 + ½ + ⅛ = 6.625

    View Slide

  15. Floating Point Numbers
    Solve Another Integers
    Limitation:
    Size

    View Slide

  16. Largest Integer
    echo PHP_INT_MAX."\n";
    9223372036854775807
    9.22 x 1018

    View Slide

  17. Largest integer: 9.22 x 10
    Largest integer: 9.22 x 1018
    18
    Earthworms mass: 7.6 x 10
    Earthworms mass: 7.6 x 1012
    12 kg
    kg

    View Slide

  18. Largest integer: 9.22 x 10
    Largest integer: 9.22 x 1018
    18
    Oceans mass: 1.42 x 10
    Oceans mass: 1.42 x 1021
    21 kg
    kg

    View Slide

  19. Largest integer: 9.22 x 10
    Largest integer: 9.22 x 1018
    18
    Earth mass: 5.42 x 10
    Earth mass: 5.42 x 1024
    24 kg
    kg

    View Slide

  20. Largest integer: 9.22 x 10
    Largest integer: 9.22 x 1018
    18
    Milky way mass: 2.98 x 10
    Milky way mass: 2.98 x 1042
    42 kg
    kg

    View Slide

  21. Largest integer: 9.22 x 10
    Largest integer: 9.22 x 1018
    18
    Observable Universe mass:
    Observable Universe mass:
    1.5 x 10
    1.5 x 1053
    53 kg
    kg

    View Slide

  22. Integers are very limited
    in size

    View Slide

  23. Largest oating point
    number
    echo number_format(PHP_FLOAT_MAX, 0, "", "")."\n";
    1797693134862315708145274237317043567980705675258449965989174768031
    5726078002853876058955863276687817154045895351438246423432132688946
    4182768467546703537516986049910576551282076245490090389328944075868
    5084551339423045832369032229481658085593321233482747978262041447231
    68738177180919299881250404026184124858368
    1.80 x 10308
    290 orders of magnitude bigger than max integer

    View Slide

  24. Floating point numbers
    are better than integers at
    representing
    very small and very large
    numbers

    View Slide

  25. Float range is immensely
    wider than integers
    Integer: 1 to 9.2 x 1018
    vs
    Float: 2.25 x 10-308 to 1.80 x 10308
    But both use 64 bit storage...

    View Slide

  26. Float can only represent
    a small sample of the
    numbers inside the
    boundaries

    View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. Why
    0.1 + 0.2 ==
    0.30000000000000004
    ini_set('precision', 20);
    $op1 = 0.1;
    $op2 = 0.2;
    $expectedSum = 0.3;
    echo "Op #1: ".$op1."\n";
    echo "Op #2: ".$op2."\n";
    $actualSum = $op1 + $op2;
    echo "Actual Sum: ".$actualSum."\n";
    echo "Expected Sum: ".$expectedSum."\n";
    Op #1: 0.10000000000000000555 ← closest approximation of 0.1
    Op #2: 0.2000000000000000111 ← closest approximation of 0.2
    Actual Sum: 0.30000000000000004441 ← this is an approximation as well

    View Slide

  37. Floating point numbers rule #1:
    Most oat numbers are
    approximations

    View Slide

  38. How to have a consistent behavior
    between applications and
    hardware?
    754 Standard
    So most computers and programming languages have
    the same behavior

    View Slide

  39. Floating Point Numbers
    Dangers and Feats

    View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. Ariane 5 Maiden
    Flight
    4 June 1996
    Payload: 4 satellites of 1.2 T each.
    Total cost: $500 million

    View Slide

  45. View Slide

  46. Oops...
    Oops...

    View Slide

  47. View Slide

  48. View Slide

  49. Same code as Ariane 4, but with 4x thrust...
    //64 bits float
    float horizontalVelocity = Inertial.getHorizontalVelocity();
    // conversion to 16 bits integer (max 32767)
    int correction = computeCorrection((int) horizontalVelocity);
    // overflow!
    applyHorizontalCorrection(correction);

    View Slide

  50. Lesson learned
    Converting oat to
    integer is highly risky
    Due to the oat wide range

    View Slide

  51. Same in PHP
    $myFloat = 600000000000000000000;
    $myInt = (int) $myFloat;
    echo $myInt."\n";
    -8742554432415203328
    No error, and the result doesn't make sense...

    View Slide

  52. The Gulf War & the
    The Gulf War & the
    Patriot Missile Event
    Patriot Missile Event

    View Slide

  53. View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. View Slide

  62. View Slide

  63. View Slide

  64. View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. // return the time in seconds in a float
    $timeBefore = getElapsedTimeFromStartup();
    1
    2
    3
    sendRadarPingAndWaitForEcho();
    4
    5
    $timeAfter = getElapsedTimeFromStartup();
    6
    7
    $targetDistance = ($timeAfter - $timeBefore) * LIGHT_SPEED/2;
    8
    // return the time in seconds in a float
    $timeBefore = getElapsedTimeFromStartup();
    sendRadarPingAndWaitForEcho();
    1
    2
    3
    4
    5
    $timeAfter = getElapsedTimeFromStartup();
    6
    7
    $targetDistance = ($timeAfter - $timeBefore) * LIGHT_SPEED/2;
    8
    // return the time in seconds in a float
    $timeBefore = getElapsedTimeFromStartup();
    sendRadarPingAndWaitForEcho();
    $timeAfter = getElapsedTimeFromStartup();
    1
    2
    3
    4
    5
    6
    7
    $targetDistance = ($timeAfter - $timeBefore) * LIGHT_SPEED/2;
    8
    // return the time in seconds in a float
    $timeBefore = getElapsedTimeFromStartup();
    sendRadarPingAndWaitForEcho();
    $timeAfter = getElapsedTimeFromStartup();
    $targetDistance = ($timeAfter - $timeBefore) * LIGHT_SPEED/2;
    1
    2
    3
    4
    5
    6
    7
    8

    View Slide

  69. Float rule #2
    The bigger a oat, the less precise it
    becomes
    aka Loss of precision

    View Slide

  70. // return the time in seconds in a float
    $timeBefore = getElapsedTimeFromStartup();
    sendRadarPingAndWaitForEcho();
    $timeAfter = getElapsedTimeFromStartup();
    $targetDistance = ($timeAfter - $timeBefore) * LIGHT_SPEED/2;
    The longer the missile battery runs,
    the less precise getElapsedTimeFromStartup()
    becomes

    View Slide

  71. Missile battery run time: 100 hours
    Inaccuray due to loss of precision: 0.3433s
    Scud missile speed: Mach 5
    Patriot missile offset to its target: 589m

    View Slide

  72. Lesson learned
    A oat number cannot be
    big and precise at the
    same time.

    View Slide

  73. But Floating Point
    numbers can bring cool
    stuff

    View Slide

  74. View Slide

  75. Quake
    Quake
    Released in Feb. 1997
    Released in Feb. 1997
    Full real-time 3D
    Full real-time 3D
    rendering without 3D
    rendering without 3D
    accelerator
    accelerator
    Runs on a 60Mhz
    Runs on a 60Mhz
    Pentium
    Pentium

    View Slide

  76. Lesson learned
    Floating Point Numbers
    calculations are crazy fast

    View Slide

  77. What about
    us?

    View Slide

  78. About me
    E-COMMERCE
    MOBILE APPLICATION
    PRINT CATALOG
    POINTS OF SALE
    ERP
    MEDIA SERVER
    SUPPLIERS
    PURCHASING DPT
    MARKETING DPT
    CSV
    FTP
    XML
    XLS
    SUPPLIERS
    PORTAL
    ENRIC
    H
    TRAN
    SLATE
    CONTROL
    Single Source of Truth for Product Information

    View Slide

  79. View Slide

  80. Storing real numbers in database
    mysql> CREATE TABLE my_float(f FLOAT);
    mysql> INSERT INTO my_float(f) VALUES(0.1);
    mysql> SELECT * FROM my_float;
    +------+
    | f |
    +------+
    | 0.1 |
    +------+
    mysql> SELECT * FROM my_float WHERE f = 0.1;
    Empty set (0.00 sec)
    mysql> SELECT ROUND(f, 17) FROM my_float;
    +---------------------+
    | ROUND(f, 17) |
    +---------------------+
    | 0.10000000149011612 |
    +---------------------+
    Loss of data due to approximation

    View Slide

  81. Storing real numbers with Decimal
    mysql> CREATE TABLE my_decimal(d DECIMAL(10,5));
    mysql> INSERT INTO my_decimal(d) VALUES(0.1);
    mysql> SELECT * FROM my_decimal;
    +---------+
    | d |
    +---------+
    | 0.10000 |
    +---------+
    mysql> SELECT * FROM my_decimal WHERE d = 0.1;
    +---------+
    | d |
    +---------+
    | 0.10000 |
    +---------+
    No approximation with xed point type

    View Slide

  82. Transmitting real
    numbers
    ini_set('precision', 17);
    $myJsonFromAPI = '{"name": "foo", "price": 29.99}';
    $myObject = json_decode($myJsonFromAPI, false);
    echo "Price:".$myObject->price."\n";
    Price:29.989999999999998
    Loss of data due to approximation
    Use strings to avoid approximation

    View Slide

  83. When oat are shining
    Non-exhaustive list

    View Slide

  84. Scienti c Calculations
    Scienti c Calculations

    View Slide

  85. Sensors
    Sensors

    View Slide

  86. Statistics
    Statistics

    View Slide

  87. Machine Learning
    Machine Learning

    View Slide

  88. When avoiding oat...
    ...and how.
    Non-exhaustive list

    View Slide

  89. Money, payment & billing
    $price = 49.99;
    $coupon = 20.25;
    $card = 29.74;
    1
    2
    3
    4
    echo "Price: $price. Using coupon $coupon.\n";
    5
    $remaining = $price - $coupon;
    6
    7
    echo "$remaining still to pay. Using card $card.\n";
    8
    $remaining = $remaining - $card;
    9
    10
    if ($remaining == 0) {
    11
    echo "Thank you for your payment!\n";
    12
    } else {
    13
    echo "Paiement not finished: $remaining to pay\n";
    14
    }
    15
    echo "Price: $price. Using coupon $coupon.\n";
    $remaining = $price - $coupon;
    $price = 49.99;
    1
    $coupon = 20.25;
    2
    $card = 29.74;
    3
    4
    5
    6
    7
    echo "$remaining still to pay. Using card $card.\n";
    8
    $remaining = $remaining - $card;
    9
    10
    if ($remaining == 0) {
    11
    echo "Thank you for your payment!\n";
    12
    } else {
    13
    echo "Paiement not finished: $remaining to pay\n";
    14
    }
    15
    echo "$remaining still to pay. Using card $card.\n";
    $remaining = $remaining - $card;
    $price = 49.99;
    1
    $coupon = 20.25;
    2
    $card = 29.74;
    3
    4
    echo "Price: $price. Using coupon $coupon.\n";
    5
    $remaining = $price - $coupon;
    6
    7
    8
    9
    10
    if ($remaining == 0) {
    11
    echo "Thank you for your payment!\n";
    12
    } else {
    13
    echo "Paiement not finished: $remaining to pay\n";
    14
    }
    15
    $price = 49.99;
    $coupon = 20.25;
    $card = 29.74;
    echo "Price: $price. Using coupon $coupon.\n";
    $remaining = $price - $coupon;
    echo "$remaining still to pay. Using card $card.\n";
    $remaining = $remaining - $card;
    if ($remaining == 0) {
    echo "Thank you for your payment!\n";
    } else {
    echo "Paiement not finished: $remaining to pay\n";
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    Price: 49.99. Using coupon 20.25.
    1
    29.74 still to pay. Using card 29.74.
    2
    Paiement not finished: 3.5527136788005E-15 to pay
    3
    Price: 49.99. Using coupon 20.25.
    29.74 still to pay. Using card 29.74.
    1
    2
    Paiement not finished: 3.5527136788005E-15 to pay
    3
    Price: 49.99. Using coupon 20.25.
    29.74 still to pay. Using card 29.74.
    Paiement not finished: 3.5527136788005E-15 to pay
    1
    2
    3

    View Slide

  90. Money, payment & billing
    ini_set('precision', 17); //Only change!
    $price = 49.99;
    $coupon = 20.25;
    $card = 29.74;
    echo "Price: $price. Using coupon $coupon.\n";
    $remaining = $price - $coupon;
    echo "$remaining still to pay. Using card $card.\n";
    $remaining = $remaining - $card;
    if ($remaining == 0) {
    echo "Thank you for your payment!\n";
    } else {
    echo "Paiement not finished: $remaining to pay\n";
    }
    Price: 49.990000000000002. Using coupon 20.25.
    29.740000000000002 still to pay. Using card 29.739999999999998.
    Paiement not finished: 3.5527136788005009E-15 to pay

    View Slide

  91. Using Floating Point Numbers with
    money:
    It seems to work...
    ... until it doesn't.

    View Slide

  92. Money is a discrete entity.
    No approximation in money!

    View Slide

  93. Use cents and integers
    $price = 4999;
    $coupon = 2025;
    $card = 2974;
    function f(int $amount): string {
    return substr($amount, 0, -2).'.'.substr($amount, -2);
    }
    echo "Price:".f($price).". Using coupon ".f($coupon).".\n";
    $remaining = $price - $coupon;
    echo f($remaining).' still to pay. Using card '.f($card)."\n";
    $remaining = $remaining - $card;
    if ($remaining == 0) {
    echo "Thank you for your payment!\n";
    } else {
    echo "Payement not finished.\nRemaining amount to pay:".f($remaining)."\n";
    }
    Price:49.99. Using coupon 20.25.
    29.74 still to pay. Using card 29.74
    Thank you for your payment!

    View Slide

  94. All amount in cents

    View Slide

  95. Other alternatives to
    Floating point numbers

    View Slide

  96. Strings
    ini_set('precision', 20);
    $myRealNumber = "0.1";
    echo $myRealNumber."\n";
    0.1
    Simple
    No computation
    No comparison

    View Slide

  97. BCMath
    Arbitrary Precision Mathematics
    bcscale(2);
    $sum = bcadd("0.1", "0.2"); // strings!!
    if (bccomp($sum, "0.3")) {
    echo "$sum is NOT EQUAL to 0.3\n";
    } else {
    echo "$sum is EQUAL to 0.3\n";
    }
    0.30 IS EQUAL to 0.3
    Computation
    Comparison
    Needs an
    extension

    View Slide

  98. php decimal
    Arbitrary-Precision Decimal Arithmetic For PHP 7
    use Decimal\Decimal;
    $op1 = new Decimal("0.1");
    $op2 = new Decimal("0.2");
    $sum = $op1->add($op2);
    if (!$sum->equals("0.3")) {
    echo "$sum is NOT EQUAL to 0.3\n";
    } else {
    echo "$sum is EQUAL to 0.3\n";
    }
    0.3 is EQUAL to 0.3
    Nice object
    interface
    Not easily
    available
    php-decimal.io

    View Slide

  99. Floating Point Numbers
    Fear them
    Respect them
    Love them
    Thank you!
    joind.in/talk/1d52e

    View Slide