Upgrade to Pro — share decks privately, control downloads, hide ads and more …

"Posits: A proposed new floating point number format for ML" by Kevin Colville

Pycon ZA
October 09, 2020

"Posits: A proposed new floating point number format for ML" by Kevin Colville

A Lightning Talk at PyCon ZA 2020

Pycon ZA

October 09, 2020
Tweet

More Decks by Pycon ZA

Other Decks in Programming

Transcript

  1. POSITS
    A NEW FLOATING POINT NUMBER FORMAT PROPOSED BY JOHN GUSTAFSON AND ISAAC YONEMOTO
    Kevin Colville
    NICIS Centre for High Performance Computing
    Presented at PyConZA 2020
    9 October 2020

    View Slide

  2. IEEE half-precision: float16
    16 bits
    s exponent fraction
    5 10
    1

    View Slide

  3. Posit16
    | 16 bits |
    | 1..15 |
    s r1 r2 .. rm r* e f1 f2 .. fn
    regime fraction
    2

    View Slide

  4. Posit32
    | 32 bits |
    1..31 3
    s r1 r2 .. rm r* e f1 f2 .. fn
    regime fraction
    3

    View Slide

  5. Trying out posits
    sfpy
    https://pypi.org/project/sfpy/
    pip install sfpy
    4
    So ware simulation only: limited hardware exists (*)

    View Slide

  6. Multiply by
    X = 1.23456789
    i X float16(M) float16(X) posit16(M) posit16(X)
    0: 1.23456789 1 1.234375 1 1.234619
    1: 12.34567890 10 12.343750 10 12.347656
    2: 123.45678900 100 123.437500 100 123.500000
    3: 1234.56789000 1000 1234.000000 1000 1232.000000
    4: 12345.67890000 10000 12344.000000 9984 12288.000000
    5: 123456.78900000 inf inf 98304 122880.000000
    6: 1234567.89000000 inf inf 983040 1310720.000000
    7: 12345678.90000000 inf inf 8388608 8388608.000000
    8: 123456788.99999999 inf inf 67108864 67108864.000000
    9: 1234567890.00000000 inf inf 268435456 268435456.000000
    10: 12345678899.99999809 inf inf 268435456 268435456.000000
    5

    View Slide

  7. 6
    Plays off extended range with reduced precision
    Increased precision for “ordinary” scale numbers
    Physics calculations in MKS units
    Saturate instead of overflow
    Unsigned and
    Numerical error analysis has to be rebuilt
    factors inexact
    Multiplicative cancellation
    Non-constant relative error

    View Slide

  8. Constructing a pseudo-sigmoid by manipulating bits of posit8
    Speeding up ML
    Figure 6. §2.2. John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic
    7

    View Slide

  9. posits?
    8
    Numerical concerns
    Analytical tools needed
    Potential as a storage format
    Specialised applications like ML

    View Slide

  10. Further Reading
    John L. Gustafson, Isaac Yonemoto.
    Beating Floating Point at its Own Game: Posit Arithmetic
    http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf
    Florent de Dinechin, Luc Forget, Jean-Michel Muller, Yohann Uguen.
    Posits: the good, the bad and the ugly.
    CoNGA 2019 - Conference on Next-Generation Arithmetic, Mar 2019,
    Singapore, Singapore. pp.1-10, 10.1145/3316279.3316285. hal-01959581v4
    https://hal.inria.fr/hal-01959581v4
    9

    View Slide