"Posits: A proposed new floating point number format for ML" by Kevin Colville

A Lightning Talk at PyCon ZA 2020

Transcript

1. POSITS
   A NEW FLOATING POINT NUMBER FORMAT PROPOSED BY JOHN GUSTAFSON AND ISAAC YONEMOTO
   Kevin Colville
   NICIS Centre for High Performance Computing
   Presented at PyConZA 2020
   9 October 2020
   

   View Slide

2. IEEE half-precision: float16
   16 bits
   s exponent fraction
   5 10
   1
   

   View Slide

3. Posit16
   | 16 bits |
   | 1..15 |
   s r1 r2 .. rm r* e f1 f2 .. fn
   regime fraction
   2
   

   View Slide

4. Posit32
   | 32 bits |
   1..31 3
   s r1 r2 .. rm r* e f1 f2 .. fn
   regime fraction
   3
   

   View Slide

5. Trying out posits
   sfpy
   https://pypi.org/project/sfpy/
   pip install sfpy
   4
   So ware simulation only: limited hardware exists (*)
   

   View Slide

6. Multiply by
   X = 1.23456789
   i X float16(M) float16(X) posit16(M) posit16(X)
   0: 1.23456789 1 1.234375 1 1.234619
   1: 12.34567890 10 12.343750 10 12.347656
   2: 123.45678900 100 123.437500 100 123.500000
   3: 1234.56789000 1000 1234.000000 1000 1232.000000
   4: 12345.67890000 10000 12344.000000 9984 12288.000000
   5: 123456.78900000 inf inf 98304 122880.000000
   6: 1234567.89000000 inf inf 983040 1310720.000000
   7: 12345678.90000000 inf inf 8388608 8388608.000000
   8: 123456788.99999999 inf inf 67108864 67108864.000000
   9: 1234567890.00000000 inf inf 268435456 268435456.000000
   10: 12345678899.99999809 inf inf 268435456 268435456.000000
   5
   

   View Slide

7. 6
   Plays off extended range with reduced precision
   Increased precision for “ordinary” scale numbers
   Physics calculations in MKS units
   Saturate instead of overflow
   Unsigned and
   Numerical error analysis has to be rebuilt
   factors inexact
   Multiplicative cancellation
   Non-constant relative error
   

   View Slide

8. Constructing a pseudo-sigmoid by manipulating bits of posit8
   Speeding up ML
   Figure 6. §2.2. John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic
   7
   

   View Slide

9. posits?
   8
   Numerical concerns
   Analytical tools needed
   Potential as a storage format
   Specialised applications like ML
   

   View Slide

10. Further Reading
    John L. Gustafson, Isaac Yonemoto.
    Beating Floating Point at its Own Game: Posit Arithmetic
    http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf
    Florent de Dinechin, Luc Forget, Jean-Michel Muller, Yohann Uguen.
    Posits: the good, the bad and the ugly.
    CoNGA 2019 - Conference on Next-Generation Arithmetic, Mar 2019,
    Singapore, Singapore. pp.1-10, 10.1145/3316279.3316285. hal-01959581v4
    https://hal.inria.fr/hal-01959581v4
    9
    

    View Slide