"Posits: A proposed new floating point number format for ML" by Kevin Colville

A Lightning Talk at PyCon ZA 2020

Transcript

1. POSITS A NEW FLOATING POINT NUMBER FORMAT PROPOSED BY JOHN GUSTAFSON AND ISAAC YONEMOTO Kevin Colville NICIS Centre for High Performance Computing Presented at PyConZA 2020 9 October 2020
2. IEEE half-precision: float16 16 bits s exponent fraction 5 10 1
3. Posit16 | 16 bits | | 1..15 | s r1 r2 .. rm r* e f1 f2 .. fn regime fraction 2
4. Posit32 | 32 bits | 1..31 3 s r1 r2 .. rm r* e f1 f2 .. fn regime fraction 3
5. Trying out posits sfpy https://pypi.org/project/sfpy/ pip install sfpy 4 So ware simulation only: limited hardware exists (*)
6. Multiply by X = 1.23456789 i X float16(M) float16(X) posit16(M) posit16(X) 0: 1.23456789 1 1.234375 1 1.234619 1: 12.34567890 10 12.343750 10 12.347656 2: 123.45678900 100 123.437500 100 123.500000 3: 1234.56789000 1000 1234.000000 1000 1232.000000 4: 12345.67890000 10000 12344.000000 9984 12288.000000 5: 123456.78900000 inf inf 98304 122880.000000 6: 1234567.89000000 inf inf 983040 1310720.000000 7: 12345678.90000000 inf inf 8388608 8388608.000000 8: 123456788.99999999 inf inf 67108864 67108864.000000 9: 1234567890.00000000 inf inf 268435456 268435456.000000 10: 12345678899.99999809 inf inf 268435456 268435456.000000 5
7. 6 Plays off extended range with reduced precision Increased precision for “ordinary” scale numbers Physics calculations in MKS units Saturate instead of overflow Unsigned and Numerical error analysis has to be rebuilt factors inexact Multiplicative cancellation Non-constant relative error
8. Constructing a pseudo-sigmoid by manipulating bits of posit8 Speeding up ML Figure 6. §2.2. John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic 7
9. posits? 8 Numerical concerns Analytical tools needed Potential as a storage format Specialised applications like ML
10. Further Reading John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf Florent de Dinechin, Luc Forget, Jean-Michel Muller, Yohann Uguen. Posits: the good, the bad and the ugly. CoNGA 2019 - Conference on Next-Generation Arithmetic, Mar 2019, Singapore, Singapore. pp.1-10, 10.1145/3316279.3316285. hal-01959581v4 https://hal.inria.fr/hal-01959581v4 9