"Posits: A proposed new floating point number format for ML" by Kevin Colville

Slide 1

Slide 1 text

POSITS A NEW FLOATING POINT NUMBER FORMAT PROPOSED BY JOHN GUSTAFSON AND ISAAC YONEMOTO Kevin Colville NICIS Centre for High Performance Computing Presented at PyConZA 2020 9 October 2020

Slide 2

Slide 2 text

IEEE half-precision: float16 16 bits s exponent fraction 5 10 1

Slide 3

Slide 3 text

Posit16 | 16 bits | | 1..15 | s r1 r2 .. rm r* e f1 f2 .. fn regime fraction 2

Slide 4

Slide 4 text

Posit32 | 32 bits | 1..31 3 s r1 r2 .. rm r* e f1 f2 .. fn regime fraction 3

Slide 5

Slide 5 text

Trying out posits sfpy https://pypi.org/project/sfpy/ pip install sfpy 4 So ware simulation only: limited hardware exists (*)

Slide 6

Slide 6 text

Multiply by X = 1.23456789 i X float16(M) float16(X) posit16(M) posit16(X) 0: 1.23456789 1 1.234375 1 1.234619 1: 12.34567890 10 12.343750 10 12.347656 2: 123.45678900 100 123.437500 100 123.500000 3: 1234.56789000 1000 1234.000000 1000 1232.000000 4: 12345.67890000 10000 12344.000000 9984 12288.000000 5: 123456.78900000 inf inf 98304 122880.000000 6: 1234567.89000000 inf inf 983040 1310720.000000 7: 12345678.90000000 inf inf 8388608 8388608.000000 8: 123456788.99999999 inf inf 67108864 67108864.000000 9: 1234567890.00000000 inf inf 268435456 268435456.000000 10: 12345678899.99999809 inf inf 268435456 268435456.000000 5

Slide 7

Slide 7 text

6 Plays oﬀ extended range with reduced precision Increased precision for “ordinary” scale numbers Physics calculations in MKS units Saturate instead of overflow Unsigned and Numerical error analysis has to be rebuilt factors inexact Multiplicative cancellation Non-constant relative error

Slide 8

Slide 8 text

Constructing a pseudo-sigmoid by manipulating bits of posit8 Speeding up ML Figure 6. §2.2. John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic 7

Slide 9

Slide 9 text

posits? 8 Numerical concerns Analytical tools needed Potential as a storage format Specialised applications like ML

Slide 10

Slide 10 text

Further Reading John L. Gustafson, Isaac Yonemoto. Beating Floating Point at its Own Game: Posit Arithmetic http://www.johngustafson.net/pdfs/BeatingFloatingPoint.pdf Florent de Dinechin, Luc Forget, Jean-Michel Muller, Yohann Uguen. Posits: the good, the bad and the ugly. CoNGA 2019 - Conference on Next-Generation Arithmetic, Mar 2019, Singapore, Singapore. pp.1-10, 10.1145/3316279.3316285. hal-01959581v4 https://hal.inria.fr/hal-01959581v4 9