Share
Embed
Float
下記で取り扱った浮動小数点(Float型)についての記述を公開します。
・DeepLearningと量子化(Quantization)
https://lib-arts.booth.pm/items/6074054
IEEE 754に基づく小数の表現については大まかに抑えておくと良いのではないかと思います。
LiberalArts
September 13, 2024
Transcript
5 slides
-
A.2 BNN(1-bit ͰͷྔࢠԽ)* Deep Neural Networks) Ͱਤ A.2 ͷΑ͏ͳΞϧΰϦζϜʹجֶ͍ͮͯश ਪ͕ߦΘΕ·͢ɻ
A.2 BNN(1-bit ͰͷྔࢠԽ)* BNN(Binarized Neural Networks) ֶशύϥϝʔλ (weights) Λ +1 ͱ −1 ͷ 1-bit Ͱදݱͨ͠ BinaryConnect Λ֦ுͨ͠ݚڀͰ͢ɻBNN Ͱύ ϥϝʔλͱಉ༷ʹ׆ੑԽؔͷྔࢠԽߦ͍·͢ɻ ˛ਤ A.3 BNN(Binarized Neural Networks) จ https://arxiv.org/abs/1602.02830v3 A.3 ුಈখɾFloat ܕ અ (A.3 અ) ͰҎԼɺԼهͷ Wikipedia Λࢀߟʹුಈখʹ͍ͭͯ ·ͱΊ·͢ɻ 41 -
A Appendix https://ja.wikipedia.org/wiki/ුಈখ A.3.1 ුಈখͷΈ ුಈখ (floating-point) ࣮ΛίϯϐϡʔλͰॲཧ͢Δʹ͋ͨͬͯ ༗ݶݸͷখͰऔΓѻ͏ํࣜͰ͢ɻFloat
ܕ͜ͷΑ͏ͳํࣜʹج͍ͮͯ ͕ද͞Ε·͢ɻҎԼɺුಈখʹ͓͚ΔͷදݱͷΈʹ͍ͭͯ֬ೝ ͠·͢ɻ ·ͣɺ10 ਐͰ − 1 3 Λදݱ͢Δ߹ΛԾఆ͠·͢ɻ− 1 3 ࣜ A.1 ͷΑ͏ ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.1: − 1 3 ͷۙࣅ (10 ਐ) − 1 3 = −1 × 0.33333 · · · ≃ −1 × 0.3333 × 100 = −1 × 3.333 × 10−1 ࣜ A.1 Ͱ 0.33333 · · · Λ 4 ܻͷখͰۙࣅ͠·ͨ͠ɻԼͷ 2 ͭͷࣜ ͷදهΛՊֶه๏ (scientific notation) ͱ͍͍·͢ɻࣜ A.1 ͷҰ൪Լͷ −1 × 3.333 × 10−1 ͷΑ͏ͳදهΛਖ਼نԽ (normalized number) ͱݺͼ ·͢ɻ ਖ਼نԽ (normalized number) ɺ్தʹग़ͯ͘Δখͷ࠷্Ґͷܻ͕ 0 Ҏ֎ͷࣈΛऔΔࣈΛݴ͍·͢ɻ·ͨɺࣜ A.1 ͷ 10 ਐͷ߹ͱಉ༷ ʹ − 1 3 ࣜ A.2 ͷΑ͏ʹ 2 ਐͰۙࣅ͢Δ͜ͱͰ͖·͢ɻ 42 -
A.3 ුಈখɾFloat ܕ ࣜ A.2: − 1 3 ͷۙࣅ (2
ਐ) ᶃ ɹ − 1 3 = −1 × 0.01010101010101010101010101010101 · · · ≃ −1 × 0.0101010101010101 = −1 × 2−2 + 2−4 + 2−6 + 2−8 + 2−10 + 2−12 + 2−14 + 2−16 = −1 × 1 4 + 1 16 + 1 64 + 1 256 + 1 1024 + 1 4096 + 1 16834 + 1 65536 = −0.333328247 (ۙࣅͷ 10 ਐදه) ࣜ A.2 ͷΑ͏ͳ 2 ਐͷදهࣜ A.3 ͷΑ͏ʹՊֶه๏ (scientific notation) Ͱද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.3: − 1 3 ͷۙࣅ (2 ਐ) ᶄ − 1 3 ≃ −1 × 0.0101010101010101 × 20 = −1 × 0.101010101010101 × 2−1 = −1 × 1.01010101010101 × 2−2 ࣜ A.3 ͷҰ൪Լͷ͕ࣜਖ਼نԽʹରԠ͠·͢ɻਖ਼نԽ (normalized number) ͷࣜූ߸ (sign) ͷ SɺԾ (significand) ͷ Fɺࢦ (exponent) ͷ E Λݩʹࣜ A.4 ͷΑ͏ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.4: ਖ਼نԽ (normalized number) ͷࣜ (−1)S × F × 2E ࣜ A.4 ͷΑ͏ͳਖ਼نԽͷࣜΛݩʹ IEEE ํࣜ (IEEE 754 ํࣜ) IBM ํ͕ࣜఆٛ͞Ε·͢ɻҎԼɺ࠷͘࠾༻͞Ε͍ͯΔඪ४ن֨Ͱ͋Δ IEEE 754 ํࣜʹ͍ͭͯ֬ೝ͠·͢ɻIEEE 754 Ͱද A.1 ͷΑ͏ʹʮਫ਼ʯ ɺ ʮ୯ਫ਼ʯ ɺ ʮഒਫ਼ʯ͕ఆٛ͞Ε·͢ɻ 43 -
A Appendix ˝ද A.1 ຊॻͰऔΓѻͬͨΈࠐΈؔͷҰཡ ਫ਼ ූ߸෦ ࢦ෦ Ծ෦
ਫ਼ (16 Ϗοτ) 1 Ϗοτ 5 Ϗοτ 10 Ϗοτ ୯ਫ਼ (32 Ϗοτ) 1 Ϗοτ 8 Ϗοτ 23 Ϗοτ ഒਫ਼ (64 Ϗοτ) 1 Ϗοτ 11 Ϗοτ 52 Ϗοτ ූ߸෦ਖ਼ෛͷූ߸ɺࢦ෦ 2 ͷྦྷɺԾ෦খ෦ʹͦΕͧΕର Ԡ͠·͢ɻͨͱ͑ਫ਼ͷ߹ɺූ߸෦Λ sɺԾ෦ (෦͕ 1 Ͱ͋ Δ 2 ਐখͷখ෦) Λ fɺࢦ෦Λ e ͱ͓͘ͱࣜ A.5 ͷΑ͏ʹখΛද ݱ͠·͢ɻ ࣜ A.5: ਫ਼ (IEEE 754) ͷখͷදݱ (−1)s × 2e−15 × (1 + f) ࣜ A.5 Ͱ 0 ≤ e ≤ 25 − 1 ΑΓɺ−15 ≤ e − 15 ≤ 16 ཱ͕͠·͢ɻ· ͨɺ210 = 1024 ΑΓɺf 1 1024 ࠁΈͰදݱ͞Ε·͢ɻ ಉ༷ʹ ʮ୯ਫ਼ʯ 0 ≤ e ≤ 28 −1 ΑΓࣜ A.6ɺ ʮഒਫ਼ʯ 0 ≤ e ≤ 211 −1 ΑΓࣜ A.7 ͷΑ͏ͳࣜͰͦΕͧΕදݱ͢Δ͜ͱ͕Ͱ͖·͢ɻ ࣜ A.6: ୯ਫ਼ (IEEE 754) ͷখͷදݱ (−1)s × 2e−127 × (1 + f) ࣜ A.7: ഒਫ਼ (IEEE 754) ͷখͷදݱ (−1)s × 2e−1023 × (1 + f) A.3.2 IEEE 754 ͰුಈখΛදݱ͢Δ·ͰͷྲྀΕ ᶃ ࢦͷجΛ 2 ͱ͢Δ ᶄ ԾΛਖ਼نԽ͢Δ 44 -
A.3 ුಈখɾFloat ܕ ᶅ ූ߸෦ɺࢦ෦ɺԾ෦Λ 2 ਐͷϏοτͰදݱ͠ɺ࠷্ҐϏοτ ͔Βූ߸෦ɺࢦ෦ɺԾ෦ͷॱʹهࡌ جຊతʹ্هͷྲྀΕʹجͮ͘͜ͱͰ 10
ਐͷখΛුಈখͷදه Ͱද͢͜ͱ͕Ͱ͖·͢ɻҎԼɺ2.5 ͷ୯ਫ਼ͰͷුಈখԽΛྫʹྲྀΕͷ ֬ೝΛߦ͍·͢ɻ ·ͣɺᶃͰ 2.5 Λ 2.5 = (−1)0 × 2.5 × 20 ͷΑ͏ʹද͠·͢ɻ࣍ʹᶄͰ (−1)0 × 2.5 × 20 Λ (−1)0 × 2.5 × 20 = (−1)0 × (1 + 0.25) × 2128−127 ͷΑ͏ʹม͠·͢ɻ ଓ͘ᶅͰූ߸෦ͷ 0 Λ 0ɺԾ෦ͷ 0.25 Λ 010 0000 0000 0000 000 0 0000ɺࢦ෦ͷ 128 Λ 1000 0000 Ͱද͠·͢ɻ͜ΕΒʹج͍ͮͯɺ010 0 0000 0010 0000 0000 0000 0000 0000 ͕ಘΒΕ·͢ɻ 45