Tweet
Tweet

Slide 1

Slide 1 text

A.2 BNN(1-bit ͰͷྔࢠԽ)* Deep Neural Networks) Ͱ͸ਤ A.2 ͷΑ͏ͳΞϧΰϦζϜʹجֶ͍ͮͯश ΍ਪ࿦͕ߦΘΕ·͢ɻ A.2 BNN(1-bit ͰͷྔࢠԽ)* BNN(Binarized Neural Networks) ͸ֶशύϥϝʔλ (weights) Λ +1 ͱ −1 ͷ 1-bit Ͱදݱͨ͠ BinaryConnect Λ֦ுͨ͠ݚڀͰ͢ɻBNN Ͱ͸ύ ϥϝʔλͱಉ༷ʹ׆ੑԽؔ਺ͷྔࢠԽ΋ߦ͍·͢ɻ ˛ਤ A.3 BNN(Binarized Neural Networks) ࿦จ https://arxiv.org/abs/1602.02830v3 A.3 ුಈখ਺఺ɾFloat ܕ ౰અ (A.3 અ) Ͱ͸ҎԼɺԼهͷ Wikipedia Λࢀߟʹුಈখ਺఺ʹ͍ͭͯ ·ͱΊ·͢ɻ 41

Slide 2

Slide 2 text

෇࿥ A Appendix https://ja.wikipedia.org/wiki/ුಈখ਺఺਺ A.3.1 ුಈখ਺఺ͷ࢓૊Έ ුಈখ਺఺ (floating-point) ͸࣮਺ΛίϯϐϡʔλͰॲཧ͢Δʹ͋ͨͬͯ ༗ݶݸͷখ਺ͰऔΓѻ͏ํࣜͰ͢ɻFloat ܕ͸͜ͷΑ͏ͳํࣜʹج͍ͮͯ ਺͕ද͞Ε·͢ɻҎԼɺුಈখ਺఺ʹ͓͚Δ਺ͷදݱͷ࢓૊Έʹ͍ͭͯ֬ೝ ͠·͢ɻ ·ͣɺ10 ਐ਺Ͱ − 1 3 Λදݱ͢Δ৔߹ΛԾఆ͠·͢ɻ− 1 3 ͸ࣜ A.1 ͷΑ͏ ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.1: − 1 3 ͷۙࣅ (10 ਐ਺) − 1 3 = −1 × 0.33333 · · · ≃ −1 × 0.3333 × 100 = −1 × 3.333 × 10−1 ࣜ A.1 Ͱ͸ 0.33333 · · · Λ 4 ܻͷখ਺Ͱۙࣅ͠·ͨ͠ɻԼͷ 2 ͭͷࣜ ͷදهΛՊֶه਺๏ (scientific notation) ͱ͍͍·͢ɻࣜ A.1 ͷҰ൪Լͷ −1 × 3.333 × 10−1 ͷΑ͏ͳදهΛਖ਼نԽ਺ (normalized number) ͱݺͼ ·͢ɻ ਖ਼نԽ਺ (normalized number) ͸ɺ్தʹग़ͯ͘Δখ਺ͷ࠷্Ґͷܻ͕ 0 Ҏ֎ͷ਺ࣈΛऔΔ਺ࣈΛݴ͍·͢ɻ·ͨɺࣜ A.1 ͷ 10 ਐ਺ͷ৔߹ͱಉ༷ ʹ − 1 3 ͸ࣜ A.2 ͷΑ͏ʹ 2 ਐ਺Ͱۙࣅ͢Δ͜ͱ΋Ͱ͖·͢ɻ 42

Slide 3

Slide 3 text

A.3 ුಈখ਺఺ɾFloat ܕ ࣜ A.2: − 1 3 ͷۙࣅ (2 ਐ਺) ᶃ ɹ − 1 3 = −1 × 0.01010101010101010101010101010101 · · · ≃ −1 × 0.0101010101010101 = −1 × 2−2 + 2−4 + 2−6 + 2−8 + 2−10 + 2−12 + 2−14 + 2−16 = −1 × 1 4 + 1 16 + 1 64 + 1 256 + 1 1024 + 1 4096 + 1 16834 + 1 65536 = −0.333328247 (ۙࣅ஋ͷ 10 ਐ਺දه) ࣜ A.2 ͷΑ͏ͳ 2 ਐ਺ͷදه΋ࣜ A.3 ͷΑ͏ʹՊֶه਺๏ (scientific notation) Ͱද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.3: − 1 3 ͷۙࣅ (2 ਐ਺) ᶄ − 1 3 ≃ −1 × 0.0101010101010101 × 20 = −1 × 0.101010101010101 × 2−1 = −1 × 1.01010101010101 × 2−2 ࣜ A.3 ͷҰ൪Լͷ͕ࣜਖ਼نԽ਺ʹରԠ͠·͢ɻਖ਼نԽ਺ (normalized number) ͷࣜ͸ූ߸ (sign) ͷ SɺԾ਺ (significand) ͷ Fɺࢦ਺ (exponent) ͷ E Λݩʹࣜ A.4 ͷΑ͏ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.4: ਖ਼نԽ਺ (normalized number) ͷࣜ (−1)S × F × 2E ࣜ A.4 ͷΑ͏ͳਖ਼نԽ਺ͷࣜΛݩʹ IEEE ํࣜ (IEEE 754 ํࣜ) ΍ IBM ํ͕ࣜఆٛ͞Ε·͢ɻҎԼɺ࠷΋޿͘࠾༻͞Ε͍ͯΔඪ४ن֨Ͱ͋Δ IEEE 754 ํࣜʹ͍ͭͯ֬ೝ͠·͢ɻIEEE 754 Ͱ͸ද A.1 ͷΑ͏ʹʮ൒ਫ਼౓ʯ ɺ ʮ୯ਫ਼౓ʯ ɺ ʮഒਫ਼౓ʯ͕ఆٛ͞Ε·͢ɻ 43

Slide 4

Slide 4 text

෇࿥ A Appendix ˝ද A.1 ຊॻͰऔΓѻͬͨ૊ΈࠐΈؔ਺ͷҰཡ ਫ਼౓ ූ߸෦ ࢦ਺෦ Ծ਺෦ ൒ਫ਼౓ (16 Ϗοτ) 1 Ϗοτ 5 Ϗοτ 10 Ϗοτ ୯ਫ਼౓ (32 Ϗοτ) 1 Ϗοτ 8 Ϗοτ 23 Ϗοτ ഒਫ਼౓ (64 Ϗοτ) 1 Ϗοτ 11 Ϗοτ 52 Ϗοτ ූ߸෦͸ਖ਼ෛͷූ߸ɺࢦ਺෦͸ 2 ͷྦྷ৐ɺԾ਺෦͸খ਺෦෼ʹͦΕͧΕର Ԡ͠·͢ɻͨͱ͑͹൒ਫ਼౓ͷ৔߹ɺූ߸෦Λ sɺԾ਺෦ (੔਺෦෼͕ 1 Ͱ͋ Δ 2 ਐখ਺ͷখ਺෦෼) Λ fɺࢦ਺෦Λ e ͱ͓͘ͱࣜ A.5 ͷΑ͏ʹখ਺Λද ݱ͠·͢ɻ ࣜ A.5: ൒ਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−15 × (1 + f) ࣜ A.5 Ͱ͸ 0 ≤ e ≤ 25 − 1 ΑΓɺ−15 ≤ e − 15 ≤ 16 ͕੒ཱ͠·͢ɻ· ͨɺ210 = 1024 ΑΓɺf ͸ 1 1024 ࠁΈͰදݱ͞Ε·͢ɻ ಉ༷ʹ ʮ୯ਫ਼౓ʯ ͸ 0 ≤ e ≤ 28 −1 ΑΓࣜ A.6ɺ ʮഒਫ਼౓ʯ ͸ 0 ≤ e ≤ 211 −1 ΑΓࣜ A.7 ͷΑ͏ͳ਺ࣜͰͦΕͧΕදݱ͢Δ͜ͱ͕Ͱ͖·͢ɻ ࣜ A.6: ୯ਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−127 × (1 + f) ࣜ A.7: ഒਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−1023 × (1 + f) A.3.2 IEEE 754 Ͱුಈখ਺఺Λදݱ͢Δ·ͰͷྲྀΕ ᶃ ࢦ਺ͷج਺Λ 2 ͱ͢Δ ᶄ Ծ਺Λਖ਼نԽ͢Δ 44

Slide 5

Slide 5 text

A.3 ුಈখ਺఺ɾFloat ܕ ᶅ ූ߸෦ɺࢦ਺෦ɺԾ਺෦Λ 2 ਐ਺ͷϏοτͰදݱ͠ɺ࠷্ҐϏοτ ͔Βූ߸෦ɺࢦ਺෦ɺԾ਺෦ͷॱʹهࡌ جຊతʹ͸্هͷྲྀΕʹجͮ͘͜ͱͰ 10 ਐ਺ͷখ਺Λුಈখ਺఺ͷදه Ͱද͢͜ͱ͕Ͱ͖·͢ɻҎԼɺ2.5 ͷ୯ਫ਼౓Ͱͷුಈখ਺఺ԽΛྫʹྲྀΕͷ ֬ೝΛߦ͍·͢ɻ ·ͣɺᶃͰ͸ 2.5 Λ 2.5 = (−1)0 × 2.5 × 20 ͷΑ͏ʹද͠·͢ɻ࣍ʹᶄͰ ͸ (−1)0 × 2.5 × 20 Λ (−1)0 × 2.5 × 20 = (−1)0 × (1 + 0.25) × 2128−127 ͷΑ͏ʹม׵͠·͢ɻ ଓ͘ᶅͰ͸ූ߸෦ͷ 0 Λ 0ɺԾ਺෦ͷ 0.25 Λ 010 0000 0000 0000 000 0 0000ɺࢦ਺෦ͷ 128 Λ 1000 0000 Ͱද͠·͢ɻ͜ΕΒʹج͍ͮͯɺ010 0 0000 0010 0000 0000 0000 0000 0000 ͕ಘΒΕ·͢ɻ 45