ColorNet: Investigating the importance of color spaces for image classification

Transcript

1. @mocobt ColorNet: Investigating the importance of color spaces for image

   classification [Gowda et al. ACCV 2018]

2. • OpticsدΓͷCV & CGܥR&D - ۙʑത࢜՝ఔߦ͖͍ͨ" • ΞϓϦέʔγϣϯ࿦จΛΑ͘ಡΉ • ຖ݄࿦จͷࡶSummary΍ͬͯ·͢(αϘϦؾຯ)

   - https://scrapbox.io/HumanComputerInteraction/ About me @mocobt

3. CNNʹ࠷దͳ৭ۭؒ͸Կ͔ʁ ࠓճͷςʔϚ !3

4. ৭ۭؒͱ͸ʁ ৭ۭؒ: ৭Λද͢ෳ਺ͷϕΫτϧ͔Β੒Δۭؒ (e.g.) RGB: ҎԼͰදݱ͞ΕΔ৭ۭؒ R: Red (0~255) G:

   Green (0~255) B: Blue (0~255) !4

5. ৭ۭؒ͸ม׵Մೳʂ RGB !5 HSV

6. ৭ʑͳ৭ۭؒ !6 RGB HSV XYZ LAB, LCH YIQ, YUV, YPbPr,

   YCbCr CMYK R: Red (੺) G: Green (྘) B: Blue (੨) ༻్: ͳͲ H: Hue (৭૬) S: Saturation (࠼౓) V: Value (ً౓) ༻్: , ৭ݟຊ X: ෳ਺ਲ਼ମࡉ๔ͷײ౓ Y: ً౓ Z: Sਲ਼ମࡉ๔ (੨৭ड༰) ༻్: උߟ: ෺ཧྔΛߟྀ L: Luminance (໌౓) A: ิ৭༻ (੺&྘) B: ิ৭༻ (ԫ&੨) ༻్: Ϩλονιϑτ උߟ: LCH͸೿ੜܥ Y: ً౓ U: ৭ࠩ V: ৭ࠩ ༻్: ಈը૾ѹॖ උߟ: େମ೿ੜܥ C: Cyan (γΞϯ) M: Magenta (Ϛθϯλ) Y: Yellow (ԫ) K: Key plate (ࠇ) ༻్:

7. ঺հ࿦จ: ColorNet [Gowda et al. ACCV 2018] !7 എܠ: CNNͷը૾෼ྨͰ͸৭ۭ͕ؒ༗ޮ׆༻͞Ε͍ͯͳ͔ͬͨ

   ৽نੑ: - ৭ۭ͕ؒ෼ྨਫ਼౓ʹٴ΅͢Өڹௐࠪ → ͋ΔΫϥεͷਫ਼౓͕ಛఆۭؒͰ޲্ - ෳ਺৭ۭؒը૾Λೖྗͱ͢ΔCNN: ColorNetΛఏҊ → ߴਫ਼౓ & গύϥϝλ ੍ݶ: ৭ۭؒΛҠ͢ܭࢉίετେ → ਫ਼౓ͱܭࢉ͕࣌ؒτϨʔυΦϑ ॴײ: - ͜ΕΑΓޮ཰తͳΞʔΩςΫνϟ͸طʹ૬౰਺ଘࡏ͢Δ͕ɼൃ૝͸໘ന͍ - ը૾෼ྨҎ֎ͷλεΫͰͲ͏ͳΔͷ͔ؾʹͳΔ - CNNʹదͨ͠৭ۭؒΛ৽ͨʹఆٛͯ͠Έͯ΋໘ന͍͔΋͠Εͳ͍

8. Ϋϥε෼ྨ࣮ݧ: ؀ڥ CIFAR-10 (10 class Dataset)Ͱೖྗͷ৭ۭؒΛม׵ͨ͠ࡍͷਫ਼౓ൺֱ !8 ΞʔΩςΫνϟ͸͜Μͳײ͡ɽ࣮ݧ؀ڥ͸ෆ໌

9. Ϋϥε෼ྨ࣮ݧ: ݁Ռ LAB͕Ұ൪ྑ͍݁Ռʹͳͬͨʂ ଞۭؒͷAccuracy͸ͱΜͱΜ !9

10. Ϋϥε෼ྨ࣮ݧ: ࠞಉߦྻ ֤৭ۭ͕ؒҟͳΔग़ྗΛ͍ͯ͠Δ͜ͱ͕֬ೝͰ͖Δ !10

11. ෳ਺ۭؒΛ࢖ͬͨΒਫ਼౓Ͳ͏ͳΔͷʁ !11 جຊతʹՃॏฏۉΛऔΕ͹औΔ΄Ͳਫ਼౓޲্ +͸Ճॏฏۉ ࢖༻ۭ͕ؒ૿͑Δͱ֓Ͷਫ਼౓޲্

12. ࢖͏ͱਫ਼౓͕མͪΔۭؒ΋… !12 XYZΛ࢖͏ͱAccuracy Down

13. Late Fusion v.s. Early Fusion !13 Late Fusionͷਫ਼౓͕࠷ྑ CNN CNN

    FC Late Fusion CNN Early Fusion

14. ColorNet ࣮ݧͰಘΒΕͨ஌ݟΛ૊Έ߹ΘͤͨΫϥε෼ྨ༻ΞʔΩςΫνϟ (1) Late FusionͰ֤৭ۭؒͷग़ྗΛ૊Έ߹ΘͤΔͱਫ਼౓޲্ (2) ͨͩ͠ɼXYZΛ࢖͏ͱѱԽ͢ΔͷͰ࢖Θͳ͍ !14 DenseNet FC

    Output Input RGB Image LAB Image HSV Image YUV Image YCbCr Image HED Image YIQ Image DenseNet DenseNet DenseNet DenseNet DenseNet DenseNet

15. ColorNetͷධՁ !15 (࣮ݧઃఆ͕ṖͳͷͰ͓ؾ࣋ͪఔ౓ʹߟ͍͑ͯͩ͘͞) 4ͭͷDatasetͰ෼ྨਫ਼౓ධՁ →ߴਫ਼౓ & গύϥϝλ

16. ·ͱΊ: ColorNet [Gowda et al. ACCV 2018] !16 എܠ: CNNͷը૾෼ྨͰ͸৭ۭ͕ؒ༗ޮ׆༻͞Ε͍ͯͳ͔ͬͨ

    ৽نੑ: - ৭ۭ͕ؒ෼ྨਫ਼౓ʹٴ΅͢Өڹௐࠪ → ͋ΔΫϥεͷਫ਼౓͕ಛఆۭؒͰ޲্ - ෳ਺৭ۭؒը૾Λೖྗͱ͢ΔCNN: ColorNetΛఏҊ → ߴਫ਼౓ & গύϥϝλ ੍ݶ: ৭ۭؒΛҠ͢ܭࢉίετେ → ਫ਼౓ͱܭࢉ͕࣌ؒτϨʔυΦϑ ॴײ: - ͜ΕΑΓޮ཰తͳΞʔΩςΫνϟ͸طʹ૬౰਺ଘࡏ͢Δ͕ɼൃ૝͸໘ന͍ - ը૾෼ྨҎ֎ͷλεΫͰͲ͏ͳΔͷ͔ؾʹͳΔ - CNNʹదͨ͠৭ۭؒΛ৽ͨʹఆٛͯ͠Έͯ΋໘ന͍͔΋͠Εͳ͍

17. Q&A • આ໌ૣ͗ͯ͢ԿݴͬͯΔ͔Θ͔Μͳ͍ - https://mocobt.hatenablog.com/entry/2019/07/29/005512 ʹ·ͱΊͨͷͰڐ͍ͯͩ͘͠͞ • ৭ۭؒͷ͜ͱ΋ͬͱ஌Γ͍ͨ - https://mocobt.hatenablog.com/entry/2019/07/28/205710ΛಡΜͰ͍ͩ͘͞

    • ΦϨɼλλϛίϛΰϦϥ….ΠϚεά…λλϛίϜ…. - ஶऀ࣮૷͸ͳ͍Α͏Ͱ͕͢ɼhttps://github.com/jorge-pessoa/pytorch-colors ͕ࢀߟʹͳΓ·͢ • ͳΜͰ͜ͷ࿦จΛબΜͩͷʁ - ৭࠼޻ֶΛ෮शͨͯ͘͠ɼͦͷΩοΧέͮ͘ΓͷͨΊͰ͢