機械学習のブルーオーシャン Core ML by 堤修一 #iOSDC Japan 2020

by shu223

Slide 1

Slide 1 text

ػցֶशͷϒϧʔΦʔγϟϯ Core ML

Slide 2

Slide 2 text

ࣗݾ঺հ • ϑϦʔϥϯεiOSΤϯδχΞ • GitHub, Twitter, note: shu223 • BOOTHͰ͍Ζ͍ΖຊΛग़ͯ͠·͢ • YouTube: ΤϯδχΞͱਓੜ

Slide 3

Slide 3 text

ۙ೥ͷٕज़తϒϨʔΫεϧʔ͸ ΄΅ػցֶशབྷΈ • ҩྍ • ࣗಈӡస • ೶ۀ • ϩϘοτ • etc.

Slide 4

Slide 4 text

ػցֶश΍Γ͍ͨʂ

Slide 5

Slide 5 text

૚ͷް͍ઌਓͨͪ • େֶ౳Ͱઐ໳తʹػցֶशΛֶΜͩਓͨͪ • ϑϧλΠϜͰػցֶशΤϯδχΞͱͯ͠ಇ͍͍ͯΔਓͨͪ • Kaggleͷڧ͍ਓୡ • ΦϯϥΠϯίʔεͰ΋ٕज़ॻͰ΋ػցֶशܥ͸௒ਓؾ

Slide 6

Slide 6 text

׬શͳΔϨουΦʔγϟϯ

Slide 7

Slide 7 text

զʑiOSΤϯδχΞʹ͸ ػցֶशͷϒϧʔΦʔγϟϯ ͕࢒͞Ε͍ͯΔ

Slide 8

Slide 8 text

ͦͷ໊΋ Core ML

Slide 9

Slide 9 text

Slide 10

Slide 10 text

! • ͳͥCore MLʁ • Core ML௒؆୯ͩ͠ɺԿΛࠓߋʁ

Slide 11

Slide 11 text

ͳͥCore MLʁ iOSσόΠεͰMLಈ͔͢ʹͯ͠΋͍Ζ͍Ζબ୒ࢶ͸͋Δ • PyTorch MobileʢLibTorchʣ • Tensor Flow for iOS • Tensor Flow Lite

Slide 12

Slide 12 text

iOSσόΠεͷੑೳΛ࠷΋׆͔ͤΔͷ ͸Core ML • PyTorch MobileʢLibTorchʣ -> CPU • Tensor Flow for iOS -> CPU • Tensor Flow Lite -> CPU, GPU • Core ML -> CPU, GPU, Neural Engine

Slide 13

Slide 13 text

Core ML௒؆୯ͩ͠ɺԿΛࠓߋʁ

Slide 14

Slide 14 text

ଟ͘ͷਓ͕৮ͬͨ͜ͱ͕͋Δͷ͸ɺ CoreML.framework • ֶशࡁΈͷը૾෼ྨϞσϧΛಈ͔͠ ͯΈ͚ͨͩͩͬͨΓ...

Slide 15

Slide 15 text

CoreML.framework͸࠷ऴతͳϞσϧΛiOSͰѻ͏෦෼͠ ͔ఏڙ͠ͳ͍

Slide 16

Slide 16 text

࣮຿Ͱ͸ʮCore MLϞσϧΛͭ͘Δʯ෦෼΋ؚΊͯશମΛݟ Δඞཁ͕͋Δ

Slide 17

Slide 17 text

ྫɿ ΧελϜ෺ମݕग़ • ਫ଒ؗͷڕ΍ಈ෺Ԃͷಈ෺ • ໊ࢗ΍ாථ • → ඪ४ػೳʹ͸ͳ͍ʢʮҰൠ෺ମݕ ग़ʯͳΒ͋Δ͕ʣ • → ϞσϧΛࣗ࡞͢Δඞཁ͕͋Δ

Slide 18

Slide 18 text

࣮Ҋ݅Λྫʹղઆ͠·͢ʂ

Slide 19

Slide 19 text

࣮ྫ1: طଘϞσϧͷߴ଎Խ

Slide 20

Slide 20 text

՝୊ɿ ෺ମݕग़ͷϦΞϧλΠϜॲཧ͕஗͍ ໿18fps

Slide 21

Slide 21 text

ঢ়گ • TensorFlowͰಠࣗσʔληοτͰֶशͨ͠.pbϞσϧΛiOS Ͱ௚઀ར༻ • TensorFlow for iOSʢC++Ͱॻ͔Εͨެࣜϥούʔʣར༻

Slide 22

Slide 22 text

ํ਑ɿ Core MLԽ͢Δ • TensorFlow for iOSʢC++Ͱॻ͔Εͨެࣜϥούʔʣ = CPU ॲཧ • → Core MLԽ͢Ε͹GPU + Neural EngineͰॲཧ͞ΕΔΑ͏ ʹͳΔ

Slide 23

Slide 23 text

Core MLϞσϧΛͲ͏࡞Δ͔ʁ

Slide 24

Slide 24 text

Core MLϞσϧΛͲ͏࡞Δ͔ʁ

Slide 25

Slide 25 text

TensorFlowϞσϧΛCore MLϞσϧʹม׵

Slide 26

Slide 26 text

Core ML Tools • ͞·͟·ͳϞσϧΛCore MLϞσϧ ʢ.mlmodelʣʹม׵͢Δπʔϧ • Φʔϓϯιʔε • https://github.com/apple/ coremltools • Pythonύοέʔδͱͯ͠͸ coremltools

Slide 27

Slide 27 text

coremltools 3.xͱ4 • 2020.9.9ݱࡏɺ4͸·ͩϕʔλ ʢ4.0b3ʣ • ࠓճ͸3.xϕʔεͰղઆ • Uniﬁed Converter • PyTorchϞσϧͷ௚઀ม׵

Slide 28

Slide 28 text

coremltoolsͷΠϯετʔϧ $ pip install -U coremltools

Slide 29

Slide 29 text

coremltoolsͰTensorFlowϞσϧΛม׵ import tfcoreml mlmodel = tfcoreml.convert(tf_model,...)

Slide 30

Slide 30 text

SSDϞσϧͷม׵ ʢ˞࣮Ҋ݅ͱ͸ผͷҰൠެ։ϞσϧͰ୅༻ͯ͠ղઆʣ

Slide 31

Slide 31 text

SSD: Single Shot Multibox Detector

Slide 32

Slide 32 text

ม׵ͯ͠ΈΔ import tfcoreml mlmodel = tfcoreml.convert( tf_model_path = tf_model_path, # .pbϞσϧͷύε mlmodel_path = mlmodel_path, # .mlmodelग़ྗύε input_name_shape_dict = in_dict, # ೖྗϊʔυ໊ͱshape output_feature_names = out_names, # ग़ྗϊʔυ໊ ...)

Slide 33

Slide 33 text

Τϥʔൃੜ ValueError: Graph has cycles.

Slide 34

Slide 34 text

ϞσϧͷάϥϑΛݟͯΈΔ Netronར༻

Slide 35

Slide 35 text

ϞσϧͷάϥϑΛݟͯΈΔ Netronར༻ʢ֦େʣ

Slide 36

Slide 36 text

ϞσϧͷάϥϑΛݟͯΈΔ Tensorboardར༻

Slide 37

Slide 37 text

αϒάϥϑΛ੾Γग़͢

Slide 38

Slide 38 text

αΠΫϧͷͳ͍෦෼Λ੾Γग़͢ • tensorboardͰϞσϧͷத਎Λݟͯɺ੾Γग़͢෦෼ͷ͋ͨΓ Λ͚ͭΔ • Ͳ͕͜୅༻Մೳ͔ʁ • Preprocessor෦෼͸ม׵ίϚϯυͷΦϓγϣϯͰ෇༩ • MultiGridAnchorGenerator͸੩తͳ஋Ͱ୅ସՄೳ • Postprocessor͸ಉ༷ͷॲཧΛSwiftͰ࣮૷Մೳ

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

strip_unused_lib ͰαϒάϥϑΛ੾Γग़͢ from tensorflow.python.tools import strip_unused_lib in_names = ['Preprocessor/sub'] # ೖྗϊʔυ໊ͷ഑ྻ out_names = ['Squeeze', 'concat_1'] # ग़ྗϊʔυ໊ͷ഑ྻ gdef = strip_unused_lib.strip_unused( input_graph_def = original_gdef, input_node_names = in_names, output_node_names = out_names, placeholder_type_enum = dtypes.float32.as_datatype_enum)

Slide 41

Slide 41 text

৽͍͠Ϟσϧͷάϥϑ

Slide 42

Slide 42 text

Core MLϞσϧ΁ม׵͢Δ import tfcoreml input_shapes = {"Preprocessor/sub:0":[1,300,300,3]} output_tensor_names = ['Squeeze:0', 'concat_1:0'] mlmodel = tfcoreml.convert( tf_model_path = tf_model_path, mlmodel_path = ml_model_path, input_name_shape_dict = in_shapes, output_feature_names = out_names, image_input_names = image_in_names)

Slide 43

Slide 43 text

੾Γऔͬͨ෦෼ͷิర

Slide 44

Slide 44 text

੾ΓऔͬͨPreprocessor෦෼Λม׵ΦϓγϣϯͰิర mlmodel = tfcoreml.convert( tf_model_path = tf_model_path, mlmodel_path = ml_model_path, input_name_shape_dict = in_shapes, output_feature_names = out_names, image_input_names = image_in_names, red_bias = -1.0, green_bias = -1.0, blue_bias = -1.0, image_scale = 2./255)

Slide 45

Slide 45 text

MultiGridAnchorGenerator෦෼ͷิర • ੩తͳ஋Ͱ୅ସՄೳ • ֘౰෦෼ͷग़ྗΛಘ͓ͯ͘

Slide 46

Slide 46 text

Postprocessor෦෼ͷิర • ಉ༷ͷॲཧΛSwiftͰ࣮૷͢Δ

Slide 47

Slide 47 text

SSDϞσϧม׵Ͱ΍ͬͨ͜ͱͷ੔ཧ

Slide 48

Slide 48 text

Core MLϞσϧ΁Ҡߦͨ݁͠Ռ • ໿300%ͷߴ଎Խʹ੒ޭ ! ʢ18fps -> 55fpsʣ • ͞ΒͳΔ଎౓޲্ͷݟࠐΈ΋͋Δ

Slide 49

Slide 49 text

࣮ྫ1ͷ࢓ࣄͰඞཁͳ஌ࣝ • iOSσόΠε, iOS SDK, Swift • Core ML Tools • TF, Python, ػցֶश͸গ͠ • → iOSΤϯδχΞ΋ઓ͑Δʂ

Slide 50

Slide 50 text

࣮ྫ2: σʔληοτ࡞੒ ɾֶश͔Βߦ͏

Slide 51

Slide 51 text

·ͱΊ • CoreML.framework͸iOSʹϞσϧΛ૊ΈࠐΉͱ͜Ζ͚ͩ • ࣮຿Ͱ͸Ϟσϧ࡞੒ΛؚΉશମΛݟΔ • Core MLϞσϧม׵Λ୲͏Core ML Tools͕؊

Slide 52

Slide 52 text

Core ML Tools iOS × Deep Learning TensorFlow, Kerasͱ coremltools ʹΑΔΧελϜϞσϧߏங ࣮ફೖ໳ Core ML Toolsʹ͍ͭͯ ΋ͬͱֶͿ • ެࣜϦϙδτϦͷαϯϓϧ • ެࣜυΩϡϝϯτ • ॻ੶ʮCore ML Tools࣮ફೖ໳ʯ • ٯҾ͖Ϩγϐ͖ͭ

Slide 53

Slide 53 text

iOSͷGPU/Metalʹ͍ͭͯ ֶͿ • εϥΠυʮඈͼಓ۩Ͱ͸ͳ͍Metalʯ ʢiOSDC 2017ʣ • ॻ੶ʮMetalೖ໳ʯ

Slide 54

Slide 54 text

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ