ML Ops Study 2

Ece52fe9ce913851256726020707febd?s=47 Keiji ARIYAMA
May 29, 2018
Technology
0
82

ML Ops Study 2

5月29日にクックパッド株式会社で開催されたML Ops Study #2の発表資料です。

Ece52fe9ce913851256726020707febd?s=128

Keiji ARIYAMA

May 29, 2018
Tweet

More Decks by Keiji ARIYAMA

See All by Keiji ARIYAMA
Ece52fe9ce913851256726020707febd?s=48 keiji
0
23
Ece52fe9ce913851256726020707febd?s=48 keiji
1
66
Ece52fe9ce913851256726020707febd?s=48 keiji
1
250
Ece52fe9ce913851256726020707febd?s=48 keiji
1
380
Ece52fe9ce913851256726020707febd?s=48 keiji
0
290
Ece52fe9ce913851256726020707febd?s=48 keiji
0
120
Ece52fe9ce913851256726020707febd?s=48 keiji
0
60
Ece52fe9ce913851256726020707febd?s=48 keiji
6
1.1k
Ece52fe9ce913851256726020707febd?s=48 keiji
1
240

Other Decks in Technology

See All in Technology
A2cd163fd3cfca5204b0d8f0c17eea29?s=48 yujif
2
100
53850955f15249a1a9dc49df6113e400?s=48 line_developers
0
480
53850955f15249a1a9dc49df6113e400?s=48 line_developers
1
760
9bf153d8c0ce36ba6d9d20c2914b70f4?s=48 shimastripe
11
560
53e2d354b3299d64a54af680865516d5?s=48 satotakeshi
3
410
Cb88f765dd38cd942c39aacb5abbea06?s=48 ufoo68
0
110
74cec195bfb6cb5165256d88cb7fcf0f?s=48 miu_crescent
5
1k
921515dc03f89bafe3d8f5d0b3ca0b74?s=48 mmarukaw
0
470
53850955f15249a1a9dc49df6113e400?s=48 line_developers
2
170
0b28f5164b540e5a31bc63cca39b2917?s=48 curry9999
0
150
Aa254f26f91d4776b9cd19a7ce632867?s=48 ciniml
0
120
E2640ce3612d9e7848dc957b519d8ca0?s=48 polar3130
2
240

Featured

See All Featured
C44e1f7e22c3f23cff7bc130871047ef?s=48 eileencodes
112
24k
245cee81a9c424266e5e401d844ea881?s=48 lara
13
2.3k
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
165
6.8k
E14f55d3f939977cecbf51b64ff6f861?s=48 keithpitt
399
20k
A9179349dd2bdc67f377719f56d85656?s=48 garrettdimon
284
110k
1f74b13f1e5c6c69cb5d7fbaabb1e2cb?s=48 sferik
607
54k
F68cb25ae3e5c2106997454b13b7737d?s=48 brad_frost
154
6.2k
Dafc4723e9a1c067796c0fec6f509774?s=48 lemiorhan
621
40k
A60068bce2e73de3a37ca9d2dbe36092?s=48 bryan
28
3.1k
766b9746b59e6f09e280cd33cf4ed419?s=48 skipperchong
4
490
1519587b1a0febb658c36c94f6272b0d?s=48 roundedbygravity
84
7.7k
Bfd6b681ec6e8c9bef82ff0521c364f7?s=48 kastner
52
1.8k

Transcript

  1. C-LIS CO., LTD.

  2. .-0QT4UVEZ BUΫοΫύουגࣜձࣾ   5FOTPS'MPXͷ܇࿅ࡁΈϞσϧΛ
 "OESPJEΞϓϦʹࡌͤΔͱ͖ʹ
 ۤ࿑ͨ͠࿩ 5FOTPS'MPXͰझຯͷը૾ऩूαʔόʔΛ࡞Δ೥݄߸

  3. C-LIS CO., LTD.   ༗ࢁܓೋʢ,FJKJ"3*:"."ʣ $-*4$0 -5% Photo :

    Koji MORIGUCHI (MORIGCHOWDER) "OESPJEΞϓϦ։ൃνϣοτσΩϧ ػցֶश͸ͪΐͬͱ΍ͬͨ͜ͱ͋Γ·͢

  4. C-LIS CO., LTD.  

  5. C-LIS CO., LTD.   ΍ͬ΂

  6.   ຊ൪؀ڥ΁ͷ౤ೖܦݧ͕ඞཁͳΒ ౤ೖͯ͠͠·͑͹͍͍͡Όͳ͍

  7. ؟ ڸ ͬ ່ ˜ࠜઇΕ͍

  8.   ޷Έͷ؟ڸ່ͬը૾Λ
 ࣗಈͰऩू͍ͨ͠

  9. ̎Ϋϥε෼ྨ   1 0

  10.   ݱࡏͷγεςϜ

  11.   ධՁ༻ αʔόʔ ܇࿅ɾֶश༻αʔόʔ σʔληοτసૹ ʢTFRecordʣ ֶशࡁΈ ύϥϝʔλʔऔಘ ը૾औಘ

    ը૾औಘ ϥϕϧ ෇͚ σʔληοτ؅ཧ αʔόʔ σʔλऩूݩ αʔϏε ը૾औಘ ϥϕϧ ෇͚ Android
 ΞϓϦ

  12. σʔληοτ؅ཧαʔόʔ $16"UISPO/FP()[ .FNPSZ(# 4UPSBHF44%(#
 )%%5#  3"*%  

  13. σʔληοτ؅ཧαʔόʔͷ໾ׂ ը૾σʔλͷऩू ϝλσʔλʢΞϊςʔγϣϯɾϥϕϧʣͷ؅ཧ "1*ͷఏڙ ֶश༻σʔλʢ5'3FDPSEʣͷੜ੒  

  14. ը૾σʔλͷऩू  

  15. ϝλσʔλͷ؅ཧ  

  16. ϝλσʔλͷ؅ཧ   label: 2 left: 283 top: 190 right:

    435 bottom:301 = 1.0

  17. ϥϕϧͷछྨ PSJHJOBM@BSU OTGX GBWPSJUF QIPUP JMMVTU DPNJD   GBDF

    GFNBMF NFHBOF TDISPPM@VOJGPSN CMB[FS@VOJGPSN TBJMPS@VOJGPSN HM LFNPOP NBMF CM DBU EPH GPPE EJTMJLF

  18. "1*ͷఏڙ ը૾Ϧετͷऔಘ ը૾ͷऔಘ ը૾ͷݕࡧʢϥϕϧʣ ϥϕϧͷઃఆ ϥϕϧະઃఆը૾ͷϦετΛऔಘ σʔλऔಘݩ5XJUUFS*%ͷ௥Ճɾ࡟আ  

  19. ֶश༻σʔλͷੜ੒ 5'3FDPSEܗࣜPS+1&( $47ܗࣜ ը૾ͷϦαΠζ͸͜ͷஈ֊Ͱߦ͏ʢτϥϑΟοΫΛ௿ݮ   $ python ./create_dataset.py \

    --base_dir /dataset/source/ \ --output_dir ~/tfrecords_classifier \ --image_size 256 \ --tag_names megane,nsfw,favorite,illust

  20. σʔλͷάϧʔϓԽ   0 1 2 3 4 5 6

    7 8 9 σʔληοτ ςετσʔληοτ

  21.   ධՁ༻ αʔόʔ ܇࿅ɾֶश༻αʔόʔ σʔληοτసૹ ʢTFRecordʣ ֶशࡁΈ ύϥϝʔλʔऔಘ ը૾औಘ

    ը૾औಘ ϥϕϧ ෇͚ σʔληοτ؅ཧ αʔόʔ σʔλऩूݩ αʔϏε ը૾औಘ ϥϕϧ ෇͚ Android
 ΞϓϦ

  22.   ֶश༻αʔόʔ

  23. { "tag_name": "megane", "train_catalog_numbers": "0,1,2,3,4,5,6,7,8", "eval_catalog_numbers": "9", "data_augmentation": { "random_crop":

    false, "random_colorize": true } }   ֶश༻ͷઃఆϑΝΠϧ

  24. ֶशͱݕূͷ࣮ߦ   $ CUDA_VISIBLE_DEVICES=0,1 python ./train.py \ --learning_config config_megane.json

    \ --tfrecords_dir ~/tfrecords_classifier \ --train_dir ~/train_single_discriminator \ --summary_dir ~/summary_single_discriminator \ --batch_size 64 \ --learning_rate 0.0001 \ --num_gpus 2 \ --max_step 100000 $ CUDA_VISIBLE_DEVICES=2 python ./eval.py \ --learning_config config_megane.json \ --tfrecords_dir ~/tfrecords_classifier \ --train_dir ~/train_single_discriminator \ --summary_dir ~/summary_single_discriminator $ tensorboard \ --logdir ~/summary_single_discriminator/megane/

  25.  

  26.   ධՁ༻ αʔόʔ ܇࿅ɾֶश༻αʔόʔ σʔληοτసૹ ʢTFRecordʣ ֶशࡁΈ ύϥϝʔλʔऔಘ ը૾औಘ

    ը૾औಘ ϥϕϧ ෇͚ σʔληοτ؅ཧ αʔόʔ σʔλऩूݩ αʔϏε ը૾औಘ ϥϕϧ ෇͚ Android
 ΞϓϦ

  27.   ධՁ༻αʔόʔ

  28. ධՁͷ࣮ߦ   $ python3 client.py \ --tag_name megane \

    --train_base_path ~/train_single_discriminator \ --train_file_name precision-0.956463/megane.ckpt-294000 \ --batch_size 16 \ --limit_batch 100

  29.   ධՁ༻ αʔόʔ ܇࿅ɾֶश༻αʔόʔ σʔληοτసૹ ʢTFRecordʣ ֶशࡁΈ ύϥϝʔλʔऔಘ ը૾औಘ

    ը૾औಘ ϥϕϧ ෇͚ σʔληοτ؅ཧ αʔόʔ σʔλऩूݩ αʔϏε ը૾औಘ ϥϕϧ ෇͚ Android
 ΞϓϦ

  30.   "OESPJEΞϓϦ

  31.  

  32.  

  33.  

  34.  

  35.   ௚ۙͷ՝୊

  36.   ෆద੾ը૾͕ଟ͗͢Δ

  37. ෆద੾ʢ/4'8ʣը૾ͱ؟ڸը૾ /4'8    positive: 36,083 → 7.17%
 negative:

    466,738 ؟ڸ    positive: 23,559 → 2.44%
 negative: 938,563  

  38.   ෆద੾ը૾ϑΟϧλʔΛ ΞϓϦʹ૊ΈࠐΉ

  39. NPEFM   NSFW positive: 5,628 negative: 17,253

  40. NPEFM   ֶशࡁΈύϥϝʔλʔϑΝΠϧ ֶशࡁΈϞσϧ .pb 170MB

  41. NPEFM@MJUF   ֶशࡁΈύϥϝʔλʔϑΝΠϧ ֶशࡁΈϞσϧ .pb 10.7MB

  42. NPEFM@MJUF  

  43. Ϟσϧͷߏ଄   .pb input result 128x128x3 1

  44. private val IMAGE_WIDTH = 128 private val IMAGE_HEIGHT = 128

    private val IMAGE_CHANNEL = 3 private val IMAGE_BYTES_LENGTH = IMAGE_WIDTH * IMAGE_HEIGHT * IMAGE_CHANNEL val imageByteBuffer: ByteBuffer = ByteBuffer.allocate(IMAGE_BYTES_LENGTH) val scaledBitmap = Bitmap.createScaledBitmap(bitmap, IMAGE_WIDTH, IMAGE_HEIGHT, false) scaledBitmap.copyPixelsToBuffer(imageByteBuffer)   ը૾ΛόοϑΝʹ֨ೲ

  45. val resultArray = FloatArray(1) fun recognize(imageByteArray: ByteArray): Float { val

    start = Debug.threadCpuTimeNanos() tfInference.feed("input", imageByteArray, imageByteArray.size.toLong()) tfInference.run(arrayOf("result")) tfInference.fetch("result", resultArray) val elapsed = Debug.threadCpuTimeNanos() - start Log.d(TAG, "Elapsed: %d ns".format(elapsed)) return resultArray[0] }   GFFESVOGFUDI

  46. private val IMAGE_WIDTH = 128 private val IMAGE_HEIGHT = 128

    private val IMAGE_CHANNEL = 3 private val IMAGE_BYTES_LENGTH = IMAGE_WIDTH * IMAGE_HEIGHT * IMAGE_CHANNEL val imageByteBuffer: ByteBuffer = ByteBuffer.allocate(IMAGE_BYTES_LENGTH) val scaledBitmap = Bitmap.createScaledBitmap(bitmap, IMAGE_WIDTH, IMAGE_HEIGHT, false) scaledBitmap.copyPixelsToBuffer(imageByteBuffer) scaledBitmap.recycle()   ݪҼΒ͖͠΋ͷ ˢ"MQIBνϟϯωϧ͕ೖ͍ͬͯΔ

  47. Ϟσϧͷߏ଄   .pb input result 128x128x4 1

  48. with tf.Graph().as_default() as g: image_ph = tf.placeholder( tf.uint8, [model.IMAGE_SIZE *

    model.IMAGE_SIZE * 4], name='input') image = tf.cast(image_ph, tf.float32) image = tf.reshape( image, [model.IMAGE_SIZE, model.IMAGE_SIZE, 4]) image = image[:, :, :3]   QCग़ྗ࣌ʹDIΛड͚ೖΕΔΑ͏ʹάϥϑΛมߋ

  49.   σϞ

  50. 'PPE(BMMFSZ   https://github.com/keiji/food_gallery_with_tensorflow ΪϟϥϦʔʹอଘ͞Ε͍ͯΔ৯෺ͷը૾Λදࣔ http://techlife.cookpad.com/entry/2017/09/14/161756 ΫοΫύου։ൃऀϒϩά ྉཧ͖Ζ͘ʹ͓͚Δྉཧʗඇྉཧ൑ผϞσϧͷৄࡉ

  51.   ૊ΈࠐΜͰΈ͚ͨΕͲɺ ਫ਼౓͸͋·Γߴ͘ͳ͍ʜʜ

  52. ࠓޙͷ՝୊ 5FOTPS'MPX-JUF΁ͷҠߦ ߴਫ਼౓ͷϞσϧͷѹॖʢল༰ྔԽʣ 1SVOJOH 2VBOUJ[BUJPO %JTUJMMBUJPO   ML Kit:

    Machine Learning SDK for mobile developers (Google I/O '18) https://youtu.be/Z-dqGRSsaBs?t=32m10s

  53. ࠓޙͷ՝୊ ֶशαΠΫϧΛࣗಈԽ͍͖͍ͯͨ͠ɻ ʢఆظతʹֶशσʔλͷੜ੒ͱసૹΛߦ͍ɺ࠶ֶश͢ΔͳͲʣ σʔλϕʔεͷߴ଎Խ
 ɹϥϕϧݕࡧ͕ඇৗʹ஗͍ͷ͕՝୊ɻઃܭΛݟ௚͢ඞཁ͋Γ
 ɹݕࡧΠϯσοΫεͷ(PPHMF$MPVE4UPSFҠߦΛݕ౼
  

  54. ࠓޙͷ՝୊ σʔληοτ؅ཧαʔόʔͷߋ৽ 5FOTPS'MPXΑΓɺ$16ʹ*OUFM"79͕ඞਢʹͳͬͨʢQJQ൛ʣ  

  55. C-LIS CO., LTD. ຊࢿྉ͸ɺ༗ݶձࣾγʔϦεͷஶ࡞෺Ͱ͢ɻຊࢿྉͷશ෦ɺ·ͨ͸Ұ෦ʹ͍ͭͯɺஶ࡞ऀ͔ΒจॻʹΑΔڐ୚Λಘͣʹෳ੡͢Δ͜ͱ͸ې͡ΒΕ͍ͯ·͢ɻ 5IF"OESPJE4UVEJPJDPOJTSFQSPEVDFEPSNPEJpFEGSPNXPSLDSFBUFEBOETIBSFECZ(PPHMFBOEVTFEBDDPSEJOHUPUFSNTEFTDSJCFEJOUIF$SFBUJWF$PNNPOT"UUSJCVUJPO-JDFOTF ֤੡඼໊ɾϒϥϯυ໊ɺձ໊ࣾͳͲ͸ɺҰൠʹ֤ࣾͷ঎ඪ·ͨ͸ొ࿥঎ඪͰ͢ɻຊࢿྉதͰ͸ɺ˜ɺšɺäΛׂѪ͍ͯ͠·͢ɻ 5IF"OESPJESPCPUJTSFQSPEVDFEPSNPEJpFEGSPNXPSLDSFBUFEBOETIBSFECZ(PPHMFBOEVTFEBDDPSEJOHUPUFSNTEFTDSJCFEJOUIF$SFBUJWF$PNNPOT"UUSJCVUJPO-JDFOTF