20161120-MLHands-on

Ece52fe9ce913851256726020707febd?s=47 Keiji ARIYAMA
November 20, 2016

 20161120-MLHands-on

2016年11月20日にGCPUG Tokushimaで開催した機械学習ハンズオンの資料です。

犬猫の画像を分類する畳み込みニューラルネットワーク(CNN)を構築します。
データセットにはCNNThe Oxford-IIIT Pet Datasetを使います。

Ece52fe9ce913851256726020707febd?s=128

Keiji ARIYAMA

November 20, 2016
Tweet

Transcript

  1. C-LIS CO., LTD. ػցֶशϋϯζΦϯ ($16(5PLVTIJNB

  2. C-LIS CO., LTD.  ༗ࢁܓೋʢ,FJKJ"3*:"."ʣ $-*4$0 -5% "OESPJEΞϓϦ։ൃνϣοτσΩϧ ػցֶशॳ৺ऀ Photo

    : Koji MORIGUCHI (AUN CREATIVE FIRM)
  3. C-LIS CO., LTD. TM 

  4.  10% 1SJOU0O%FNBOE ిࢠॻ੶ Impress R&D͔Βൃചத http://amzn.to/2axRog

  5.  *OUSPEVDUJPO ʰػցֶशʱͱ͸ͳʹ͔

  6. ػցֶशʢ͖͔͍͕͘͠Ύ͏ɺӳNBDIJOFMFBSOJOHʣͱ͸ɺਓ޻஌ೳʹ͓͚ Δݚڀ՝୊ͷҰͭͰɺਓ͕ؒࣗવʹߦ͍ͬͯΔֶशೳྗͱಉ༷ͷػೳΛίϯ ϐϡʔλͰ࣮ݱ͠Α͏ͱ͢Δٕज़ɾख๏ͷ͜ͱͰ͋Δɻ  ػցֶश ʢWikipedia: https://ja.wikipedia.org/wiki/%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92ʣ

  7. σΟʔϓϥʔχϯάɺਂ૚ֶशʢӳEFFQMFBSOJOHʣͱ͸ɺଟ૚ߏ଄ͷχϡʔ ϥϧωοτϫʔΫʢσΟʔϓχϡʔϥϧωοτϫʔΫɺӳEFFQOFVSBM OFUXPSLʣΛ༻͍ͨػցֶशͰ͋Δɻ  σΟʔϓϥʔχϯά ʢWikipedia: https://ja.wikipedia.org/wiki/%E3%83%87%E3%82%A3%E3%83%BC%E3%83%97%E3%83%A9%E3%83%BC%E3%83%8B%E3%83%B3%E3%82%B0ʣ

  8. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳOFVSBMOFUXPSL // ͸ɺ೴ػೳʹ ݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱ Λ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ  χϡʔϥϧωοτϫʔΫ ʢWikipedia: https://ja.wikipedia.org/wiki/%E3%83%8B%E3%83%A5%E3%83%BC%E3%83%A9%E3%83%AB%E3%83%8D%E3%83%83%E3%83%88%E3%83%AF%E3%83%BC%E3%82%AFʣ

  9.  ཁ͢Δʹ਺ֶ

  10. C-LIS CO., LTD. TM 5FOTPS'MPXJTBO0QFO4PVSDF4PGUXBSF-JCSBSZ GPS.BDIJOF*OUFMMJHFODF 

  11.  ϋϯζΦϯ

  12. w 1ZUIPOʢ(PPHMF$PNQVUFS1MBUGPSN5PPMT͕ରԠͷͨΊʣ w ґଘύοέʔδ w TJY w 1*-1JMMPX w MYNM

    w QZUIPOHqBHT w 5FOTPS'MPXɹɹ˞ཁ֬ೝ  ։ൃ؀ڥͷ֬ೝ
  13.  ͋ΔͱΑ͍΋ͷ https://www.jetbrains.com/pycharm/

  14.  ը૾෼ྨ ʢग़యIUUQTHJUIVCDPNDB[BMBNOJTUʣ ʢग़యIUUQTXXXDTUPSPOUPFEV_LSJ[DJGBSIUNMʣ MNIST CIFAR-10

  15.  ݘೣ෼ྨ

  16.  σʔλͱਖ਼ղϥϕϧΛ༻ҙ͢Δ  σʔλΛ5'3FDPSEܗࣜʹม׵͢Δ  ϞσϧΛ࡞੒͢Δ  σʔλΛධՁʢJOGFSFODFʣ͢Δ  ϞσϧΛ܇࿅͢Δ

     ύϥϝʔλʔΛอଘ͢Δ  ܇࿅ࡁΈϞσϧΛݕূ͢Δ  େ·͔ͳखॱ
  17.  εςοϓ̍ σʔλͱਖ਼ղϥϕϧΛ༻ҙ͢Δ

  18.  σʔληοτͷμ΢ϯϩʔυ http://www.robots.ox.ac.uk/~vgg/data/pets/

  19.  JNBHFT

  20.  BOOPUBUJPOT ೣ: Abyssinian ೣ: Ragdoll ೣ: Russian_Blue ݘ: Shiba

    ݘ: saint_bernard ݘ: samoyed
  21.  30*ʢ3FHJPO0G*OUFSFTUʣ

  22.  ίʔυͷμ΢ϯϩʔυ IUUQTHPPHM,TW,I2

  23.  εςοϓ̎ σʔλΛ5'3FDPSEܗࣜʹม׵͢Δ

  24.  લॲཧ Abyssinian_1.jpg

  25. <annotation> <folder>OXIIIT</folder> <filename>Abyssinian_1.jpg</filename> <source> <database>OXFORD-IIIT Pet Dataset</database> <annotation>OXIIIT</annotation> <image>flickr</image> </source>

    <size> <width>600</width> <height>400</height> <depth>3</depth> </size> <segmented>0</segmented>  BOOPUBUJPOTYNMT <object> <name>cat</name> <pose>Frontal</pose> <truncated>0</truncated> <occluded>0</occluded> <bndbox> <xmin>333</xmin> <ymin>72</ymin> <xmax>425</xmax> <ymax>158</ymax> </bndbox> <difficult>0</difficult> </object> </annotation> Abyssinian_1.xml
  26.  30*ͷ੾Γൈ͖

  27.  ίʔυΛมߋ bndboxͷ֤஋͔Βը૾Λ੾Γൈ͍ͯอଘ͢Δ dataset/crop_roi_image.py

  28. FLAGS = gflags.FLAGS
 gflags.DEFINE_string('image_dir', None, 'ॲཧ͢Δը૾ͷσΟϨΫτϦ')
 gflags.DEFINE_string('xml_dir', None, 'ॲཧ͢ΔXMLͷσΟϨΫτϦ')
 gflags.DEFINE_string('output_dir',

    None, 'ग़ྗ͢ΔσΟϨΫτϦ')  HqBHT $ python crop_roi_image.py \ --image_dir /Users/keiji_ariyama/Downloads/catsanddogs/images \ --xml_dir /Users/keiji_ariyama/Downloads/catsanddogs/annotations/xmls \ --output_dir /Users/keiji_ariyama/Desktop/test
  29.  ը૾ʹϥϕϧΛ෇Ճͯ͠5'3FDPSE΁ม׵ + Cat (1) Abyssinian_1.jpg TFRecord

  30. #Image CLASS-ID SPECIES BREED ID #ID: 1:37 Class ids #SPECIES:

    1:Cat 2:Dog #BREED ID: 1-25:Cat 1:12:Dog #All images with 1st letter as captial are cat images #images with small first letter are dog images Abyssinian_196 1 1 1 Abyssinian_197 1 1 1 Abyssinian_19 1 1 1 Abyssinian_1 1 1 1  BOOPUBUJPOTUSBJOWBMUYU Abyssinian_1.xml
  31.  ίʔυΛมߋ 5'3FDPSEʹSPECIES ͷ஋Λ௥Ճ dataset/convert_to_tfrecord.py

  32.  ςετσʔλΛ෼཭͢Δ σʔληοτ ςετσʔλ

  33.  εςοϓ̏ ϞσϧΛ࡞੒͢Δ

  34.  ৞ࠐΈχϡʔϥϧωοτϫʔΫʢ̘̣̣ʣ DPOW QPPM GD PVUQVU JOQVU

  35.  ৞ࠐΈʢ$POWPMVUJPOʣ૚

  36. # conv1
 with tf.variable_scope('conv1') as scope:
 weights = _get_weights([5, 5,

    3, 32], stddev=0.01)
 conv1 = tf.nn.conv2d(image_node, weights, [1, 1, 1, 1], padding='SAME')
 biases = tf.get_variable('biases', [32],
 initializer=tf.constant_initializer(0.0))
 bias = tf.nn.bias_add(conv1, biases)
 conv1 = tf.nn.relu(bias, name=scope.name)
  ৞ࠐΈʢ$POWPMVUJPOʣ૚
  37.  ৞ࠐΈʢ$POWPMVUJPOʣ૚ ৞ΈࠐΉϑΟϧλͷܗʢshapeʣ ຕ਺ εϥΠυ෯ ύσΟϯάͷ༗ແ ׆ੑԽؔ਺

  38.  ίʔυΛมߋ ৞ࠐΈ૚ͷઃఆΛܾΊΔ model.py

  39.  ϓʔϦϯάʢ1PPMJOHʣ૚ εϥΠυ෯2

  40. pool1 = tf.nn.max_pool(conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2,

    1],
 padding='SAME', name='pool1')
  ϓʔϦϯάʢ1PPMJOHʣ૚
  41.  ϓʔϦϯάʢ1PPMJOHʣ૚ ϓʔϦϯάͷछྨ Χʔωϧʢ΢Οϯυ΢ʣͷେ͖͞ εϥΠυ෯ ύσΟϯάͷ༗ແ

  42.  ίʔυΛมߋ ϓʔϦϯά૚ͷઃఆΛܾΊΔ model.py

  43.  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚ ※ ʮதؒ૚ʯʮӅΕ૚ʯͱ͍͏৔߹΋͋Δ : : : : :

  44. reshape = tf.reshape(pool1, [batch_size, -1])
 dim = reshape.get_shape()[1].value
 
 #

    fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384], stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚
  45.  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚ : : : : : ϊʔυͷ਺ ׆ੑԽؔ਺

  46.  ίʔυΛมߋ શ݁߹૚ͷϊʔυ਺ΛܾΊΔ model.py

  47.  ग़ྗ૚ : : : : :

  48. with tf.variable_scope('output') as scope:
 weights = _get_weights(shape=[384, 2], stddev=1 /

    384.0)
 biases = _get_biases([2], value=0.0)
 logits = tf.add(tf.matmul(fc2, weights), biases, name='logits')
  ग़ྗ૚
  49.  ग़ྗ૚ : : : : : ෼ྨ͢ΔΫϥεͷ਺

  50.  ίʔυΛมߋ ग़ྗ૚ͷϊʔυ਺ΛܾΊΔ model.py

  51.  εςοϓ̐ σʔλΛධՁʢJOGFSFODFʣ͢Δ

  52. ظ଴͢Δ஋ͱJOGFSFODFͷ݁ՌͷࠩʢޡࠩʣΛٻΊΔɻ  ޡࠩؔ਺

  53. def __loss(logits, label):
 labels = tf.cast(label, tf.int64)
 cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(


    logits, labels, name='cross_entropy_per_example')
 cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')
 return cross_entropy_mean
  ޡࠩؔ਺
  54.  ίʔυΛมߋ εςοϓ͝ͱʹϩε཰ʢޡࠩʣΛදࣔ͢Δ train.py

  55.  εςοϓ̑ ϞσϧΛ܇࿅͢Δ

  56. ޡࠩʹج͍ͮͯύϥϝʔλʔΛௐ੔͢Δɻ w ޯ഑߱Լ๏ w "EBN0QUJNJ[FS w "EB(SBEͳͲ  ֶशʢ܇࿅ʣΞϧΰϦζϜ

  57. global_step = tf.Variable(0, trainable=False)
 loss = __loss(logits, specie_batch)
 train_op =

    tf.train.GradientDescentOptimizer(FLAGS.learning_rate).minimize(
 loss, global_step)
  ֶशʢ܇࿅ʣΞϧΰϦζϜ
  58.  ίʔυΛมߋ train_opΛ࣮ߦ͢Δ train.py

  59. # fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384],

    stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  ύϥϝʔλʔ
  60.  ॏΈʢXFJHIUʣͱόΠΞεʢCJBTʣ : : : : : w b

  61.  ׆ੑԽؔ਺ : : : : :

  62. # fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384],

    stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  ׆ੑԽؔ਺
  63.  3F-6ʢ3FDUJpFE-JOFBSʣ ʢग़య: http://cs231n.github.io/neural-networks-1/ʣ

  64.  4JHNPJE ʢग़య: https://commons.wikimedia.org/wiki/File:Sigmoid-function-2.svgʣ

  65.  )ZQFSCPMJD5BOHFOU ʢग़య: https://commons.wikimedia.org/wiki/File:Hyperbolic_Tangent.svgʣ

  66.  εςοϓ̒ ύϥϝʔλʔΛอଘ͢Δ

  67. saver = tf.train.Saver(tf.all_variables())
 
 with tf.Session() as sess:
 sess.run(tf.initialize_all_variables())
 step

    = 0
 while step < FLAGS.max_steps:
 _, step, loss_value = sess.run([train_op, global_step, loss])
 
 if step % 100 == 0:
 print('Step: %d, loss: %.4f' % (step, loss_value))
 saver.save(sess, os.path.join(FLAGS.checkpoint_dir, 'cad_checkpoint'), global_step=step)  4BWFSʹΑΔύϥϝʔλʔͷอଘ
  68.  ίʔυΛมߋ ύϥϝʔλʔΛอଘ͢Δ train.py

  69.  εςοϓ̓ ܇࿅ࡁΈϞσϧΛݕূ͢Δ

  70. def __restore(sess, saver):
 ckpt = tf.train.get_checkpoint_state(FLAGS.checkpoint_dir)
 if ckpt and ckpt.model_checkpoint_path:


    saver.restore(sess, ckpt.model_checkpoint_path)  4BWFSʹΑΔύϥϝʔλʔͷಡΈࠐΈ
  71. saver = tf.train.Saver(tf.all_variables())
 
 with tf.Session() as sess:
 __restore(sess, saver)

     4BWFSʹΑΔύϥϝʔλʔͷಡΈࠐΈ
  72. logits = model.inference(image_batch, batch_size=FLAGS.batch_size)
 top_k_op = tf.nn.in_top_k(logits, specie_batch, 1)
 for

    step in range(num_iter): predictions = sess.run(top_k_op)
 true_count += np.sum(predictions)
  ݕূ
  73.  ίʔυΛมߋ ൑ఆʹࣦഊͨ͠ϑΝΠϧ໊Λ
 දࣔ͢Δ eval.py

  74.  ܇࿅ʢֶशʣͱ͸ σʔλ inference logits ਖ਼ղϥϕϧ ޡࠩؔ਺ ޡࠩʢϩεʣ ֶशΞϧΰϦζϜ ύϥϝʔλʔ

    (weights / biases)
  75.  ܇࿅ʢֶशʣͱ͸ inference 0 1

  76.  ܇࿅ʢֶशʣͱ͸ inference 0 1

  77.  $MPVE.-Ͱ܇࿅͢Δ

  78.  (PPHMF$MPVE1MBUGPSNͷ ར༻Λ։࢝

  79.  ϓϩδΣΫτͷ࡞੒

  80.  Ϋʔϙϯͷొ࿥ https://cloud.google.com/redeem

  81. C-LIS CO., LTD. ˞ϓϩδΣΫτ࡞੒࣌ʹબ୒ͨ͠
 ੥ٻઌΞΧ΢ϯτͱ
 Ұக͍ͯ͠Δ͔֬ೝ

  82.  $MPVE.-ͷ༗ޮԽ

  83.  $MPVE.-ͷ༗ޮԽ

  84.  $MPVE4%,ͷΠϯετʔϧ https://cloud.google.com/sdk/

  85.  $MPVE4%,ͷΠϯετʔϧ

  86. $ gcloud auth $ gcloud beta ml init-project Cloud ML

    needs to add its service accounts to your project (catsanddogs-149703) as Editors. This will enable Cloud Machine Learning to access resources in your project when running your training and prediction jobs. Do you want to continue (Y/n)?  $MPVE.-ͷϓϩδΣΫτΛॳظԽ
  87. $ PROJECT_ID=$(gcloud config list project --format "value(core.project)") $ BUCKET_NAME="cats_and_dogs" $

    gsutil mb -l us-central1 gs://$BUCKET_NAME  $MPVE.-༻ͷόέοτΛ࡞੒ ※ usͷΑ͏ͳMulti-regionalͰ͸ͳ͘RegionalͰࢦఆ͢Δඞཁ͕͋Δ
 https://cloud.google.com/ml/docs/how-tos/getting-set-up
  88.  ࡞੒ͨ͠όέοτʹ܇࿅σʔλΛసૹ

  89. TSDσΟϨΫτϦʹ __init__.py Λ࡞੒͢Δɻ  εΫϦϓτΛύοέʔδԽ

  90. ϑΝΠϧγεςϜ΁ΞΫηε͢ΔॲཧΛɺ͢΂ͯ5FOTPS'MPXͷ
 tensorflow.python.lib.io.file_ioΛ࢖͏Α͏ʹ͢Δɻ  5FOTPS'MPXͷίʔυΛมߋ

  91. $ gcloud beta ml jobs submit training cat_and_dogs \ --package-path=/Users/keiji_ariyama/Developments/catsanddogs/src

    \ --module-name=src.train \ --staging-bucket=gs://cats_and_dogs \ --region=us-central1 \ -- \ --train_dir gs://cats_and_dogs/cad_train \ --checkpoint_dir gs://cats_and_dogs/cad_checkpoint \ --max_steps 100  $MPVE.-༻ʹδϣϒΛొ࿥͢Δ
  92.  ΄Βɺ؆୯ʂ

  93. ݱࡏSpeciesͰ෼ྨ͍ͯ͠Δ΋ͷΛ
 BreedͰ෼ྨ͢ΔΑ͏ʹมߋ͢Δɻ  νϟϨϯδ

  94. C-LIS CO., LTD. ֤੡඼໊ɾϒϥϯυ໊ɺձ໊ࣾͳͲ͸ɺҰൠʹ֤ࣾͷ঎ඪ·ͨ͸ొ࿥঎ඪͰ͢ɻຊࢿྉதͰ͸ɺ˜ɺšɺäΛׂѪ͍ͯ͠·͢ɻ ຊࢿྉ͸ɺ༗ݶձࣾγʔϦεͷஶ࡞෺Ͱ͋ΓɺΫϦΤΠςΟϒίϞϯζͷදࣔඇӦརܧঝ6OQPSUFEϥΠηϯεͷݩͰެ։͍ͯ͠·͢ɻ

  95.  ݄೔ʙ https://devfestkansai.connpass.com/event/42864/