20161120-MLHands-on

2016年11月20日にGCPUG Tokushimaで開催した機械学習ハンズオンの資料です。

犬猫の画像を分類する畳み込みニューラルネットワーク（CNN）を構築します。
データセットにはCNNThe Oxford-IIIT Pet Datasetを使います。

Transcript

1. C-LIS CO., LTD. ػցֶशϋϯζΦϯ ($16(
   5PLVTIJNB
   
   
2. C-LIS CO., LTD.  ༗ࢁ
   ܓೋ
   ʢ,FJKJ
   "3*:"."ʣ $-*4
   $0
   -5%
   "OESPJEΞϓϦ։ൃνϣοτσΩϧ
   ػցֶशॳ৺ऀ Photo : Koji MORIGUCHI (AUN CREATIVE FIRM)
3. C-LIS CO., LTD. TM 
4.  10% 1SJOU
   0O
   %FNBOE
   
   ిࢠॻ੶ Impress R&D͔Βൃചத http://amzn.to/2axRog
5.  *OUSPEVDUJPO
   ʰػցֶशʱͱ͸ͳʹ͔
6. ػցֶशʢ͖͔͍͕͘͠Ύ͏ɺӳ
   NBDIJOF
   MFBSOJOHʣͱ͸ɺਓ޻஌ೳʹ͓͚ Δݚڀ՝୊ͷҰͭͰɺਓ͕ؒࣗવʹߦ͍ͬͯΔֶशೳྗͱಉ༷ͷػೳΛίϯ ϐϡʔλͰ࣮ݱ͠Α͏ͱ͢Δٕज़ɾख๏ͷ͜ͱͰ͋Δɻ  ػցֶश ʢWikipedia: https://ja.wikipedia.org/wiki/%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92ʣ
7. σΟʔϓϥʔχϯάɺਂ૚ֶशʢӳ
   EFFQ
   MFBSOJOHʣͱ͸ɺଟ૚ߏ଄ͷχϡʔ ϥϧωοτϫʔΫʢσΟʔϓχϡʔϥϧωοτϫʔΫɺӳ
   EFFQ
   OFVSBM
   OFUXPSLʣΛ༻͍ͨػցֶशͰ͋Δɻ  σΟʔϓϥʔχϯά ʢWikipedia: https://ja.wikipedia.org/wiki/%E3%83%87%E3%82%A3%E3%83%BC%E3%83%97%E3%83%A9%E3%83%BC%E3%83%8B%E3%83%B3%E3%82%B0ʣ
8. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳ
   OFVSBM
   OFUXPSL
   // ͸ɺ೴ػೳʹ ݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱ Λ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ  χϡʔϥϧωοτϫʔΫ ʢWikipedia: https://ja.wikipedia.org/wiki/%E3%83%8B%E3%83%A5%E3%83%BC%E3%83%A9%E3%83%AB%E3%83%8D%E3%83%83%E3%83%88%E3%83%AF%E3%83%BC%E3%82%AFʣ
9.  ཁ͢Δʹ਺ֶ
10. C-LIS CO., LTD. TM 5FOTPS'MPX
    JT
    BO
    0QFO
    4PVSDF
    4PGUXBSF
    -JCSBSZ
    GPS
    .BDIJOF
    *OUFMMJHFODF 
11.  ϋϯζΦϯ
12. w 1ZUIPO
    ʢ(PPHMF
    $PNQVUFS
    1MBUGPSN
    5PPMT͕ରԠͷͨΊʣ
    w ґଘύοέʔδ
    w TJY
    
    w 1*-
    
    1JMMPX
    w MYNM
    w QZUIPOHqBHT
    w 5FOTPS'MPX
    ɹɹ˞ཁ֬ೝ  ։ൃ؀ڥͷ֬ೝ
13.  ͋ΔͱΑ͍΋ͷ https://www.jetbrains.com/pycharm/
14.  ը૾෼ྨ ʢग़య
    IUUQTHJUIVCDPNDB[BMBNOJTUʣ ʢग़య
    IUUQTXXXDTUPSPOUPFEV_LSJ[DJGBSIUNMʣ MNIST CIFAR-10
15.  ݘೣ෼ྨ
16.  σʔλͱਖ਼ղϥϕϧΛ༻ҙ͢Δ
     σʔλΛ5'3FDPSEܗࣜʹม׵͢Δ
     ϞσϧΛ࡞੒͢Δ
     σʔλΛධՁʢJOGFSFODFʣ͢Δ
     ϞσϧΛ܇࿅͢Δ
     ύϥϝʔλʔΛอଘ͢Δ
     ܇࿅ࡁΈϞσϧΛݕূ͢Δ  େ·͔ͳखॱ
17.  εςοϓ̍
    σʔλͱਖ਼ղϥϕϧΛ༻ҙ͢Δ
18.  σʔληοτͷμ΢ϯϩʔυ http://www.robots.ox.ac.uk/~vgg/data/pets/
19.  JNBHFT
20.  BOOPUBUJPOT ೣ: Abyssinian ೣ: Ragdoll ೣ: Russian_Blue ݘ: Shiba ݘ: saint_bernard ݘ: samoyed
21.  30*ʢ3FHJPO
    0G
    *OUFSFTUʣ
22.  ίʔυͷμ΢ϯϩʔυ IUUQTHPPHM,TW,I2
23.  εςοϓ̎
    σʔλΛ5'3FDPSEܗࣜʹม׵͢Δ
24.  લॲཧ Abyssinian_1.jpg
25. <annotation> <folder>OXIIIT</folder> <filename>Abyssinian_1.jpg</filename> <source> <database>OXFORD-IIIT Pet Dataset</database> <annotation>OXIIIT</annotation> <image>flickr</image> </source> <size> <width>600</width> <height>400</height> <depth>3</depth> </size> <segmented>0</segmented>  BOOPUBUJPOTYNMT <object> <name>cat</name> <pose>Frontal</pose> <truncated>0</truncated> <occluded>0</occluded> <bndbox> <xmin>333</xmin> <ymin>72</ymin> <xmax>425</xmax> <ymax>158</ymax> </bndbox> <difficult>0</difficult> </object> </annotation> Abyssinian_1.xml
26.  30*ͷ੾Γൈ͖
27.  ίʔυΛมߋ
    bndboxͷ֤஋͔Βը૾Λ੾Γൈ͍ͯอଘ͢Δ dataset/crop_roi_image.py
28. FLAGS = gflags.FLAGS
 gflags.DEFINE_string('image_dir', None, 'ॲཧ͢Δը૾ͷσΟϨΫτϦ')
 gflags.DEFINE_string('xml_dir', None, 'ॲཧ͢ΔXMLͷσΟϨΫτϦ')
 gflags.DEFINE_string('output_dir', None, 'ग़ྗ͢ΔσΟϨΫτϦ')  HqBHT $ python crop_roi_image.py \ --image_dir /Users/keiji_ariyama/Downloads/catsanddogs/images \ --xml_dir /Users/keiji_ariyama/Downloads/catsanddogs/annotations/xmls \ --output_dir /Users/keiji_ariyama/Desktop/test
29.  ը૾ʹϥϕϧΛ෇Ճͯ͠5'3FDPSE΁ม׵ + Cat (1) Abyssinian_1.jpg TFRecord
30. #Image CLASS-ID SPECIES BREED ID #ID: 1:37 Class ids #SPECIES: 1:Cat 2:Dog #BREED ID: 1-25:Cat 1:12:Dog #All images with 1st letter as captial are cat images #images with small first letter are dog images Abyssinian_196 1 1 1 Abyssinian_197 1 1 1 Abyssinian_19 1 1 1 Abyssinian_1 1 1 1  BOOPUBUJPOTUSBJOWBMUYU Abyssinian_1.xml
31.  ίʔυΛมߋ
    5'3FDPSEʹ
    SPECIES ͷ஋Λ௥Ճ dataset/convert_to_tfrecord.py
32.  ςετσʔλΛ෼཭͢Δ σʔληοτ ςετσʔλ
33.  εςοϓ̏
    ϞσϧΛ࡞੒͢Δ
34.  ৞ࠐΈχϡʔϥϧωοτϫʔΫʢ̘̣̣ʣ DPOW QPPM GD PVUQVU JOQVU
35.  ৞ࠐΈʢ$POWPMVUJPOʣ૚
36. # conv1
 with tf.variable_scope('conv1') as scope:
 weights = _get_weights([5, 5, 3, 32], stddev=0.01)
 conv1 = tf.nn.conv2d(image_node, weights, [1, 1, 1, 1], padding='SAME')
 biases = tf.get_variable('biases', [32],
 initializer=tf.constant_initializer(0.0))
 bias = tf.nn.bias_add(conv1, biases)
 conv1 = tf.nn.relu(bias, name=scope.name)
  ৞ࠐΈʢ$POWPMVUJPOʣ૚
37.  ৞ࠐΈʢ$POWPMVUJPOʣ૚ ৞ΈࠐΉϑΟϧλͷܗʢshapeʣ ຕ਺ εϥΠυ෯ ύσΟϯάͷ༗ແ ׆ੑԽؔ਺
38.  ίʔυΛมߋ
    ৞ࠐΈ૚ͷઃఆΛܾΊΔ model.py
39.  ϓʔϦϯάʢ1PPMJOHʣ૚ εϥΠυ෯2
40. pool1 = tf.nn.max_pool(conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1],
 padding='SAME', name='pool1')
  ϓʔϦϯάʢ1PPMJOHʣ૚
41.  ϓʔϦϯάʢ1PPMJOHʣ૚ ϓʔϦϯάͷछྨ Χʔωϧʢ΢Οϯυ΢ʣͷେ͖͞ εϥΠυ෯ ύσΟϯάͷ༗ແ
42.  ίʔυΛมߋ
    ϓʔϦϯά૚ͷઃఆΛܾΊΔ model.py
43.  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚ ※ ʮதؒ૚ʯʮӅΕ૚ʯͱ͍͏৔߹΋͋Δ : : : : :
44. reshape = tf.reshape(pool1, [batch_size, -1])
 dim = reshape.get_shape()[1].value
 
 # fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384], stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚
45.  શ݁߹ʢ'VMMZDPOOFDUFEʣ૚ : : : : : ϊʔυͷ਺ ׆ੑԽؔ਺
46.  ίʔυΛมߋ
    શ݁߹૚ͷϊʔυ਺ΛܾΊΔ model.py
47.  ग़ྗ૚ : : : : :
48. with tf.variable_scope('output') as scope:
 weights = _get_weights(shape=[384, 2], stddev=1 / 384.0)
 biases = _get_biases([2], value=0.0)
 logits = tf.add(tf.matmul(fc2, weights), biases, name='logits')
  ग़ྗ૚
49.  ग़ྗ૚ : : : : : ෼ྨ͢ΔΫϥεͷ਺
50.  ίʔυΛมߋ
    ग़ྗ૚ͷϊʔυ਺ΛܾΊΔ model.py
51.  εςοϓ̐
    σʔλΛධՁʢJOGFSFODFʣ͢Δ
52. ظ଴͢Δ஋ͱJOGFSFODFͷ݁ՌͷࠩʢޡࠩʣΛٻΊΔɻ  ޡࠩؔ਺
53. def __loss(logits, label):
 labels = tf.cast(label, tf.int64)
 cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
 logits, labels, name='cross_entropy_per_example')
 cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')
 return cross_entropy_mean
  ޡࠩؔ਺
54.  ίʔυΛมߋ
    εςοϓ͝ͱʹϩε཰ʢޡࠩʣΛදࣔ͢Δ train.py
55.  εςοϓ̑
    ϞσϧΛ܇࿅͢Δ
56. ޡࠩʹج͍ͮͯύϥϝʔλʔΛௐ੔͢Δɻ
    w ޯ഑߱Լ๏
    w "EBN0QUJNJ[FS
    w "EB(SBE
    ͳͲ  ֶशʢ܇࿅ʣΞϧΰϦζϜ
57. global_step = tf.Variable(0, trainable=False)
 loss = __loss(logits, specie_batch)
 train_op = tf.train.GradientDescentOptimizer(FLAGS.learning_rate).minimize(
 loss, global_step)
  ֶशʢ܇࿅ʣΞϧΰϦζϜ
58.  ίʔυΛมߋ
    train_op
    Λ࣮ߦ͢Δ train.py
59. # fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384], stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  ύϥϝʔλʔ
60.  ॏΈʢXFJHIUʣͱόΠΞεʢCJBTʣ : : : : : w b
61.  ׆ੑԽؔ਺ : : : : :
62. # fc2
 with tf.variable_scope('fc2') as scope:
 weights = _get_weights([dim, 384], stddev=0.04)
 biases = _get_biases([384], value=0.1)
 fc2 = tf.nn.relu(tf.nn.bias_add(tf.matmul(reshape, weights), biases),
 name=scope.name)
  ׆ੑԽؔ਺
63.  3F-6ʢ3FDUJpFE
    -JOFBSʣ ʢग़య: http://cs231n.github.io/neural-networks-1/ʣ
64.  4JHNPJE ʢग़య: https://commons.wikimedia.org/wiki/File:Sigmoid-function-2.svgʣ
65.  )ZQFSCPMJD
    5BOHFOU ʢग़య: https://commons.wikimedia.org/wiki/File:Hyperbolic_Tangent.svgʣ
66.  εςοϓ̒
    ύϥϝʔλʔΛอଘ͢Δ
67. saver = tf.train.Saver(tf.all_variables())
 
 with tf.Session() as sess:
 sess.run(tf.initialize_all_variables())
 step = 0
 while step < FLAGS.max_steps:
 _, step, loss_value = sess.run([train_op, global_step, loss])
 
 if step % 100 == 0:
 print('Step: %d, loss: %.4f' % (step, loss_value))
 saver.save(sess, os.path.join(FLAGS.checkpoint_dir, 'cad_checkpoint'), global_step=step)  4BWFSʹΑΔύϥϝʔλʔͷอଘ
68.  ίʔυΛมߋ
    ύϥϝʔλʔΛอଘ͢Δ train.py
69.  εςοϓ̓
    ܇࿅ࡁΈϞσϧΛݕূ͢Δ
70. def __restore(sess, saver):
 ckpt = tf.train.get_checkpoint_state(FLAGS.checkpoint_dir)
 if ckpt and ckpt.model_checkpoint_path:
 saver.restore(sess, ckpt.model_checkpoint_path)  4BWFSʹΑΔύϥϝʔλʔͷಡΈࠐΈ
71. saver = tf.train.Saver(tf.all_variables())
 
 with tf.Session() as sess:
 __restore(sess, saver)  4BWFSʹΑΔύϥϝʔλʔͷಡΈࠐΈ
72. logits = model.inference(image_batch, batch_size=FLAGS.batch_size)
 top_k_op = tf.nn.in_top_k(logits, specie_batch, 1)
 for step in range(num_iter): predictions = sess.run(top_k_op)
 true_count += np.sum(predictions)
  ݕূ
73.  ίʔυΛมߋ
    ൑ఆʹࣦഊͨ͠ϑΝΠϧ໊Λ
 දࣔ͢Δ eval.py
74.  ܇࿅ʢֶशʣͱ͸ σʔλ inference logits ਖ਼ղϥϕϧ ޡࠩؔ਺ ޡࠩʢϩεʣ ֶशΞϧΰϦζϜ ύϥϝʔλʔ (weights / biases)
75.  ܇࿅ʢֶशʣͱ͸ inference 0 1
76.  ܇࿅ʢֶशʣͱ͸ inference 0 1
77.  $MPVE
    .-Ͱ܇࿅͢Δ
78.  (PPHMF
    $MPVE
    1MBUGPSNͷ
    ར༻Λ։࢝
79.  ϓϩδΣΫτͷ࡞੒
80.  Ϋʔϙϯͷొ࿥ https://cloud.google.com/redeem
81. C-LIS CO., LTD. ˞
    ϓϩδΣΫτ࡞੒࣌ʹબ୒ͨ͠
 ੥ٻઌΞΧ΢ϯτͱ
 Ұக͍ͯ͠Δ͔֬ೝ
82.  $MPVE
    .-ͷ༗ޮԽ
83.  $MPVE
    .-ͷ༗ޮԽ
84.  $MPVE
    4%,ͷΠϯετʔϧ https://cloud.google.com/sdk/
85.  $MPVE
    4%,ͷΠϯετʔϧ
86. $ gcloud auth $ gcloud beta ml init-project Cloud ML needs to add its service accounts to your project (catsanddogs-149703) as Editors. This will enable Cloud Machine Learning to access resources in your project when running your training and prediction jobs. Do you want to continue (Y/n)?  $MPVE
    .-ͷϓϩδΣΫτΛॳظԽ
87. $ PROJECT_ID=$(gcloud config list project --format "value(core.project)") $ BUCKET_NAME="cats_and_dogs" $ gsutil mb -l us-central1 gs://$BUCKET_NAME  $MPVE
    .-༻ͷόέοτΛ࡞੒ ※ usͷΑ͏ͳMulti-regionalͰ͸ͳ͘RegionalͰࢦఆ͢Δඞཁ͕͋Δ
 https://cloud.google.com/ml/docs/how-tos/getting-set-up
88.  ࡞੒ͨ͠όέοτʹ܇࿅σʔλΛసૹ
89. TSDσΟϨΫτϦʹ __init__.py Λ࡞੒͢Δɻ  εΫϦϓτΛύοέʔδԽ
90. ϑΝΠϧγεςϜ΁ΞΫηε͢ΔॲཧΛɺ͢΂ͯ5FOTPS'MPXͷ
 tensorflow.python.lib.io.file_ioΛ࢖͏Α͏ʹ͢Δɻ  5FOTPS'MPXͷίʔυΛมߋ
91. $ gcloud beta ml jobs submit training cat_and_dogs \ --package-path=/Users/keiji_ariyama/Developments/catsanddogs/src \ --module-name=src.train \ --staging-bucket=gs://cats_and_dogs \ --region=us-central1 \ -- \ --train_dir gs://cats_and_dogs/cad_train \ --checkpoint_dir gs://cats_and_dogs/cad_checkpoint \ --max_steps 100  $MPVE
    .-༻ʹδϣϒΛొ࿥͢Δ
92.  ΄Βɺ؆୯ʂ
93. ݱࡏ
    Species
    Ͱ෼ྨ͍ͯ͠Δ΋ͷΛ
 Breed
    Ͱ෼ྨ͢ΔΑ͏ʹมߋ͢Δɻ  νϟϨϯδ
94. C-LIS CO., LTD. ֤੡඼໊ɾϒϥϯυ໊ɺձ໊ࣾͳͲ͸ɺҰൠʹ֤ࣾͷ঎ඪ·ͨ͸ొ࿥঎ඪͰ͢ɻຊࢿྉதͰ͸ɺ˜ɺšɺäΛׂѪ͍ͯ͠·͢ɻ ຊࢿྉ͸ɺ༗ݶձࣾγʔϦεͷஶ࡞෺Ͱ͋ΓɺΫϦΤΠςΟϒίϞϯζͷදࣔඇӦརܧঝ
    
    6OQPSUFE
    ϥΠηϯεͷݩͰެ։͍ͯ͠·͢ɻ
95.  ݄೔
    ʙ https://devfestkansai.connpass.com/event/42864/