Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ゼロから作るDeep Learning読書会 7章 畳み込みニューラルネットワーク

ゼロから作るDeep Learning読書会 7章 畳み込みニューラルネットワーク

「7章 畳み込みニューラルネットワーク」の内容をまとめました

Takuya Kitamura

October 02, 2017
Tweet

More Decks by Takuya Kitamura

Other Decks in Technology

Transcript

  1. ର৅ॻ੶ θϩ͔Β࡞ΔDEEP LEARNING ʙPYTHONͰֶͿσΟʔϓϥʔχϯάͷཧ࿦ͱ࣮૷ʙ ▸ ࡈ౻ ߁ؽ ஶ ▸ O’REILLY

    ΦϥΠϦʔɾδϟύϯ ▸ https://www.oreilly.co.jp/books/9784873117584/ 1ষɹPythonೖ໳ 2ষɹύʔηϓτϩϯ 3ষɹχϡʔϥϧωοτϫʔΫ 4ষɹχϡʔϥϧωοτϫʔΫͷֶश 5ষɹޡࠩٯ఻೻๏ 6ষɹֶशʹؔ͢ΔςΫχοΫ 7ষɹ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ 8ষɹσΟʔϓϥʔχϯά ⾢ࠓճ͸ίίʂ
  2. 7.1 શମߏ଄ ৞ΈࠐΈχϡʔϥϧωοτϫʔΫʢCNNʣͱ͸ɺը૾ೝࣝ΍Ի੠ೝࣝ ͳͲͰ෯޿͘࢖ΘΕ͍ͯΔσΟʔϓϥʔχϯάͷํࣜ ▸ CNNͰ͸ɺAffineϨΠϠ΍׆ੑԽؔ਺ϨΠϠʹՃ͑ɺ৽ͨʹConvolutionϨΠϠ ͱPoolingϨΠϠ͕ొ৔͢Δ ▸ ConvolutionϨΠϠɿը૾ͷೱ୶ύλʔϯʢಛ௃ʣΛݕग़͢Δ૚ ▸

    PoolingϨΠϠɿ෺ମͷҐஔ͕มಈͯ͠΋ಉҰͷ෺ମͱͯ͠ೝࣝ͢Δ૚ʢҐஔͷζϨΛڐ༰͢Δʣ ▸ ैདྷͷશͯͷ݁߹૚ΛAffineϨΠϠͰ࣮૷ͨ͠৔߹(શ݁߹) ▸ CNNͷ৔߹ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  3. 7.2 ৞ΈࠐΈ૚ɹʙશ݁߹૚ͷ໰୊఺ʙ σʔλͷܗঢ়͕ແࢹ͞Εͯ͠·͏ ▸ ը૾Λྫʹ͢Δͱɺ ▸ ը૾͸ॎɺԣɺνϟϯωϧʢ৭Λ੒෼ผʹ෼཭ͨ͠৘ใʣͷ̏࣍ݩߏ଄ ▸ ͜Ε·Ͱͷଟ૚ύʔηϓτϩϯʹΑΔશ݁߹૚Λ࢖ͬͯը૾ೝࣝॲཧΛ͢Δͱ ▸

    ͜ͷॲཧํ๏ͩͱɺ࣍ݩؒͷؔ܎΍࣍ݩ಺ͷσʔλͷؔ܎Λແࢹͯ͠ಉ౳ͷχϡʔϩ ϯͱͯ͠ॲཧ͢ΔͨΊɺͦͷؔ܎Λ׆͔ͤͳ͍ ▸ RBGͷ֤νϟωϧؒͷؔ܎ɺڑ཭ͷ͍ۙϐΫηϧɺԕ͍ϐΫηϧͷؔ܎ɺetc… ▸ CNNͰ͸ɺը૾ͷܗঢ়Λ׆͔ͨ͠··ޮ཰తʹσʔλΛཧղ͢ΔͨΊʹɺ৞ΈࠐΈԋࢉ Λ༻͍Δ Ҿ༻ɿʮߴଔͰ΋Θ͔Δػցֶश (7) ৞ΈࠐΈχϡʔϥϧωοτ ͦͷ1 | ಄ͷதʹࢥ͍ු͔΂ͨ࣌ʹ͸ʯhttp://hokuts.com/2016/12/13/cnn1/
  4. 7.2 ৞ΈࠐΈ૚ɹʙ৞ΈࠐΈԋࢉͱ͸ʙ ৞ΈࠐΈԋࢉɿը૾ͷ֤ըૉʹରͯ͠ϑΟϧλʔΛద༻͢Δॲཧ ▸ ը૾ॲཧʹ࢖ΘΕɺ΅͔͠ɺγϟʔϓԽɺΤοδநग़ͳͲΛߦ͏ͨΊͷԋࢉ ▸ ΅͔͠༻ϑΟϧλʔɺγϟʔϓԽ༻ϑΟϧλʔɺΤοδநग़༻ϑΟϧλʔɺetc… ▸ CNNͰ͸͜ͷϑΟϧλʔͷ஋Λಛఆ໨తͷ΋ͷʢ΅͔͠ͳͲʣʹݻఆͤͣɺ͜ ΕΛॏΈͱֶͯ͠श͍ͤͯ͘͞ʢղੳ༻ͷϑΟϧλʔΛ࡞Δʣ

    ▸ ৞ΈࠐΈԋࢉ͸ɺ֤Ґஔͷըૉʢೖྗσʔλʣʹରͯ͠ϑΟϧλʔͷ΢Οϯυ΢ΛҰ ఆִؒͰεϥΠυͤ͞ͳ͕Βద༻͢Δ ɾɾɾ ೖྗσʔλ ϑΟϧλʔ ग़ྗσʔλ ೖྗσʔλ ϑΟϧλʔ ग़ྗσʔλ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  5. 7.3 ϓʔϦϯά૚ ॎɾԣํ޲ͷۭؒΛখ͘͢͞ΔԋࢉΛߦ͏ ▸ ϓʔϦϯά૚ͷಛ௃ ▸ ֶशύϝʔλ͕ͳ͍ ▸ νϟϯωϧ਺͕มԽ͠ͳ͍ ▸

    νϟϯωϧຖʹಠཱͯ͠ܭࢉ͞ΕΔ ▸ ඍখͳมԽʹରͯ͠ϩόετʢؤ݈ʣ ▸ ೖྗσʔλͷζϨΛϓʔϦϯάͰٵऩ͢Δ ೖྗσʔλ͕ԣํ޲ʹ̍ཁૉ෼͚ͩζϨͨ৔߹Ͱ΋ɺग़ྗ͸ಉ͡Α͏ͳ݁ՌʹͳΔ MaxϓʔϦϯάͷॲཧखॱ
 ɹɹ࠷େ஋ΛऔΔԋࢉ ϓʔϦϯάॲཧํࣜʹ͸AverageϓʔϦϯ άͳͲ΋͋Δ͕ɺը૾ॲཧʹ͸ओʹMax ϓʔϦϯά͕࢖ΘΕΔ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ Ҿ༻ɿ
 ॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ
 7ষ
  6. 7.4 Convolution/PoolingϨΠϠͷ࣮૷ ▸ 4࣍ݩ഑ྻ ▸ CNNͰ͸֤૚ΛྲྀΕΔσʔλ͸̐࣍ݩͷσʔλʹͳΔ ▸ ྫɿߴ͞28ɾԣ෯28Ͱ1νϟϯωϧͷσʔλ10ݸ͋Δ৔߹ ▸ (10,

    1, 28, 28)ͱදݱ͞ΕΔ ▸ im2colʹΑΔల։ ▸ im2col͸̏࣍ݩͷೖྗσʔλʢը૾σʔλʣΛ2࣍ݩͷߦྻʹม׵͢ΔϥΠϒϥϦ ▸ ೖྗσʔλʹରͯ͠ϑΟϧλʔΛద༻͢Δ৔ॴͷྖҬʢ̏࣍ݩͷϒϩοΫʣΛԣํ޲ͷ̍ྻʹల ։͢ΔͨΊʹར༻ ▸ ը૾σʔλΛߦྻʹม׵͢ΔࣄʹΑΓϑΟϧλʔԋࢉΛߦྻॲཧͱ࣮ͯ͠૷Մೳ ϑΟϧλʔద༻ྖҬΛઌ಄͔Βॱ൪ʹ̍ྻʹల։͢Δ Ҿ༻ɿ
 ॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ
 7ষ
  7. 7.4 Convolution/PoolingϨΠϠͷ࣮૷ɹʙConvolutionϨΠϠͷ࣮૷ʙ ▸ im2colΛར༻ͨ͠৞ΈࠐΈϨΠϠͷॲཧ 1. im2colʹΑΓը૾σʔλΛߦྻʹ ల։ 2. ϑΟϧλʔΛ2࣍ݩ഑ྻʹల։ 3.

    2ͭͷߦྻͷੵΛܭࢉ 4. ग़ྗ͕2࣍ݩσʔλʹͳ͍ͬͯΔ ͨΊɺద੾ͳܗঢ়ʹ੔ܗ class Convolution: def __init__(self, W, b, stride=1, pad=0): self.W = W # ϑΟϧλʔ self.b = b self.stride = stride self.pad = pad def forward(self, x): FN, C, FH, FW = self.W.sape # FN͸ϑΟϧλʔͷݸ਺ N, C, H, W = x.shape # ग़ྗσʔλͷߴ͞ͱ෯Λܭࢉ out_h = int(1 + (H + 2*self.pad - FH) / self.stride) out_w = int(1 + (W + 2*self.pad - FW) / self.stride) # 1. ೖྗσʔλΛ2࣍ݩͷߦྻʹม׵ col = im2col(x, FH, FW, self.stride, self.pad) # 2. ϑΟϧλʔ΋2࣍ݩ഑ྻʹల։ col_W = self.W.reshape(FN, -1).T # 3. ೖྗσʔλͷߦྻͱϑΟϧλʔͷߦྻͷੵΛܭࢉ out = np.dot(col, col_W) + self.b # 4. ग़ྗ͕2࣍ݩσʔλʹͳ͍ͬͯΔͨΊɺద੾ͳܗঢ়ʹ੔ܗ out = out.reshape(N, out_h, out_w, -1).transpose(0, 3, 1, 2) return out def backward(self, dout): # im2col ʹΑͬͯల։͢Δ͜ͱͰɺAffineϨΠϠͱ΄΅ಉ͡ཁྖͰ࣮૷Ͱ͖Δ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  8. 7.4 Convolution/PoolingϨΠϠͷ࣮૷ɹʙPoolingϨΠϠͷ࣮૷ʙ ▸ PoolingϨΠϠͰ΋im2colΛ࢖ͬͯ ೖྗσʔλΛల։͢Δ ▸ ೖྗσʔλΛల։͢Δ ▸ ߦ͝ͱʹ࠷େ஋ΛٻΊΔ ▸

    ద੾ͳग़ྗαΠζʹ੔ܗ͢Δ class Pooling: def __init__(self, pool_h, pool_w, stride=1, pad=0): self.pool_h = pool_h self.pool_w = pool_w self.stride = stride self.pad = pad def forward(self, x): N, C, H, W = x.shape out_h = int(1 + (H - self.pool_h) / self.stride) out_w = int(1 + (W - self.pool_w) / self.stride) # 1. ೖྗσʔλΛల։͢Δ col = im2col(x, self.pool_h, self.pool_w, self.stride, self.pad) col = col.reshape(-1, self.pool_h*self.pool_w) # 2. ߦ͝ͱʹ࠷େ஋ΛٻΊΔ out = np.max(col, axis=1) # 3. ద੾ͳग़ྗαΠζʹ੔ܗ͢Δ out = out.reshape(N, out_h, out_w, C).transpose(0, 3, 1, 2) return out def backward(self, dout): # im2col ʹΑͬͯల։͢Δ͜ͱͰɺAffineϨΠϠͱ΄΅ಉ͡ཁྖͰ࣮૷Ͱ͖Δ ೖྗσʔλͷల։ ϓʔϦϯάॲཧͷྲྀΕ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞Δ
 ɹɹɹDeep Learningʯ7ষ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞Δ
 ɹɹɹDeep Learningʯ7ষ
  9. 7.5 CNNͷ࣮૷ # ୯७ͳCNN class SimpleConvNet: def __init__(self, input_dim=(1, 28,

    28), conv_param={'filter_num':30, 'filter_size':5, 'pad':0, 'stride':1}, hidden_size=100, output_size=10, weight_init_std=0.01): # ม਺ͷ४උ filter_num = conv_param['filter_num'] filter_size = conv_param['filter_size'] filter_pad = conv_param['pad'] filter_stride = conv_param['stride'] input_size = input_dim[1] conv_output_size = (input_size - filter_size + 2*filter_pad) \ / filter_stride pool_output_size = int(filter_num * (conv_output_size/2) * (conv_output_size/2)) # ॏΈͷॳظԽ self.params = {} # 1૚໨ͷ৞ΈࠐΈ૚ͷॏΈʢϑΟϧλʔʣͱόΠΞεͷॳظԽ self.params['W1'] = weight_init_std \ * np.random.randn(filter_num, input_dim[0], filter_size, filter_size) self.params['b1'] = np.zeros(filter_num) # 2૚໨ɺ3૚໨ͷશ݁߹૚ͷॏΈͱόΠΞεͷॳظԽ self.params['W2'] = weight_init_std \ * np.random.randn(pool_output_size, hidden_size) self.params['b2'] = np.zeros(hidden_size) self.params['W3'] = weight_init_std \ * np.random.randn(hidden_size, output_size) self.params['b3'] = np.zeros(output_size) # ֤ϨΠϠͷΠϯελϯεԽ self.layers = OrderedDict() self.layers['Conv1'] = Convolution(self.params['W1'], self.params['b1'], conv_param['stride'], conv_param['pad']) self.layers['Relu1'] = Relu() self.layers['Pool1'] = Pooling(pool_h=2, pool_w=2, stride=2) self.layers['Affine1'] = Affine(self.params['W2'], self.params['b2']) self.layers['Relu2'] = Relu() self.layers['Affine2'] = Affine(self.params['W3'], self.params['b3']) self.last_layer = SoftmaxWithLoss() (ଓ͘) (ଓ͖) # ਪ࿦ def predict(self, x): for layer in self.layers.values(): x = layer.forward(x) return x # ଛࣦؔ਺ॲཧ def loss(self, x, t): y = self.predict(x) return self.last_layer.forward(y, t) # ޡࠩٯ఻ൖॲཧ def gradient(self, x, t): # ॱ఻ൖॲཧͱଛࣦؔ਺ͷܭࢉ self.loss(x, t) # ٯ఻ൖॲཧ dout = 1 dout = self.last_layer.backward(dout) layers = list(self.layers.values()) layers.reverse() for layer in layers: dout = layer.backward(dout) # ॲཧ݁ՌΛ֨ೲ grads = {} grads['W1'] = self.layers['Conv1'].dW grads['b1'] = self.layers['Conv1'].db grads['W2'] = self.layers['Affine1'].dW grads['b2'] = self.layers['Affine1'].db grads['W3'] = self.layers['Affine2'].dW grads['b3'] = self.layers['Affine2'].db return grads Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  10. 7.7 ୅දతͳCNNɹʙLeNetʙ LeNet - खॻ͖਺ࣈೝࣝΛߦ͏ωοτϫʔΫ ▸ 1998೥ʹॳΊͯఏҊ͞ΕͨCNNͷݩ૆ ▸ ৞ΈࠐΈ૚ͱαϒαϯϓϦϯά૚ΛॏͶͯɺ࠷ޙʹશ݁߹૚Λܦ༝ͯ݁͠ՌΛ ग़ྗ͢Δ

    ▸ ϓʔϦϯά૚͕୯ʹʮཁૉΛؒҾ͘ʯ͚ͩͷαϒαϯϓϦϯά૚ʹͳ͍ͬͯΔ ▸ ׆ੑԽؔ਺͕γάϞΠυؔ਺Λར༻ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  11. 7.7 ୅දతͳCNNɹʙAlexNetʙ AlexNet : σΟʔϓϥʔχϯάͷՐ෇͚໾ͱͳͬͨωοτϫʔΫ ▸ ৞ΈࠐΈ૚ͱϓʔϦϯά૚ΛॏͶͯɺ࠷ޙʹશ݁߹૚Λܦ༝ͯ݁͠ՌΛग़ྗ͢Δ ▸ ׆ੑԽؔ਺ʹReLUΛ༻͍Δ ▸

    LRNʢLocal Response Normalizationʣͱ͍͏ہॴతਖ਼نԽΛߦ͏૚Λ༻͍Δ ▸ DropoutΛ࢖༻͢Δ Ҿ༻ɿॻ੶ʮθϩ͔Β࡞ΔDeep Learningʯ7ষ
  12. ࢀߟจݙ 1. θϩ͔Β࡞ΔDeep LearningʙPythonͰֶͿσΟʔϓϥʔχϯάͷཧ࿦ͱ࣮૷ʙ
 ࡈ౻ ߁ؽ ஶʗO’REILLY ΦϥΠϦʔɾδϟύϯ
 https://www.oreilly.co.jp/books/9784873117584/ 2.

    ը૾σʔλͷجຊతͳߏ଄ ʔ NEAREAL
 http://neareal.com/470/ 3. νϟϯωϧͱ͸ | GIMPೖ໳(2.8൛)
 http://www.gimp.jp.net/knowledge_channels_overview.html 4. ߴଔͰ΋Θ͔Δػցֶश (7) ৞ΈࠐΈχϡʔϥϧωοτ ͦͷ1 | ಄ͷதʹࢥ͍ු͔΂ͨ࣌ʹ͸
 http://hokuts.com/2016/12/13/cnn1/