Upgrade to Pro — share decks privately, control downloads, hide ads and more …

自然言語処理と深層学習の最先端

Avatar for tkng tkng
January 15, 2016

 自然言語処理と深層学習の最先端

第4回 JustTechTalk の発表資料

Avatar for tkng

tkng

January 15, 2016
Tweet

More Decks by tkng

Other Decks in Technology

Transcript

  1. Show, Attend and Tell: Neural Image Caption Generation with Visual

    Attention (Xu+, 2015) • ը૾ʹର͢Δղઆจͷੜ੒ IUUQLFMWJOYVHJUIVCJPQSPKFDUTDBQHFOIUNM
  2. Effective Approaches to Attention- based Neural Machine Translation (Bahdanau+, 2015)

    • Deep LearningΛ༻͍ͯػց຋༁ • Local Attentionͱ͍͏৽͍͠ख๏ΛఏҊ • ͍͔ͭ͘ͷݴޠϖΞͰɺstate of the artΛୡ੒ ࠷ߴਫ४
  3. Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

    (Kumar+, 2015) • ৽͍͠ϞσϧʢDynamic Memory Networksʣ ΛఏҊͨ͠ • Recurrent Neural NetworkΛ૊Έ߹ΘͤͨΑ ͏ͳϞσϧʹͳ͍ͬͯΔ • ࣭໰Ԡ౴ɺ඼ࢺλά෇͚ɺڞࢀরղੳɺධ൑ ෼ੳͰstate of the art
  4. Seq2seqϞσϧͰͷ຋༁ 5IJT JT B QFO &04 ͜Ε ͸ ϖϯ Ͱ͢

    &04 ͜Ε ͸ ϖϯ Ͱ͢ 5IJTJTBQFOΛݻఆ௕ʹ Τϯίʔυ͍ͯ͠Δʂ
  5. ࠓ೔࿩͞ͳ͔ͬͨ͜ͱ • ֶशʢback propagation, minibatchʣ • ଛࣦؔ਺ʢlog loss, cross entropy

    lossʣ • ਖ਼ଇԽͷςΫχοΫ • dropout, batch normalization • ࠷దԽͷςΫχοΫ • RMSProp, AdaGrad, Adam • ֤छ׆ੑԽؔ਺ • (Very) Leaky ReLU, Maxout