Upgrade to Pro — share decks privately, control downloads, hide ads and more …

自然言語処理と深層学習の最先端

tkng
January 15, 2016

 自然言語処理と深層学習の最先端

第4回 JustTechTalk の発表資料

tkng

January 15, 2016

Transcript

44 slides
  1. Show, Attend and Tell: Neural Image Caption Generation with Visual

    Attention (Xu+, 2015) • ը૾ʹର͢Δղઆจͷੜ੒ IUUQLFMWJOYVHJUIVCJPQSPKFDUTDBQHFOIUNM
  2. Effective Approaches to Attention- based Neural Machine Translation (Bahdanau+, 2015)

    • Deep LearningΛ༻͍ͯػց຋༁ • Local Attentionͱ͍͏৽͍͠ख๏ΛఏҊ • ͍͔ͭ͘ͷݴޠϖΞͰɺstate of the artΛୡ੒ ࠷ߴਫ४
  3. Ask Me Anything: Dynamic Memory Networks for Natural Language Processing

    (Kumar+, 2015) • ৽͍͠ϞσϧʢDynamic Memory Networksʣ ΛఏҊͨ͠ • Recurrent Neural NetworkΛ૊Έ߹ΘͤͨΑ ͏ͳϞσϧʹͳ͍ͬͯΔ • ࣭໰Ԡ౴ɺ඼ࢺλά෇͚ɺڞࢀরղੳɺධ൑ ෼ੳͰstate of the art
  4. Seq2seqϞσϧͰͷ຋༁ 5IJT JT B QFO &04 ͜Ε ͸ ϖϯ Ͱ͢

    &04 ͜Ε ͸ ϖϯ Ͱ͢ 5IJTJTBQFOΛݻఆ௕ʹ Τϯίʔυ͍ͯ͠Δʂ
  5. ࠓ೔࿩͞ͳ͔ͬͨ͜ͱ • ֶशʢback propagation, minibatchʣ • ଛࣦؔ਺ʢlog loss, cross entropy

    lossʣ • ਖ਼ଇԽͷςΫχοΫ • dropout, batch normalization • ࠷దԽͷςΫχοΫ • RMSProp, AdaGrad, Adam • ֤छ׆ੑԽؔ਺ • (Very) Leaky ReLU, Maxout

More Decks by tkng

Other Decks in Technology