文献紹介 9月3日

Pervasive Attention: 2D Convolutional Neural Networks
for Sequence-to-Sequence Prediction

Transcript

1. () Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
2. 
   ▪  ▪ Maha Elbayad, Laurent Besacier, Jakob Verbeek ▪ Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction. ▪ The SIGNLL Conference on Computational Natural Language Learning. 2018. ▪  ▪
   ,   2
3. 
    ▪ - A - A ▪ E 3
4. 
    ▪ rn ▪ RNN Decoder c n drn ▪ - E Attention d ▪ Decoder dc d ▪ Attention e c d ▪ Decoder dD 4
5. 
    ▪ )( ) )( ) ▪ CNN ▪ N N C Attention 5
6. 
    ▪  DenseNet ▪ 
    SoTA
7. 
    ▪ 546(.' ▪  &  2-$ ▪ !/ ) Mask$ ▪ #"  !2 + ▪ 120 &%
8. 
    ▪
      Source Sequence  Pooling
9. 
    ▪ Self-Attention ▪ Hpool   
    
10. 
    ▪ Dataset ▪ IWSLT (De-En) ▪ Parameters ▪  ! &(')%#  ▪ DenseNetlayer size : 24 ▪ DenseNetgrowth rate : 32 ▪ embedding size : 128 ▪  Pooling Attention 2#" 
11. 
    11 ▪ 
    SoTA ▪  
12. 
    12
13. 
     ▪ ▪ TE N T N A C T ▪ S 13