Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Structure of NLP Papers

Structure of NLP Papers

The slides for paper writing tutorial 2022 in NAIST NLP Lab.

1ce0216a2274827385dab9a066077d98?s=128

Ukyo Honda

May 19, 2022
Tweet

Other Decks in Research

Transcript

  1. 4USVDUVSFPG/-11BQFST 6LZP)POEB /"*45/-1-BC .BZ 

  2. 1 0WFSWJFX • 5JQTPOIPXUPTUSVDUVSFZPVSQBQFS • 4USVDUVSFPGTFDUJPOT • 4USVDUVSFJOTJEFPGFBDITFDUJPO • .BOZUIBOLTUPNZDPBVUIPST

    NFOUPST BOEUIJTBXFTPNFUVUPSJBM • IUUQXXXQIPOUSPODPNTMJEFTOFVCJHOMQUVUPSJBMQEG
  3. 2 $BWFBUT • 5IJTUVUPSJBMBTTVNFTUIBUZPVSQBQFSTBSF • *O/-1PSPUIFSDMPTFEPNBJOTTVDIBT$7BOE.- • /PUTVSWFZUIFNFEFNPQBQFST • 5IJTUVUPSJBMEPFTOPUDPWFSBMMTFDUJPOT

    • 4FFUIFMJOLCFMPXGPSUIFOFXNBOEBUPSZTFDUJPOT • -JNJUBUJPOT  &UIJDBM 3JTLT FUD • IUUQTBDMSPMMJOHSFWJFXPSHSFTQPOTJCMF/-1SFTFBSDI • 5IFUJQTBSFCBTFEPONZQFSTPOBMQSBDUJDFBOEQSFGFSFODF • $PNNFOUTBOETVHHFTUJPOTBSFXFMDPNF
  4. 8IBUEPQBQFSTMPPLMJLF

  5. 4 3PVHITLFUDI 8IBUEPQBQFSTMPPLMJLF Abstract Introduction Methods Related Work Experiments Conclusion

    References !Consists of many different sections !Each section seems to have different roles and contents "‍‍What should we write in each section? "‍‍How to make sure the consistency?
  6. 5 'JOFSHSBJOFETLFUDI 8IBUEPQBQFSTMPPLMJLF Methods Related Work Experiments References !Papers are

    intentionally redundant for better understanding !Repeat of the basic arguments in different resolution "Just focus on structuring the basic arguments! 1. Background 2. Methods 3. Experiments Methods Experiments Background Methods Experiments Background Background Methods Experiments
  7. 6 #BTJDBSHVNFOUT #BDLHSPVOE Related Work Experiments References Methods Experiments Background

    Methods Experiments Background Background Methods Experiments Methods
  8. 7 #BTJDBSHVNFOUT #BDLHSPVOE • *OUSPEVDFTUIFQSPCMFNTZPVBEESFTTJOUIJTQBQFS  (PBMTBOETFUUJOHTPGUIFUBTL  #SJFGTLFUDIPGQSFWJPVTBQQSPBDIFT •

    %FUBJMFEEFTDSJQUJPOHPFTUP3FMBUFE8PSL  3FNBJOJOHQSPCMFNT UPBEESFTTJOUIJTQBQFS v Describe in this order
  9. 8 #BTJDBSHVNFOUT .FUIPET Related Work Experiments References Methods Experiments Background

    Methods Experiments Background Methods Background Methods Experiments
  10. 9 #BTJDBSHVNFOUT .FUIPET • %FTDSJCFTIPXUPTPMWFUIFQSPCMFNT  *OUVJUJWFFYQMBOBUJPO • )PXZPVSNFUIPETTPMWFUIFQSPCMFNT 

    %FUBJMTPGZPVSNFUIPET v Describe in this order
  11. 10 #BTJDBSHVNFOUT &YQFSJNFOUT Related Work Experiments References Methods Experiments Background

    Methods Experiments Background Methods Background Methods Experiments
  12. 11 #BTJDBSHVNFOUT &YQFSJNFOUT • %FNPOTUSBUFT UIBUZPVSNFUIPETTPMWFUIFQSPCMFNT PSOPU q &YQFSJNFOUBMTFUUJOHT q

    0WFSBMMDPNQBSJTPOXJUICBTFMJOFTBOEQSFWJPVTNPEFMT • 5PWFSJGZUIBUZPVSNFUIPETBDIJFWFUIFHPBM CFUUFS PSOPU q "CMBUJPOTUVEJFT • 5PTFFXIJDIQBSUPGZPVSNFUIPETDPOUSJCVUFT PSOPU q 'PDVTFEBOBMZTJT • 5PGVSUIFSJOWFTUJHBUFIPX ZPVSNFUIPETBDIJFWFUIFHPBMCFUUFS PSOPU JF XIFUIFSZPVSNFUIPETTPMWFUIFQSPCMFN ZPVQPJOUFEPVU q FUD
  13. 12 4VNNBSZTPGBS • 1BQFSTSFQFBUFEMZFYQMBJO  (PBMTBOEQSPCMFNT #BDLHSPVOE  4PMVUJPOT .FUIPET

     %FNPOTUSBUJPO &YQFSJNFOUT • 5IJTSFQFBUFOTVSFT • $MFBSFYQMBOBUJPOPGZPVSBSHVNFOUT • $POTJTUFODZUISPVHIUIFFOUJSFQBQFS
  14. -FUTTFFUIFTUSVDUVSFPGBOBDUVBMQBQFS

  15. 14 &YBNQMF • Hereafter we use this paper as the

    example • https://arxiv.org/abs/2202.06417
  16. 15 &YBNQMF 0VUMJOF Abstract Introduction Methods Related Work Experiments Conclusion

    References Background • Slightly different in the colored sections • But basically the same as the structure we have seen above
  17. 16 3PVHITLFUDI "CTUSBDU • Background • Methods • Experiments !Explanation

    in order
  18. 17 'JOFSHSBJOFETLFUDI "CTUSBDU ! Explanation in order 1. Task and

    goal • Text generation for app. 2. Prev. approaches • Stochastic sampling • Modified objective function 3. Remaining problems • Lack of coherence • Anisotropic dist. of rep. • Solutions • Calibrate the rep. space • Diversity-aware decoding • Demonstration • Outperform SOTA 1. 2. 3.
  19. 18 &YBNQMF *OUSPEVDUJPO !5IFTBNFTUSVDUVSF BTUIF"CTUSBDU • #VUXJUINPSFEFUBJMT • *OIJHIFSSFTPMVUJPO •

    Background • Methods • Experiments
  20. 19 &YBNQMF #BDLHSPVOE • &YUFOEFE #BDLHSPVOE • &MBCPSBUJPOPOUIFUBTLTFUUJOHT • 4PNFUJNFT#BDLHSPVOEJTFYUFOEFEBOEDVUPVUGPSFMBCPSBUJPO

    • 5PJOUSPEVDFGPSNBMUBTLTFUUJOHT • 5PFYQMBJOQSFWJPVTMZVOLOPXOQSPCMFNT • FUD Methods Related Work Experiments References Methods Experiments Background Methods Experiments Background Background Methods Experiments Background
  21. 20 &YBNQMF .FUIPEPMPHZ ! Explanation in order 1. Intuitive explanation

    on how your methods solve the problem 2. Then describe the details 1. 2. 1. 2.
  22. 21 &YBNQMF &YQFSJNFOUT • %PDVNFOU(FOFSBUJPO • FYQFSJNFOUBMTFUUJOHT • 3FTVMUT •

    )VNBO&WBMVBUJPO • 0QFO%PNBJO%JBMPHVF(FOFSBUJPO • FYQFSJNFOUBMTFUUJOHT • SFTVMUT • 'VSUIFS"OBMZTJT q Experimental settings q Overall comparison q Ablation study q Experimental settings q Overall comparison q Ablation study q Focused analysis
  23. 22 0WFSBMMDPNQBSJTPO &YQFSJNFOUT q Overall comparison • Demonstrates that the

    proposed method achieves the goal better than the SOTA methods
  24. 23 q Ablation study • Two proposed methods • Each

    of them works well and works the best in combination "CMBUJPOTUVEZ &YQFSJNFOUT
  25. 24 'PDVTFEBOBMZTJT &YQFSJNFOUT q Focused analysis • Proposed methods improve

    the overall performance, but how? ü Do they work as you hypothesized? ü Do they solve the problem you pointed out? Ø Confirming here that the methods mitigate the anisotropic distribution of token representations
  26. 25 'PDVTFEBOBMZTJT &YQFSJNFOUT q Focused analysis • Qualitative analysis

  27. 26 &YBNQMF $PODMVTJPO !4BNFBT"CTU*OUSP CVU • -FTTGPDVTPO#BDLHSPVOE • 1SPCMFNPOMZ •

    .PSFGPDVTPO.FUIPETBOE&YQ • .PSFDPODJTFUIBO"CTU • +VTUUPTVNNBSJ[FSFFNQIBTJ[F ZPVSDPOUSJCVUJPOT • 'VUVSF8PSL
  28. 27 &YBNQMF 3FGFSFODFT • 6TFBVOJGJFEGPSNBU ! *O"$-  *O/""$- 

    " *O"$-  *O1SPDFFEJOHTPGUIFUI"OOVBM.FFUJOHPGUIF"TTPDJBUJPOGPS $PNQVUBUJPOBM-JOHVJTUJDT *O1SPDFFEJOHTPGUIF$POGFSFODFPGUIF/PSUI"NFSJDBO$IBQUFS PGUIF"TTPDJBUJPOGPS$PNQVUBUJPOBM-JOHVJTUJDT • IUUQTZPTIJUBLBVTIJLVOFUIPX@UP@XSJUF@SFGFSFODFTIUNM
  29. 28 &YBNQMF "QQFOEJY3FMBUFE8PSL • #SJFGEFTDSJQUJPOPGQSFWJPVTBQQSPBDIFTBOEEJGGFSFODF GSPNZPVST • 3FMBUFE8PSLPGUFODPNFTSJHIUBGUFS*OUSPEVDUJPO • 4PVOETGBJSBTJUJTBQBSUPG#BDLHSPVOE

    • #VUQFSTPOBMMZ*EPOPUFNQMPZUIBUTUSVDUVSF • 1SFWJPVTBQQSPBDIFTBSFOPUZPVSNBJOQPJOUTDPOUSJCVUJPOT • 3FMBUFE8PSLUFOETUPCFUPPMPOH Ø+VTUHPPOUP.FUIPETGJSTUBOEHFUCBDLCFGPSF$PODMVTJPO v3FMBUFE8PSLJOBQQFOEJYIFSFJTBOFYUSFNFDBTFBOEOPUSFDPNNFOEFE
  30. "OZRVFTUJPOTTPGBS

  31. 30 5JQT 8SJUF"CTUSBDUGJSTU • "CTUSBDU≈ 0VUMJOF • %POUCFXPSSJFEJGUIJTTUFQUBLFTUPPMPOH • 3FNFNCFSUIBUQBQFSTDPOTJTUPG"CTUSBDUJOEJGGFSFOUSFTPMVUJPOT

    • *GZPVBSFEPOFXJUIUIJT ZPVBSFBMNPTUUIFSF • 4IBSFZPVSESBGUPG"CTUSBDUXJUIZPVSDPBVUIPSTBOESFWJTF • 3FWJTFJUCBTFEPOGFFECBDLCFGPSFHPJOHPOUPUIFPUIFSTFDUJPOT • %POUIBWFUPCFTFOTJUJWFBCPVUMFOHUIBUUIFESBGUTUBHF
  32. 31 5JQT 8SJUF"CTUSBDUCFGPSFTUBSUJOHSFTFBSDI • +VTUBTIPSUNFNPJTGJOF • *UIFMQTZPVUP  4NPPUIMZNPWFPOUPXSJUJOHQBQFST 

    ,FFQJONJOEZPVSIZQPUIFTJTEVSJOHSFTFBSDI ≈ ,FFQJONJOEXIBUUPEPOFYU • 8IBUJTUIFHPBMBOEXIBUJTUIFSFNBJOJOHQSPCMFN  #BDLHSPVOE • 8IBUJTUIFQPTTJCMFTPMVUJPO UPUIFQSPCMFN  .FUIPET • )PXTIPVMEZPVEFNPOTUSBUF UIBUZPVSNFUIPETPMWFTUIFQSPCMFN  &YQFSJNFOUT  3PVHIMZKVEHFXIFUIFSZPVSSFTFBSDIJTJOUFSFTUJOHPSOPUCFGPSFXBTUJOH UJNFPOJNQMFNFOUBUJPOFYQFSJNFOUT
  33. 32 5JQT %POPUDPOGVTFFOETXJUINFBOT ! "DPNNPONJTUBLFJOGJOEJOHUIFQSPCMFNTUPTPMWF ! :PVSNFUIPESFTPVSDFFUDJTUIFNFBOT PGTPMWJOHQSPCMFNT • 5IFNFUIPESFTPVSDFFUDJUTFMGJTOPUUIFFOE

    Ø/PUVTJOH9JTOPUBQSPCMFNUPTPMWF " 9IBTOPUCFFOVTFEJOUIFQSFWJPVTXPSL TPXFVTF9 FH $POUSBTUJWFMFBSOJOHIBTOPUCFFOVTFEJOUFYUHFOFSBUJPO TPXFVTFJU # 9DBOTPMWFUIJTQSPCMFNMFGUCZUIFQSFWJPVTXPSL TPXFVTF9 FH $POUSBTUJWFMFBSOJOHDBOTPMWFUIFBOJTPUSPQJDEJTUPGSFQ TPXFVTFJU
  34. 33 8IZEPFTJUNBUUFS %POPUDPOGVTFFOETXJUINFBOT X Current Performance ! 9IBTOPUCFFOVTFEJOUIFQSFWJPVTXPSL TPXFVTF9 X

    Better Performance Apply X ü Overall performance ü Ablation study ✘ Focused analysis • As you don't have any guess on how/why X works here • "X does something awesome!" • You don't have any hypotheses to test regarding how/why X works ! Shallow contribution
  35. 34 8IZEPFTJUNBUUFS %POPUDPOGVTFFOETXJUINFBOT Problem X Current Performance ! 9DBOTPMWFUIJTQSPCMFNMFGUCZUIFQSFWJPVTXPSL TPXFVTF9

    Problem X Better Performance Apply X ü Overall performance ü Ablation study ü Focused analysis • You can check how X solves the problem based on hypotheses • That gives the explanation of why X works here ! Deeper insight into the limitation/generalization of X ! More critical contribution
  36. 35 5BLFIPNFNFTTBHFT! ü 1BQFSTSFQFBUFEMZFYQMBJO  (PBMTBOEQSPCMFNT #BDLHSPVOE  4PMVUJPOT .FUIPET

     %FNPOTUSBUJPO &YQFSJNFOUT ü 8SJUF"CTUSBDU JTI CFGPSFBOZUIJOH ü %POPUDPOGVTFFOETXJUINFBOT