試行回数の増やし方

 試行回数の増やし方

東北大学 乾研究室の研修用に使っているスライドです

A8e5603efe2e904e5643163d1e0cb0d6?s=128

Shun Kiyono

October 16, 2020
Tweet

Transcript

  1. ࢼߦճ਺ͷ૿΍͠ํ RIKEN AIP / Tohoku University Shun Kiyono

  2. ࢼߦճ਺ͱ͸ʁ October 16, 2020 Inui Laboratory 2 ੒ޭͷํఔࣜ [լ୔ 2012]

    ੒ޭճ਺=ࢼߦճ਺º੒ޭ཰ ੒ޭͬͯͳΜͩΑʁʁ ͱࢥ͏͔΋͠Ε·ͤΜ͕… ͜͜Ͱ͸ SOTAୡ੒ͱ͔τοϓձٞ ࠾࿥ʹ͓͖ͯ͠·͠ΐ͏
  3. ੒ޭ཰͸ʮӡʯʹେ͖͘ґଘ • ӡཁૉɿݚڀςʔϚ΍पΓͷ؀ڥ • ͨ·ͨ·ಉ͡ςʔϚͰ࿦จ͕ग़Δͱ͔… • ࠓ೔໌೔Ͱίϯτϩʔϧ͢Δͷ͸ࠔ೉ • ݴޠԽࠔ೉ͳ҉໧త஌ࣝͨͪ •

    ෼໺/λεΫʹର͢Δ௚ײɺ௕೥ͷ஌ݟɺܦݧɺ೥ͷޭͳͲ • ʮےͷྑ͍ʯݚڀςʔϚΛબͿ͜ͱͰ͋Δఔ౓ίϯτϩʔϧՄ • ͱ͸͍͑… October 16, 2020 Inui Laboratory 3 ੒ޭͷํఔࣜ [լ୔ 2012] ੒ޭճ਺=ࢼߦճ਺º੒ޭ཰ ͜ͷ߲Λίϯτϩʔϧ͢Δͷ͸ࠔ೉ ஫ɿࢲͷओ؍Ͱ͢
  4. ࢼߦճ਺=ख࣋ͪͷ࣌ؒ×Ұճͷࢼߦʹ͔͔Δ࣌ؒ ͳͷͰɺࢼߦճ਺Λ૿΍͍ͨ͠ October 16, 2020 Inui Laboratory 4 ੒ޭͷํఔࣜ [լ୔

    2012] ੒ޭճ਺=ࢼߦճ਺º੒ޭ཰ ͜Ε͸༗ݶɿ ͕͖ࠩͭʹ͍͘ ʢਓੜ͔ͩΒͶ…ʣ [ߴ੉ 2016] ͜͜Λ୹͘͢Δඞཁ͕͋Δʂ
  5. ʮࢼߦʹ͔͔Δ࣌ؒʯͷ಺༁ October 16, 2020 Inui Laboratory 5 • ϓϩάϥϜͷ࣮૷࣌ؒ •

    طଘͷίʔυΛಈ͔ͯ͠ΈΔ • खݩͷΞΠσΞͷ࣮૷ • ϓϩάϥϜͷσόοά • ϓϩάϥϜͷ࣮ߦ࣌ؒ • લॲཧ • Ϟσϧͷֶश • Ϟσϧͷ༧ଌ • ධՁ ʮӡʯͬΆ͘ͳ͍ؾ͕͖͔ͯͨ͠΋ʁ
  6. ຊൃදͷ໨త • ࢼߦճ਺Λ૿΍ͨ͢Ίͷ࣮ફతͳʮٕज़ʯΛ঺հ October 16, 2020 Inui Laboratory 6

  7. ͓͜ͱΘΓ • ͜ͷൃද͸ࣗ෼ͷֶΜͰ͖ͨTipsΛಠஅͱภݟͰ ·ͱΊͨ΋ͷͰ͢ • ݸਓͷओ؍͕͔ͬ͠Γೖͬͯ·͢ • ʮશ෦Ͱ͖ͳ͍ͱͩΊʯͰ͸ͳ͍ • গͳ͘ͱ΋ࣗ෼͸શ෦͸Ͱ͖ͯͳ͍

    • ʮ͓͢͢Ίͷډञ԰঺հʯ͘Β͍ͷؾ࣋ͪͰ ݟͯཉ͍͠ October 16, 2020 Inui Laboratory 7
  8. ࣮૷ฤ October 16, 2020 Inui Laboratory 8

  9. ᶃ࣮૷ͷॱ൪ʹؾΛ͚ͭΔ October 16, 2020 Inui Laboratory 9 • Nothing works

    and everything is too slow à Panic • Simplify model à Go back to basics: bag of vectors + nnet • Make a smaller network and dataset for debugging • Once no bugs: increase model size • Make sure you can overfit to your dataset • Plot your training and dev errors over training iterations • Then regularize with L2 and Dropout • Then do hyperparameter search • Come to OH! ( 3/6/18 Richard Socher Lecture 1, Slide 2 খ͍͞؆୯ͳϞσϧɾػೳͷ࣮૷͔Β࢝ΊΑ͏ খ͍࣍͞ݩɾσʔληοτͰࢼͯ͠ɼόάΛચ͍ग़ͦ͏ όά͕ແ͘ͳͬͨΒϞσϧͷαΠζͱσʔλ਺Λ૿΍ͦ͏ 5SBJOJOHσʔλʹରͯ͠աֶशͰ͖Δ͜ͱΛ֬ೝ͠Α͏ 5SBJOJOHͱ7BMJEͷ-PTTΛಉ͡άϥϑͰ֬ೝ͠Α͏ աֶशΛ֬ೝ͔ͯ͠Βਖ਼ଇԽख๏ʢ-΍%SPQPVUʣΛࢼͦ͏ ࠷ޙʹϋΠύʔύϥϝʔλͷ୳ࡧΛ͠Α͏ https://web.stanford.edu/class/cs224n/lectures/lecture15.pdf ্͔Βॱ൪ʹ΍Δ Լ͔Βࢼͯ͠͏·͍͔͘ͳ͍ à ্ʹ໭ͬͯ΍Γ௚͠ʢख໭Γʣ͕ൃੜ ࣌ؒͷແବ
  10. ྫ͑͹ɿ෼ྨ໰୊ͷ৔߹ • Ϟσϧ: Bag-of-Words • ࣍ݩͷେ͖͞: ~30࣍ݩ • Optimizer: SGD

    (lr=0.01) or Adam (α=0.0001) • ύϥϝʔλॳظԽ: ϑϨʔϜϫʔΫͷσϑΥϧτ • ਖ਼ଇԽ: ઈରʹ͠ͳ͍ • σʔλαΠζ:10Πϯελϯε • όοναΠζ:10 • ͭ·Γɺ1όονʹରͯ͠loss͕0ʹͳΔ͔֬ೝ͢Δ • Epoch਺: ~10 • CPU/GPU: ྆ํͰಈ͔͘ʁ October 16, 2020 Inui Laboratory 10
  11. ᶄίʔυΛ͏·͘෼ׂ͢Δ • લॲཧͷྫ • Sentence Split, Tokenization, POS tagging, Lemmatization,

    Dependency Parsing, NER October 16, 2020 Inui Laboratory 11 ੜσʔλ લॲཧࡁσʔλ Ϟσϧ લॲཧ ܇࿅ ػցֶश͋Δ͋ΔύΠϓϥΠϯ
  12. ΍ͬͯ͸͍͚ͳ͍ɿ࠷ڧtrain.py October 16, 2020 Inui Laboratory 12 ੜσʔλ લॲཧࡁσʔλ Ϟσϧ

    લॲཧ ܇࿅ શ෦train.py Ͱ࣮ݱͰ͖ͦ͏ʁʂ ๻͸͜ΕΛ࠷ڧUSBJOQZͱݺΜͰ͍·͢
  13. ͋Γ͕ͪͳ࠷ڧtrain.pyͷҰ෦ • ຖճMeCabͰ෼͔ͪॻ͖ͯ͠ΔͶ… October 16, 2020 Inui Laboratory 13 EFGSFBE@EBUBTFU

    QBUI  EBUBTFU<> XJUIPQFO QBUI BTGJ GPSMJOFJOGJ EBUBTFUBQQFOE .F$BCQBSTF MJOFTUSJQ  SFUVSOEBUBTFU
  14. ࠷ڧtrain.pyͷ͕͜͜΍͹͍ • ֶशͷͨͼʹಉ͡લॲཧ͕૸Δ • ࣌ؒͷແବ • ܭࢉࢿݯͷແବ • ిؾ୅ͷແବ •

    ಉ͡ʮલॲཧʯʹͳΔͱ͸ݶΒͳ͍à όάͷԹচ • ϥΠϒϥϦͷόʔδϣϯ͕ҧ͏ͱղੳ݁Ռ͸มΘΔ • ྫ: Stanford CoreNLP • લॲཧ͕ҧ͏ --> ग़དྷ্͕Δσʔλ΋ҧ͏ • ͭ·ΓϞσϧ΋ҧ͏ • લॲཧ͸Ұ౓͚ͩʹͯ͠ɺಉ͡σʔλΛ࢖͍ଓ͚Δ΂͖ October 16, 2020 Inui Laboratory 14
  15. ᶅσʔλͷܗࣜʹؾΛ͚ͭΔ • CSV΍TSVɿ΍ΊΑ͏ • ಛʹࣗલͰreader/writerΛॻ͘ͷ͸΍ΊΑ͏ • Pythonͷࣙॻͱͷ૬ੑ͕ѱͯ͘໘౗ • JSON Linesɿ΍͍͖ͬͯ

    • ཁ͢Δʹ1ߦ1json • json.dumps / json.loads Ͱ؆୯ʹಡΈॻ͖Ͱ͖Δ October 16, 2020 Inui Laboratory 16
  16. ᶆੵۃతʹassertionΛ࢖͓͏ • ʮ͜͜͸ઈର͜ͷ৚͕݅ຬͨ͞ΕͯΔ͸ͣʂʯ ͱ͍͏ՕॴʹೖΕ͓ͯ͘ • Ξϗͳ৚݅Ͱ͋Δ΄Ͳྑ͍ͱࢥ͏ • ͍͔ͭ݁ہྫ֎Λ౿Ή͜ͱʹͳΔ • ྫ͑͹…

    • ߦྻԋࢉޙͷshapeͷ֬ೝʢίϝϯτΑΓ΋assertionʣ • ϋΠύϥͷ֬ೝ (AdamͰalpha=0.1ͱ͔Λճආʣ October 16, 2020 Inui Laboratory 17 BTTFSUMFO TSD@EBUB MFO USH@EBUB  EncDecͷ࣮૷Ͱ Α͘࢖͏
  17. ᶇargparseΛ࢖͏ • ίʔυͷதʹม਺Λϕλॻ͖͢Δͷ͸΍ΊΔ • ίʔυͷ࠶ར༻ੑ͕མͪΔ • argparseߏจΛຖճॻ͘ͷ͕໘౗ͳΒɺΤσΟλ ͷςϯϓϨʔτ΍εχϖοτʹొ࿥͓ͯ͘͠ • choices/default/required

    option ͸೺Ѳ͓ͯ͘͠ October 16, 2020 Inui Laboratory 18 γϯϓϧʹ஍ࠈ ݟ௨͕͠ྑ͍
  18. ᶈਓʹݟͤΒΕΔίʔυʹ͓ͯ͘͠ • ʮࣗ෼ͷίʔυΛଞਓ͕ݟΔػձͳ͍͚Ͳʁʯ • ·͋ɺ͔֬ʹݚڀ͸ݽಠͳӦΈ͚ͩͲ… • ͍͍͑ɺ͋Γ·͢ʂ • Ұिؒޙͷࣗ෼͸ଞਓͰ͢ •

    ࡉ͔͍࢓༷΍ҙਤ͸͙͢ʹ๨Εͯ͠·͏ • ͳͷͰ… • ͨ͘͞ΜίϝϯτΛೖΕ͓ͯ͘ • ҙຯͷ͋Δม਺໊Λ͚͓ͭͯ͘ • ݴޠॲཧ100ຊϊοΫ͸ྑ͍࿅श • ଞਓ͕ݟͯͻͱ໨Ͱ෼͔Δίʔυͩͱྑ͍ • ·ͣ͸ϦʔμϒϧίʔυΛಡΜͰΈΑ͏ October 16, 2020 Inui Laboratory 19
  19. ᶉॲཧঢ়گΛՄࢹԽ͠Α͏ • ࣌ؒͷ͔͔ΔforจͰ͸progressbar/tqdmΛ࢖͏ • ͪΌΜͱॲཧ͕ਐΜͰ͍Δ͜ͱ͕෼͔Δ • ඞཁͳ࣌ؒ΍ॲཧର৅ͷ௕͔͞Β෼͔Δόά΋͋Δ • ʁʁʁʮલॲཧʹ24࣌ؒʁົͩͳ…ʯ •

    ʢόάʹؾ͕ͭ͘·Ͱʹʣ଴ͭ࣌ؒ΋ݮΔ October 16, 2020 Inui Laboratory 20
  20. ᶊϓϩάϥϛϯά༻ͷϑΥϯτΛ࢖͏ • 1ͱlͷ۠ผɺoͱOͱ0ͷ۠ผɺIͱ|ͷ۠ผ͸ॏཁ • λʔϛφϧͱΤσΟλʹઃఆ͓ͯ͘͠ • Source Han Code JPɺRictyɺHack͋ͨΓ͕༗໊

    ͔ͳʁ • ͜͜Β΁Μ͸޷Έͷ໰୊à ޷͖ͳ΍ͭΛ࢖͏ͷͰOK • Α͘Θ͔ΒΜͳΒ: Ricty October 16, 2020 Inui Laboratory 21
  21. ᶋਪ࿦Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ • ਪ࿦Λ͢ΔͨΊͷεΫϦϓτ͸ࠓ͙͢ʹॻ͜͏ • ଞͷ࿦จͱൺֱͰ͖ΔΑ͏ʹ͠Α͏ • ܇࿅͚ͩͰ͖ͯ΋࿦จ͸ॻ͚ͳ͍ʂ • ͜ͷ࡞ۀΛʮτϯωϧͷ։௨ʯͱ๻͸ݺΜͰ͍·͢ •

    %0/& ։௨ JTCFUUFSUIBO1&3'&$5 ͖Ε͍ͳίʔυ • ࠷௿ݶඞཁͳεΫϦϓτ •  อଘͨ͠ύϥϝʔλΛಡΈࠐΜͰ •  ະ஌ͷσʔλʹରͯ͠ •  ༧ଌ݁ՌΛฦ͠ •  είΞʢFH '஋ #-&6 ਖ਼ղ཰ʣ Λܭࢉ͢Δ October 16, 2020 Inui Laboratory 22
  22. ᶋਪ࿦Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ October 16, 2020 Inui Laboratory 23 ੜσʔλ લॲཧࡁσʔλ Ϟσϧ

    લॲཧ ܇࿅ ᶃ Ϟσϧͷֶशʢ͜͜·ͰͰຬ଍͕ͪ͠ʣ ᶄ ϞσϧͷධՁʢͬͪ͜΋ॏཁʂʣ ςετσʔλ ܇࿅ࡁϞσϧ ೖྗ ༧ଌ ධՁ Compute_acc.py ਖ਼ղ཰ 80% ධՁ εΫϦϓτ
  23. ίʔυ؅ཧฤ October 16, 2020 Inui Laboratory 24

  24. αʔόͱϩʔΧϧͷ΍ΓऔΓ: SFTPҰ୒ • αʔόͰશͯͷίʔυΛॻ͘ • ϩʔΧϧ͔ΒίʔυΛҠ͢ख͕ؒແ͍ • αʔόͷࢮ⇛ݚڀɾଔ࿦ͷࢮʢσΟεΫ͸ࢮ͵΋ͷʣ • ϩʔΧϧͰίʔυΛॻ͖ɼखಈͰαʔόʹίϐʔ

    • ϩʔΧϧͷίʔυ͸৑௕ԽͰ͖ΔʢDropboxͳͲʣ • αʔό΁ͷίϐʔ͕໘౗ʢˍ๨ΕΔʣ • ϩʔΧϧͰίʔυΛॻ͖ɼࣗಈͰαʔόʹίϐʔ • ϑΝΠϧอଘͷ౓ʹཪͰಉظ͕૸Δ • ΤσΟλ໊ + SFTPͰݕࡧͯ͠ΈΑ͏ • PyCharm, Atom, VSCodeͳΒͰ͖Δ͸ͣ October 16, 2020 Inui Laboratory 25 ܁Γฦ͞ΕΔ࢒೦ͳޫܠ ຖճscp΍rsyncͯ͠ͳ͍ʁ
  25. ۩ମྫ: ਗ਼໺ͷ৔߹ October 16, 2020 Inui Laboratory 26 ͜Ε΋׬ᘳͰ͸ͳ͍͕ɼ͜͜ࡾ೥Ҏ্͸͜ΕͰམͪண͍͍ͯΔ EncDecAttn

    EncDecAttn deploy/EncDecAttn (SFTP) push pull git git MyPC GitHub (private repo) Dropbox dev ݚڀϓϩδΣΫτ͝ͱʹ͜ͷઃఆΛ͓ͯ͘͠ ׳ΕΕ͹30ඵͰͰ͖Δ ͜͜Ͱ։ൃͱ σόοά
  26. όʔδϣϯ؅ཧɿ΍Δ΂͖ • GitΛ࢖͓͏ • ίϛοτϝοηʔδ͕update΍fixͩΒ͚Ͱ΋େৎ෉ • add, commit, push, pull͕࢖͑Ε͹OK

    • GitίϚϯυΛ࢖͏ΑΓɼGUIͷΞϓϦΛ࢖͓͏ • Ωʔϫʔυ: SourceTree, Fork, GitGraken • ίϚϯυ͸ෆඞཁʹ೉͍͠ • ڞಉݚڀऀʹݟ͑ΔΑ͏ʹίʔυΛڞ༗͠Α͏ • GitHubΛ༗ޮʹ࢖͓͏ • PrivateϦϙδτϦΛ࡞Ζ͏ • ࠓ͙͢GitHubͷEducationalΞΧ΢ϯτΛ࡞Ζ͏ʢແྉͩΑʂʣ • ະൃදͷݚڀ੒ՌΛ1VCMJDͳܗͰެ։͢Δͷ͸΍ΊΑ͏ • ݚڀࣨͷGitHubάϧʔϓΛੵۃతʹ׆༻͠Α͏ • ݚڀࣨͷਓ͸σϑΥϧτͰશһݟΒΕͯศར October 16, 2020 Inui Laboratory 27 ʮࠓͷίʔυ͕ಈ͔ͳ͘ͳΔͷ͕ݏͰɼมߋΛՃ͑ͮΒ͍ʯ͜ͱ͕ͳ͍Α͏ʹ
  27. Git: CLIΑΓ΋GUI October 16, 2020 Inui Laboratory 28 • ͳΕΔ·Ͱ͸CLIΑΓ΋GUIΛ࢖͓͏

    • CLI͸֮͑Δ΂͖͜ͱ͕ଟ͗͢Δ • add, commit, push, fetch, pull, … • GUIΫϥΠΞϯτ͸ࢹ֮తʹΘ͔Γ΍͍͢ Θ͔ΒΜ Θ͔Δʂ
  28. ݚڀࣨͷGitHubάϧʔϓΛ࢖͏ • ݚڀ༻ͷϦϙδτϦ͸cl-tohokuάϧʔϓʹ࡞͓ͬͯ͘ • ݚڀࣨͷਓͳΒ୭Ͱ΋ݟΒΕΔͷͰศརʢট଴ͷख͍ؒΒͣʣ • ஫ɿϓϥΠϕʔτϦϙδτϦʹ͓ͯ͘͜͠ͱ • ະൃදͷݚڀ੒ՌΛެ։৘ใʹͯ͠͸͍͚ͳ͍ October

    16, 2020 Inui Laboratory 29
  29. σΟϨΫτϦߏ଄ʹؾΛ഑Ζ͏ ࢲͷ৔߹ workspace src lib util showcase cabocha nlp100 ml

    euler experimetal search lib target bin data project main test java script src main test java scala scala (JUͰ͸؅ཧ͠ͳ͍ tmp ϓϩδΣΫτ ୯ҐͰ (JU؅ཧ  October 16, 2020 Inui Laboratory 30 ݚڀͷ؅ཧ [দྛ 2017] http://www.cl.ecei.tohoku.ac.jp/local/handouts/2017/misc/misc-20171120-y-matsu.pdf Կ͕Կ΍Β… Θ͔Γ΍͍͢ GitͰ؅ཧ͢Δର৅Λ෼͔Γ΍͘͢
  30. ੒ޭʢࣦഊʣΛ࠶ݱ͠Α͏ • ࣮ݧϩάΛ࢒ͦ͏ɽ • ྫ͑͹: {࣮ݧͷ೔࣌ɼσΟϨΫτϦ໊ɼίϚϯυ໊ɼ argparseͷத਎ɼPythonͷόʔδϣϯɼϥΠϒϥϦͷ όʔδϣϯɼGitͷίϛοτID} ΛϑΝΠϧʹॻ͖ग़͢ •

    ϩάΛ࢒͢ͱ͖: printΑΓ΋loggerΛ࢖͓͏ • Pythonඪ४ͷloggerͰ΋ྑ͍͠ɼlogzeroͷΑ͏ͳ ϥΠϒϥϦΛ࢖ͬͯ΋ྑ͍ • ඪ४ग़ྗ/Τϥʔग़ྗΛ࢖͍෼͚Α͏ • teeΛ׆༻͠Α͏ October 16, 2020 Inui Laboratory 31 ྫ: ਗ਼໺ͷ࣮ݧεΫϦϓτ ???ʮState-of-the-Artग़ͨΜ͚ͩͲͳʙʯ
  31. ಓ۩ཱͯฤ ศརͳಓ۩Λ࢖͓͏ October 16, 2020 Inui Laboratory 32

  32. ZSHͷϓϩϯϓτΛ੔උ͢Δ October 16, 2020 Inui Laboratory 34 ΧϨϯτσΟϨΫτϦ ͱ Gitͷϒϥϯν

    ͱ PythonͷԾ૝؀ڥ Λৗʹग़͓ͯ͘͠ ͱͬͯ΋؆ૉ શͯΛهԱ͢Δྗ͕ඞཁ Α͘Θ͔Βͳ͍ͳΒspaceship-promptΛೖΕ͓ͯ͘
  33. ZSHʹΑΔݡ͍ʁิ׬ • ZSHͷੈքɿجຊ͸λϒ࿈ଧ • λϒҎ֎ΛଧͬͨΒෛ͚ͷੈք؍Ͱੜ͖͍ͯ͜͏ • ΍ΓํʁάάΓ·͠ΐ͏ October 16, 2020

    Inui Laboratory 35
  34. ΠϯλϥΫςΟϒϑΟϧλ • pecoͱ͔fzfͱ͔ • ઃఆ: άάΔͱग़·͢ October 16, 2020 Inui

    Laboratory 38 ίϚϯυཤྺͷิ׬ σΟϨΫτϦؒͷҠಈ ެ։༻ʹ࡟আ ެ։༻ʹ࡟আ
  35. ৘ใऩूฤ October 16, 2020 Inui Laboratory 47

  36. ৘ใऩूΛ͠Α͏ • Slack • #misc_chainer, #misc_python, #misc_tech, #dep_info_sys, #b_sig_nn •

    Twitter • ͜ͷล͸লུ • ݚڀࣨͷաڈͷݚڀTipsͳͲ October 16, 2020 Inui Laboratory 48 TwitterͰݚڀ͕ਐΉʢ͜ͱ͕͋Δʣ
  37. ʮਖ਼͍͠ʯ৘ใऩूΛ͠Α͏ • ΅͘ʮxxxͬͯπʔϧΛαʔόʹೖΕͯΈΑ͏ʯ • QiitaͷهࣄΛಡΜͰࢼ͢ • sudoΛ࢖͍ͬͯΔʢ͜ͱ͕ଟ͍ʣ • هࣄͷ࣭͕ۄੴࠞަ •

    ެࣜυΩϡϝϯτΛ༁͚ͨͩ͠ͷهࣄʢ͔͠΋ޡ༁ͩΒ͚ʣ • ୈҰઢͷݚڀऀ͕ॻ͍ͨղઆهࣄ • ެࣜυΩϡϝϯτʹैͬͯೖΕΔ • ։ൃऀ(ୡ)͕ॻ͍ͯΔͷͰ৴༻Ͱ͖Δ • ӳޠΛڪΕͳ͍৺͕ॏཁ • ྫQZFOWͷΠϯετʔϧ • 2JJUBͷ΍ͬͯΈͨܥهࣄʹৼΓճ͞ΕΔͷ͸࣌ؒͷແବ • ମײͱͯ͠ɼ2JJUBΑΓ΋4UBDL0WFSGMPXͷํ͕ϕ λʔ October 16, 2020 Inui Laboratory 49 ޡͬͨهࣄͷ্Ͱࢼߦࡨޡ͢Δͷ͸͕࣌ؒ΋͍ͬͨͳ͍
  38. ࢼߦͷճ਺ΛݮΒͦ͏ʢαϘΖ͏ʣ • ࢼߦ͠ͳͯ͘ྑ͍΋ͷ͸αϘΓํ΋ߟ͑Δ • ϋΠύʔύϥϝʔλ͸࿦จ΍ஶऀͷίʔυʹॻ͍ͯ͋Δ • ࿦จΛಡΈղ͘ΑΓ΋ɼ࣮૷ΛಡΜͩ΄͏͕଎͍͜ͱ΋ • ྫ: Attention

    is All You Needʹର͢ΔɼAnnotated Transformer (http://nlp.seas.harvard.edu/2018/04/03/attention.html) • ࿦จͱ࣮૷͸྆ྠ • ʮμϝݩͰஶऀʹϝʔϧʯͯ͠ΈΑ͏ • ࣮ݧͷࡉ͔͍ઃఆʢe.g. ϋΠύϥʣΛ஌Γ͍ͨͱ͖ʹ • ʮ࿦จʹ͸ॻ͖͖Εͳ͍ࡉ͔͍ઃఆʯ͸ࢁఔ͋Δ… • લॲཧࡁΈͷϑΝΠϧ͕ཉ͍͠ͱ͖ʹ • ࣗ෼ͷ৔߹: Discourse TreebankͷϑΝΠϧΛ໯ͬͨΓ • (JU)VCͷ*TTVFΑΓ΋ϝʔϧͰ࿈བྷ͠Α͏ • ฦ৴͕དྷͳͯ͘΋ٽ͔ͳ͍ October 16, 2020 Inui Laboratory 50
  39. ݚڀࣨͷ͋Δ͖͔ͨฤ October 16, 2020 Inui Laboratory 51

  40. ੵۃతʹਓʹ૬ஊ͠Α͏ • 10෼೰ΜͰ෼͔Βͳ͔ͬͨΒਓʹ૬ஊ͠Α͏ • େ఍ͷ৔߹ɼ͙͢ʹղܾ͢Δ or ͱ͔͔ͬΓ͕௫ΊΔ • ʮxxxͬͯ࿦จʹؔ࿈͢Δ͔΋…ʯ •

    ʮੲͦͷݚڀͯͯ͠ɼyyyʹσʔλ͕͋Δ͔Β…ʯ • ʮzzzͬͯϝιου͕͋Δ͔ΒͦΕͰ…ʯ • ਓʹ࿩͓͔ͯ͠͠͞ʹؾ͕ͭ͘͜ͱ΋͋Δ • ࿩ͯ͠Δؒʹղܾ͢Δ͜ͱ΋͋Δ • ݚڀࣨͷਓ͸ΈΜͳڭ͕͑ͨΓɾॿ͚͕ͨΓͩͱ ࢥͬͯOK • ຊ౰ʹOK • ͜ΕҎ্ͳ͘OK • ʮݚڀࣨͷݞʹ৐Δʂʯͱ͍͏ڧ͍ؾ࣋ͪͰ October 16, 2020 Inui Laboratory 53 ٯʹݴ͏ͱ ·ͣ10෼͸ࣗ෼ Ͱߟ͑Α͏…
  41. ΍ͬͯ͸͍͚ͳ͍૬ஊ October 16, 2020 Inui Laboratory 54 Ͷͪ͜ΌΜ ݚڀࣨͷϑϨϯζ ͳΜ͔ಈ͖·ͤΜͰͨ͠ʂ

    Ͳ͏ͨ͠Β͍͍Ͱ͠ΐ͏͔ʁʁ ʢ஌ΒΜ͕ͳʜʣ
  42. ૬ஊࣄͷࡾ఺ηοτ  Τϥʔϝοηʔδશମ • ϓϩάϥϜͷు͖ग़ͨ͠ϩάΛશͯࡌͤΔ • লུ͠ͳ͍ • εΫγϣΑΓ΋ςΩετͰʂʢը૾Ͱ͸άάΕͳ͍ʣ 

    ΤϥʔΛ࠶ݱ͢Δ࠷খͷίʔυɾ࣮ߦྫ • ਺ඦߦͷίʔυ͸ಡΈͨ͘ͳ͍ • ࠷খʹ͢Δͷ͕೉͍͠ͱ͖΋͋Δ͕ɺ࠷௿ݶ͕Μ͹Δ • ࢼ͢ଆͱͯ͠͸ɺίϚϯυҰൃͰ࠶ݱͰ͖Δͱخ͍͠  ࢼͨ͠಺༰ • ʮxxxΛٙͬͯyyyΛ΍ͬͨΜͰ͕͢zzzͰͨ͠ʯ October 16, 2020 Inui Laboratory 55
  43. October 16, 2020 Inui Laboratory 56 ָ͍͠ࢼߦͰ ָ͍͠ݚڀࣨϥΠϑΛʂ

  44. ࢀߟจݙ • ࢼߦճ਺Λ૿΍͍ͨ͠ • ੒ޭͷํఔࣜ (https://www.jstage.jst.go.jp/article/jnlp/19/1/19_1/_pdf) • χϡʔϥϧωοτϫʔΫͷϑϨʔϜϫʔΫͷ࿩ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2016/seminar/seminar-20161013-takase.pdf) •

    ࣮૷ͷॱ൪ʹؾΛ͚ͭΑ͏ • Natural Language Processing with Deep Learning (Lecture 15) (https://web.stanford.edu/class/cs224n/lectures/lecture15.pdf) • σΟʔϓϥʔχϯάͷϑϨʔϜϫʔΫΛ࢖ͬͨࣗવݴޠॲཧݚڀͷ࣮૷ͷ஌ݟ (https://www.slideshare.net/HirokiTeranishi1/ss-79363691) • ػցֶशͰٽ͔ͳ͍ͨΊͷίʔυઃܭ (https://www.slideshare.net/takahirokubo7792/ss-65413290) • ػցֶशͰٽ͔ͳ͍ͨΊͷίʔυઃܭ 2018 (https://www.slideshare.net/takahirokubo7792/2018- 97367311) • Writing Code for NLP Research (https://docs.google.com/presentation/d/17NoJY2SnC2UMbVegaRCWA7Oca7UCZ3vHnMqBV4S Uayc/edit#slide=id.p) • ઌʹग़དྷΔ͜ͱ͸ઌʹऴΘΒͤΑ͏ • ݚڀऀྲྀίʔσΟϯάͷۃҙ (http://www.chokkan.org/publication/coding-for-researchers.pdf) • όάͷೖΓʹ͍͍͘ίʔυΛॻ͜͏ • ઓ͏ͨΊͷϓϩάϥϛϯά (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2015/seminar/seminar- 20160119-tianran.pdf) • ݡ͘αʔόͱϩʔΧϧΛߦ͖དྷ͠Α͏ • ݚڀͷ؅ཧ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2017/misc/misc-20171120-y- matsu.pdf) October 16, 2020 Inui Laboratory 57
  45. ࢀߟจݙ • σΟϨΫτϦߏ଄ʹؾΛ഑Ζ͏ • ݚڀऀྲྀίʔσΟϯάͷۃҙ (http://www.chokkan.org/publication/coding-for-researchers.pdf) • ݚڀͷ؅ཧ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2017/misc/misc-20171120-y-matsu.pdf) •

    A Research to Engineering Workflow | Dustin Tran (http://dustintran.com/blog/a-research-to-engineering-workflow) • ศརͳಓ۩Λ࢖͓͏ • EffectiveSSH (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2016/seminar/seminar-20161128-nakayama.html) • ΠϯλϥΫςΟϒϑΟϧλ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2016/seminar/seminar-20160715- yokoi.pdf) • Python Packages for Research (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2018/misc/misc-20180702- kiyono.pdf) • ࡞ۀ͕ര଎ʹͳΔλʔϛφϧͷγϣʔτΧοτΩʔ·ͱΊ | bacchi.me (https://bacchi.me/linux/terminal-tips/) • ৘ใऩूΛ͠Α͏ • ಡΉ͜ͱͱه࿥͢Δ͜ͱ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2015/seminar/seminar-20160125- sosuke.k2.pdf) • AHC-Lab M1ษڧձ ࿦จͷಡΈํɾॻ͖ํ (https://www.slideshare.net/ShinagawaSeitaro/ahclab-m1) • ݡ͘ίʔυΛ؅ཧ͠Α͏ • A Research to Engineering Workflow | Dustin Tran (http://dustintran.com/blog/a-research-to-engineering-workflow) • ੒ޭʢࣦഊʣΛ࠶ݱ͠Α͏ • ࣮ݧεΫϦϓτͷجຊ3఺ηοτ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2015/disco/disco-20160124- sosuke.k.html) • ࣮ݧͷࣄނΛ๷͍͗ͨ (http://www.cl.ecei.tohoku.ac.jp/local/handouts/2018/seminar/seminar-20180712-asano.pdf) October 16, 2020 Inui Laboratory 58