最先端NLP勉強会2023にて発表した"How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases"の発表資料です。https://aclanthology.org/2023.acl-long.629/
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases հऀɿେӋ ະ༔ʢ/"*45ลݚڀࣨ.ʣ ࠷ઌ/-1ษڧձ Aaron Mueller and Tal Linzen. ACL2023. 1 ऍͷͳ͍ݶΓਤදจ͔ΒͷҾ༻
ઢܗ൚Խͱ֊൚Խ • ਪྫ: Can you repeat what the senator next to the cats said? • ઢܗతͳҐஔใΛཔΓʹͨ͠൚Խ • Ծఆ ࠷ॳͷ໊ࢺ • 👎 Can you repeat what the senator next to the cats said? • Ծఆ ಈࢺͷۙ͘ʹ͋Δ໊ࢺ • 👎 Can you repeat what the senator next to the cats said? • ֊తͳߏΛཔΓʹͨ͠൚Խ • (Ծఆ
• 👍 Can you repeat what the senator next to the cats said? ౷ޠతؼೲόΠΞεͱ 7
౷ޠมλεΫ<8BSTUBEU
.VFMMFS
> ධՁํ๏ λεΫ࣭ม λεΫडಈԽ ࣄલֶश Your newt has observed the salamanders. Has your newt observed the salamanders? The raven observed the newts. The newts were observed by the raven. ඍௐ 15
౷ޠมλεΫ<8BSTUBEU
.VFMMFS
> ධՁํ๏ ࣄલֶश Your newt has observed the salamanders. Has your newt observed the salamanders? The raven observed the newts. The newts were observed by the raven. ඍௐ Your newt has observed the salamanders. 0 1 2 3 4 5 ֊ ઢܗ ֊ The raven observed the newts. 0 1 2 3 4 ઢܗ ࠷ॳͷॿಈࢺΛ લʹҠಈͤ͞Εྑͦ͞͏ λεΫ࣭ม ओઅͷಈࢺͷલͷॿಈࢺΛ લʹҠಈͤ͞Εྑͦ͞͏ λεΫडಈԽ ؼೲόΠΞε 16
౷ޠมλεΫ<8BSTUBEU
.VFMMFS
> ධՁํ๏ ࣄલֶश Your newt has observed the salamanders. Has your newt observed the salamanders? The raven observed the newts. The newts were observed by the raven. ඍௐ Your newt has observed the salamanders. 0 1 2 3 4 5 ֊ ઢܗ ֊ The raven observed the newts. 0 1 2 3 4 ઢܗ The quails that haven’t applauded some zebra have confused my yak. Have the quails that haven’t applauded some zebra confused my yak? The quails that haven’t applauded some zebra have confused my yak. Haven’t the quails that applauded some zebra have confused my yak? The salamander behind the ravens applauded the peacock. The peacock was applauded by the salamander behind the ravens. The salamander behind the ravens applauded the peacock. The ravens were applauded by the salamander. λεΫडಈԽ λεΫ࣭ม ؼೲόΠΞε ൚Խ ਪ࣌