਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ே૔୎ਓɺٶඌ༞հʢ౦େʣ 2024-03-12 @ NLP2024 ਆށ 1 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ਺ࣜάϥ΢ϯσΟϯά ਺ࣜάϥ΢ϯσΟϯάͷ 3 ཁૉ 1. ର৅จॻʹొ৔͢Δ਺ֶ֓೦ͷྻڍ cf. ఆٛநग़ 2. ਺ࣜτʔΫϯग़ݱʹର͢Δ਺ֶ֓೦ͷׂ౰ʢຊݚڀʣ 3. ਺ֶ֓೦ͱ֎෦஌ࣝͷඥ෇͚ cf. MathIR The result of running the machine learning algorithm can be expressed as a function y(x) which takes a new digit image x as input and that generates an output vector y, encoded in the same way as the target vectors. The precise form of the function y(x) is determined during the training phase. (p. 2, PRML) Math concepts • function y(·) • output vector y 
਺ֶ֓೦ͷྻڍ 
֓೦ͷׂ౰ External knowledge Concept 1 Concept 2 
֎෦஌ࣝͱͷඥ෇͚ 2 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ຊݚڀͷର৅ɿ਺ࣜࣝผࢠͷจॻ಺ᐆດੑ The result of running the machine learning algorithm can be expressed as a function y(x) which takes a new digit image x as input and that generates an output vector y, encoded in the same way as the target vectors. The precise form of the function y(x) is determined during the training phase. (p. 2, PRML) Math concepts • function y(·) • output vector y 1 
਺ֶ֓೦ͷྻڍ 
֓೦ͷׂ౰ External knowledge Concept 1 Concept 2 
֎෦஌ࣝͱͷඥ෇͚ ▶ P2C ม׵΍ MathIR ͳͲͷԠ༻λεΫͰোนͱೝࣝ ▶ طଘݚڀͰΧόʔͰ͖ͳ͍ ∵) σʔλࢿݯ͕ෆ଍ ˠ ຊݚڀͰ͸σʔλࢿݯͷߏங͔ΒऔΓ૊Ή 3 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ຊݚڀͷ໨త ର৅ൣғ ▶ ղܾ͢Δ໰୊ɿ਺ࣜτʔΫϯͷจॻ಺ᐆດੑ ▶ ᐆດੑղফͷର৅ɿ਺ࣜࣝผࢠʢ࠷සग़ͷτʔΫϯछʣ ϦαʔνɾΫΤενϣϯ ▶ จॻ಺ᐆດੑͷղফʹॏཁͳ৘ใछ͸ͳʹ͔ʁ ▶ ༗ޮͳ৘ใछ͸ɺର৅࿦จͷ෼໺ʹґଘ͢Δ͔ʁ 4 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ λεΫ֓ཁ ೖྗ ▶ ߏ଄ԽจॻσʔλʢXHTMLʣ ▶ ֤਺ֶ֓೦ʹඥ෇͘࠷ॳͷग़ݱҐஔ ˠ ਖ਼ղϥϕϧͷ໿ 10% ʹ૬౰ ग़ྗ ▶ ֤ग़ݱʹඥ෇͘਺ֶ֓೦ ˠ ࢒Γ 90% ͷϥϕϧΛ౰͍ͯͨ λεΫͷ೉қ౓ ▶ Cascade ϕʔεϥΠϯ ʹείʔϓ੾ସ͸ॳग़ ҐஔͷΈͱԾఆ ˠ Kappa 0.6431 ▶ ਓؒΞϊςʔλ ˠ Kappa 0.7939 random baseline mode baseline cascade baseline our model human 0.0 0.2 0.4 0.6 0.8 1.0 Score 0.5894 0.6329 0.8312 0.8534 0.9515 0.0099 0.0000 0.6431 0.7330 0.7939 random baseline mode baseline cascade baseline our model human Accuracy Kappa 5 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ิ଍ɿCascadeϕʔεϥΠϯ ▶ લఏɿຊλεΫͰ͸֤֓೦ʹඥ෇͘ॳग़ग़ݱ͸ط஌ ▶ ͋Δ֓೦ʢ֓೦ 1 ͱ͢Δʣͷॳग़͔Β࣍ͷ֓೦ʢ֓೦ 2ʣ ͷॳग़Ґஔ·Ͱͷ͢΂ͯͷग़ݱΛ֓೦ 1 ʹׂΓ౰ͯΔ ˠ είʔϓΛਤࣔ͢Δͱ֊ஈঢ়ʹͳΔ Ґஔ ֓೦ ֓೦ ֓೦ ʜ ࣮ࡍ͸͜͏ͨ͠ग़ݱ͕ଘࡏ 6 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ σʔληοτ ग़൛ LREC 2022 +Ћ ▶ ࿦จͷ෼໺ʹΑͬͯσʔληοτΛ෼ׂ ▶ NLP αϒηοτɿϞσϧͷ։ൃͱධՁʹར༻ ▶ ͦͷଞαϒηοτɿධՁͷΈʹར༻ ఱจֶ ×8ɺCS ×5ɺܦࡁֶ ×3ɺ਺ֶ ×2ɺ ੜ෺ֶ ×1ɺ෺ཧֶ ×1 ▶ ܇࿅/ݕূ/ධՁσʔλ͸࿦จ୯ҐͰ෼ׂ ▶ ධՁσʔλ͸࠷ॳʹִ཭ʢNLP ͷ 4 ຊɺͦͷଞͷશ෦ʣ ▶ ։ൃσʔλ͸ LOOCV Ͱ༗ޮ׆༻ σʔληοτͷن໛ αϒηοτ ࿦จ ຊจͷ୯ޠ ࣝผࢠͷछྨ ग़ݱ ࣙॻ߲໨ NLP 20 97,045 789 9,278 1,518 ͦͷଞ 20 140,017 953 18,377 2,085 ߹ܭ 40 237,062 1,742 27,655 3,603 7 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ଟ૚ύʔηϓτϩϯϞσϧ ▶ લఏɿϥϕϧηοτʢީิ֓೦ʣ͸จॻ͝ͱʹҟͳΔ ˠ ୯७ͳଟΫϥε෼ྨ໰୊Ͱ͸ͳ͍ ▶ ଟ૚ύʔηϓτϩϯͷ࢖͍ํ͕΍΍ಛघ 1. ग़ݱ͝ͱʹʢग़ݱɺީิ֓೦ʣͷϖΞΛ࡞Δ 2. ϖΞ͕ਖ਼ղͰ͋Δ֬཰Λग़ྗ͢ΔΑ͏ʹֶश ˠ ະ஌ͷϥϕϧηοτʹ΋ରԠՄ 8 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ಛ௃ྔΤϯδχΞϦϯά c: จ຺ ▶ ग़ݱʢΛؚΉ਺ࣜʣલޙͷςΩετ ྫ ‌ feature vector $v’ {x}$ extracted from ‌ ▶ Sentence Transformer [Reimers+, 2019] ͰϕΫτϧԽ MiniLM ͕࠷΋༗ޮɺwindow size ΍਺ࣜදݱ͸Өڹখ a: ઀ࣙλΠϓ ▶ ग़ݱपลͷϩʔΧϧͳ਺ࣜߏ଄ ྫ ఴࣈͷ༗ແ ▶ ϧʔϧϕʔεͰਪఆ ˠ ਫ਼౓ 90.56% p: Ґஔσʔλ ▶ Cascade ޮՌͷ༗ແͱॳग़Ґஔ͔Βͷ૬ରڑ཭ ˠ ୯ಠͰ΋ڧ͍ɻCascade ϕʔεϥΠϯͱҰக 9 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ Ϟσϧൺֱᶃ ֓ཁ ▶ 3 ͭͷಛ௃Λ૊Έ߹Θͤͯ MLP ΛֶशɺධՁ ▶ Ϟσϧͷछྨ͸ 23 − 1 = 7 ݸ ▶ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλͷ༗ແͰදݱ ྫ c+ / a+ / p−: จ຺ͱ઀ࣙλΠϓΛ࢖༻ͨ͠Ϟσϧ 10 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ Ϟσϧൺֱᶄ ֤ಛ௃ྔͷ୯ಠ࢖༻ ‣ ͍ͣΕ΋
SBOEPNNPEF
ϕʔεϥΠϯ௒͑
‣ $BTDBEF
ϕʔεϥΠϯ
˺
Q
୯ಠ
‣ ༗ޮੑ͸
Q

B

D c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 11 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ Ϟσϧൺֱᶅ ಛ௃ྔͷ૊Έ߹Θͤ ‣ ෳ਺ͷಛ௃ྔΛ૊Έ߹ΘͤΔͱੑೳ͸޲্
‣ ಛʹɺB

Q
͸࠷ߴੑೳΛୡ੒ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 12 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ Ϟσϧൺֱᶆ Ґஔσʔλͱจ຺ ‣ Ґஔσʔλʹจ຺Λ଍ͯ͠΋ɺ΄΅ޮՌͳ͠ 
 ˠ
จ຺ͷ΋ͭ৘ใ͸ɺҐஔσʔλ͕แؚ͔ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 13 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ Ϟσϧൺֱᶇ ͢΂ͯͷಛ௃ྔ ‣ ͭͷಛ௃͢΂ͯΛ༻͍ͯ΋
B

Q
Λ௒͑ͳ͍
‣ ޡࠩͷൣғɻΉ͠Ζจ຺͕ϊΠζʹͳ͔ͬͨ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 14 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ෼໺ґଘੑᶃ ֓ཁ ໨త ༗ޮͳಛ௃ྔ͕෼໺ґଘ͔Ͳ͏͔ͷ֬ೝ ํ๏ NLP σʔλͰֶशɺଞ෼໺σʔλͰධՁ ఱจֶ ×2ɺCS ×2ɺܦࡁֶ ×3ɺ਺ֶ ×2ɺੜ෺ֶ ×1ɺ෺ཧֶ ×1 15 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ෼໺ґଘੑᶄ ֤ಛ௃ྔͷ୯ಠ࢖༻ ‣ Χού஋Ͱൺֱ͢ΔͱΑ͍
‣ ༗ޮੑ͸΍͸Γ
Q

B

D c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 16 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ෼໺ґଘੑᶅ ಛ௃ྔͷ૊Έ߹Θͤ ‣ ૊Έ߹Θͤͨ৔߹ͷ܏޲΋ෆม
‣ ޡࠩϨϕϧ͕ͩ
B

Q
͕࠷ྑ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 17 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ෼໺ґଘੑᶆ Ґஔσʔλͱจ຺ ‣ ΍͸ΓҐஔσʔλʹจ຺Λ଍ͯ͠΋ޮՌͳ͠ c: จ຺ɺa: ઀ࣙλΠϓɺp: Ґஔσʔλ 18 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ֓೦ׂ౰ͷࣗಈԽɿ·ͱΊ ϙΠϯτ ▶ ਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফΛ໨ࢦͨ͠ ▶ ଟ૚ύʔηϓτϩϯΛ܇࿅͠ɺॏཁͳ৘ใछΛಛఆ ▶ ։ൃͨ͠Ϟσϧͷੑೳ͸ϕʔεϥΠϯҎ্ਓؒະຬ ໌Β͔ʹͨ͜͠ͱ ▶ จॻ಺ᐆດੑͷղফʹॏཁͳ৘ใछ͸ͳʹ͔ʁ ˠ Ґஔσʔλͱ઀ࣙλΠϓ͕ॏཁ ▶ ༗ޮͳ৘ใछ͸ɺର৅࿦จͷ෼໺ʹґଘ͢Δ͔ʁ ˠ ্هͷ܏޲͸෼໺ʹґଘ͠ͳ͍ 19 / 20 

਺ࣜࣝผࢠͷจॻ಺ᐆດੑͷղফ ࢀߟจݙ ▶ Takuto Asakura, et al. “Towards Grounding of Formulae.” In Proceedings of SDP 2020. ▶ Takuto Asakura, et al. “MioGatto: A Math Identifier-oriented Grounding Annotation Tool.” In Proceedings of MathUI 2021. ▶ Takuto Asakura, et al. “Building Dataset for Grounding of Formulae — Annotating Coreference Relations Among Math Identifiers.” In Proceedings of LREC 2022. ▶ Ron Ausbrooks, et al. “Mathematical Markup Language (MathML) 3.0 Specification.” World Wide Web Consortium (W3C) Recommendation, (2014). ▶ Christopher M. Bishop. Pattern Recognition and Machine Learning (2006). ▶ Viet Lai et al. “SemEval 2022 Task 12: Symlink—Linking Mathematical Symbols to their Descriptions.” In Proceedings of SemEval-2022. ▶ Nils Reimers and Iryna Gurevych. “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.” In Proceedings of EMNLP2019. 20 / 20