Paper introduction: SES Lab’s Journal Club Calendar June 19, 2025 D1 Takafumi Horie Kyoto University, Symbol Emergence Systems Laboratory Metaphor Generation with Conceptual Mappings (ACL 2021) 

2 [Stowe+ 2021] Stowe, Kevin, et al. "Metaphor Generation with Conceptual Mappings." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021. †The authors’ affiliations are as of the time of publication. “Metaphor Generation with Conceptual Mappings” (ACL 2021)  link: https://aclanthology.org/2021.acl-long.524/  Authors: Dr. Kevin Stowe Technical University of Darmstadt† Dr. Tuhin Chakrabarty Columbia University † Dr. Nanyun Peng University of California Los Angeles † Dr. Smaranda Muresan Columbia University † Prof. Iryna Gurevych Technical University of Darmstadt† 

 Paper introduction  Introduction  Proposed method 1: CM-Lex  Proposed method 2: CM-BART  Experiments  Results Index 3 

 Metaphor generation:  Controlled metaphor generation offers significant advantages  Ensuring that metaphors align with the surrounding text is essential for natural understanding in context  This study specifically focuses on domains  This study aims to emphasize the mapping between the target domain (the conceptual area being described) and the source domain (the conceptual area used metaphorically) Background 4 

5 [Stowe+ 2020] Stowe, Kevin, Leonardo Ribeiro, and Iryna Gurevych. "Metaphoric paraphrase generation." arXiv preprint arXiv:2002.12854 (2020). [Chakrabarty+ 2021] Chakrabarty, Tuhin, et al. "MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding." Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. [Lewis 2020+] Lewis, Mike, et al. "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension." Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. Related works on metaphor generation: MetMask [Stowe+ 2020] MERMAID [Chakrabarty+ 2021] A Transformer-based Seq2Seq with masking metaphorical expressions BART [Lewis+ 2019] is fine-tuned by using pair of literal sentences and metaphorical sentences ➡ Control over metaphor generation with mappings between conceptual domains has not yet been achieved 

 Frames: the conceptual structure or context in which words are used  By using a lexical resource that includes frame information, mappings between the target domain and source domain can be realized ➡ This study proposes CM-Lex & CM-BART;  Two metaphor generation methods based on conceptual mappings  This study focuses specifically on verbs and achieves metaphor generation through verb substitution To generate metaphors by explicitly specifying the frames Research Goal 6 

 FrameNet [Baker+ 1998]:  A manually annotated dataset that includes frames, the words belonging to each frame, and the relationships between frames  This study uses FrameNet to determine the frame to which a verb belongs and to perform mappings between frames [Baker+ 1998] Baker, Collin F., Charles J. Fillmore, and John B. Lowe. "The berkeley framenet project.” COLING 1998 Volume 1: The 17th International Conference on Computational Linguistics. 1998. Preliminary 7 

① CM-Lex (Unsupervised)  A shared embedding space for words & FrameNet tags is learned based on Word2Vec [Mikolov 2013+]  Vector arithmetic is then used to replace target domain words with their source domain counterparts ② CM-BART (Semi-supervised)  Pairs of literal and metaphorical sentences are constructed from a poetry dataset with FrameNet Tag parser  BART [Lewis 2020+] is fine-tuned using these sentence pairs [Mikolov 2013+] Mikolov, Tomas, et al. "Efficient estimation of word representations in vector space." arXiv preprint arXiv:1301.3781 (2013). [Lewis 2020+] Lewis, Mike, et al. "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension." Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. Proposed Methods 8 

1. Construct corpus  Extraction of 1.8M Context around verbs (Verb instances)  From the full texts, a context window centered on each verb was extracted, consisting of five words around that verb  Used corpus:  Sentences annotated with FrameNet tags [Swayamdipta+ 2017].  Datasets tagged with FrameNet tag parser [Swayamdipta+ 2017]: Gutenberg Poetry Corpus [Francis+ 1979]，Brown Corpus [Jacobs 2018+]， randomly selected sentences from Wikipedia [Swayamdipta+ 2017] Swayamdipta, Swabha, et al. "Frame-semantic parsing with softmax-margin segmental rnns and a syntactic scaffold.“ arXiv preprint arXiv:1706.09528 (2017). [Francics+ 1979] W. N. Francis and H. Kucera. Brown corpusmanual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island,US. 1979. [Jacobs 2018] Jacobs, Arthur M. "The gutenberg english poetry corpus: exemplary quantitative narrative analyses.“ Frontiers in Digital Humanities 5:5. 2018. Proposed method①｜CM-Lex 9 

10 [Havens+ 2019] Havens, Sam, and Aneta Stal. "Use bert to fill in the blanks." (Computer software.) URL:https://github.com/Qordobacode/fitbert (2019). 2. Embedding FrameNet tags in the same space as words  Each verb in the “verb instances” is replaced with its FrameNet tag  Original & replaced instances embedded with a 50-dimensional word2vec skip-gram model 3. Metaphor generation  Replace the verbs in the text with those in the target frame through vector operations  This is similar to operations like “Queen = King – man + women”  Verbs are delemmatized (converted into the appropriate inflected form) by fitbert [Havens+ 2019] Die End ① ② ② − ① The party [ended] as soon … ↓ The party as soon … 

 Creating parallel data  Detect metaphorical verbs in Gutenberg Poetry Corpus [Francis+ 1979] by metaphor classifier  Quality filtering is carried out with a knowledge inference model  FrameNet tag parser [Swayamdipta+ 2017] is used to tag the verb frames in both the literal and metaphorical sentences [Francics+ 1979] W. N. Francis and H. Kucera. Brown corpusmanual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island,US. 1979. [Swayamdipta+ 2017] Swayamdipta, Swabha, et al. "Frame-semantic parsing with softmax-margin segmental rnns and a syntactic scaffold.“ arXiv preprint arXiv:1706.09528 (2017). Proposed Method②｜CM-BART 11 

 Fine-tune BART  Input: A sentence specifying the target verb, its original frame (target frame), and the intended source frame for conversion  ex)  Output: A metaphorical sentence  ex) “The party died.” ➡ This enables metaphor generation with explicit frame specification using BART Proposed Method②｜CM-BART 12 

 Purpose:  Evaluate whether the proposed methods generate metaphor with mapping between source & target domain  Evaluate whether the proposed methods can generate novel metaphors for unknown source domain  Tasks:  Converting literal sentences into metaphorical expressions  Comparison between human-generated metaphors (Gold) and generated metaphors  Human evaluation of novel metaphorical expressions Experiments｜Settings 13 

14 [Francics+ 1979] W. N. Francis and H. Kucera. Brown corpusmanual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island,US. 1979. [Mohammad+ 2016] Mohammad, Saif, Ekaterina Shutova, and Peter Turney. "Metaphor as a medium for emotion: An empirical study." Proceedings of the fifth joint conference on lexical and computational semantics. 2016. [Jacobs 2018] Jacobs, Arthur M. "The gutenberg english poetry corpus: exemplary quantitative narrative analyses.“ Frontiers in Digital Humanities 5:5. 2018. [Chakrabarty+ 2021] Chakrabarty, Tuhin, et al. "MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding." Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. [Stowe+ 2020] Stowe, Kevin, Leonardo Ribeiro, and Iryna Gurevych. "Metaphoric paraphrase generation." arXiv preprint arXiv:2002.12854 (2020).  Datasets is built based on 3 datasets:  Brown corpus [Francis+ 1979]  Gutenberg Poetry corpus [Jacobs 2018]  Mohammad 2016 [Mohammad+ 2016]  Compared methods  Compare proposed methods with two methods that do not apply control during generation:  MetMask [Stowe+ 2020]  MERMAID [Chakrabarty+ 2021] 

15 [Reimers+ 2019] Reimers, Nils, and Iryna Gurevych. "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks." Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.  Evaluation with embeddings by SBERT [Reimers+ 2019]  Cosine distance from Gold Metaphor (dis↓) dis = 1 − cos(𝑀𝑀, 𝐺𝐺)  Relational distance (rel↓) rel = |cos(L, M) − cos(𝑀𝑀, 𝐺𝐺)|  Distance between input–output similarity and the input–gold closeness  Human Evaluation  Metaphoricity (Met↑):  Is the metaphor novel and interesting?  Source Domain Evocation (Src↑):  Does the metaphor reflect designated source frame?  Both of Met & Src is evaluated by 5-point scale (0, 1, 2, 3, 4) Variable Definition 𝐿𝐿 Sentence embeddings of the literal input 𝑀𝑀 Sentence embeddings of the gold metaphor 𝐺𝐺 Sentence embeddings of the generated output 

In evaluations using sentence embeddings, CM-BART achieved the highest scores Experiments｜Result 16  CM-BART demonstrated top performance across metrics  CM-Lex, despite being unsupervised, achieved performance comparable to neural-based baseline models  As an unsupervised model, CM-Lex generates a diverse range of expressions, which likely resulted in a lower generation rate of expressions identical to those in the Gold references (%=) Mean … the mean of dis & rel, %= … the match rate with Gold 

CM-BART also outperformed in human evaluations Experiments｜Result 17  Test datas:  Gold: When human-annotated mappings exists  Rare: Generates metaphors using randomly selected mappings with median-level frequency  Unseen: Generates metaphors based on previously unseen mappings  CM-BART received high evaluations for both Met and Src, with particularly stable performance on unseen data  CM-Lex showed higher scores for Src than compared methods 

18 Qualitative evaluation  CM-Lex sometimes generates unintelligible sentences  CM-BART is more robust and fluent  MetMask and MERMAID often produce metaphors that make it difficult to recall the original domain 

19 Qualitative evaluation｜Rare/Unseen Source  CM-BART outperforms CM-Lex  CM-Lex sometimes generates incomprehensible sentences  When the target are far different from source, sentences generated by CM-BART doesn’t align with the original text while it is fluent 

 Importance of Utilizing Lexical Resources  While FrameNet frames are neither perfect nor specifically designed to capture metaphorical meanings, they provide a strong signal indicating the domain to be generated  Comparison between CM-Lex and CM-BART  CM-Lex likely generated unintelligible sentences due to its inability to capture contextual information  Directions for Extension  We aim to handle not only verbs but also nouns, and to address metaphors within long-range contexts Experiments｜Discussion 20 

 Purpose:  To generate metaphors by explicitly specifying the frames  Proposed method:  CM-Lex: based on Word2Vec & Vector arithmetic  CM-BART: based on fine-tuning of BART  Experiments result:  CM-BART outperformed other methods  CM-Lex achieved performance comparable to neural-based baseline models, even it is an unsupervised model Conclusion 21 

22 

Appendix 23 

 Datasets:  Brown corpus [Francis+ 1979]: standard fiction texts, so the metaphors tend to be conventional  Gutenberg Poetry corpus [Jacobs 2018]: consistent, novel metaphors, but often unconventional syntactic construction  Mohammad 2016 [Mohammad+ 2016]: relatively basic syntactic patterns [Francics+ 1979] W. N. Francis and H. Kucera. Brown corpusmanual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island,US. 1979. [Mohammad+ 2016] Mohammad, Saif, Ekaterina Shutova, and Peter Turney. "Metaphor as a medium for emotion: An empirical study." Proceedings of the fifth joint conference on lexical and computational semantics. 2016. [Jacobs 2018] Jacobs, Arthur M. "The gutenberg english poetry corpus: exemplary quantitative narrative analyses.“ Frontiers in Digital Humanities 5:5. 2018. Experiments｜Detail of Datasets 24