Slide 82
Slide 82 text
܇࿅σʔλ
•ڭࢣͳ͠: ຊޠWikipedia, Wiki-40B, CC-100, BCCWJ
•ڭࢣ͋Γ: JSNLI, NU-NLI (SNLI, MNLI)
࣮ݧઃఆ:
•ࣄલֶशࡁΈݴޠϞσϧ21छྨͰ࣮ݧ (base: 14छྨ, large: 7छྨ)
•όοναΠζ: {64, 128, 256, 512}, ֶश: {1e-5, 3e-5, 5e-5}
•ҟͳΔཚγʔυͰ3ճ࣮ͣͭݧͯ͠࠷ྑͷϋΠύϥͰධՁ
ݱঢ়ͷ࣮ݧ݁Ռ
•ڭࢣ͋Γ/ͳ͠ڞʹૣҴాେRoBERTa-large͕࠷ߴੑೳ
•Studio Ousia ຊޠLUKE-largeͱXLM-RoBERTa-large͕͍࣍Ͱߴੑೳ
WIP: ຊޠSimCSEϞσϧͷߏங
82
ݱࡏ·Ͱʹ…
ڭࢣͳ͠: 1559ճ
ڭࢣ͋Γ: 3172ճ