⼈間のスコア 82.304 BERTのスコア 87.433 Jacob Devlin et al.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805 (2018) https://rajpurkar.github.io/SQuAD-explorer/
[Devlin(Google)+, NAACL’19, arxiv’18/11] 被引⽤数3871←1997 [CLS] my dog is cute [SEP] he likes play [MASK] [SEP] 2⽂⽬ 1⽂⽬ IsNext my dog is cute [SEP] he likes play piano [SEP] 分類 ⽳埋め 双⽅向
ALBERT [Lan(Google)+, ICLR’20, arxiv’19/09/26] 被引⽤数48←3 [CLS] my dog is cute [SEP] he likes play [MASK] [SEP] 2⽂⽬ 1⽂⽬ 正順? my dog is cute [SEP] he likes play piano [SEP] 分類 ⽳埋め