Slide 13
Slide 13 text
13
ଛࣦؔ
l .VMUJ4JNJMBSJUZMPTT<8BOH
b>Λ༻
l ϖΞؒͷ૬ରతͳྨࣅΛߟྀ͠ɼ֤ϖΞͷྨࣅΛॏΈ͚
l ྨࣅͱͯ͠ɼප໊ͷจ຺ԽຒΊࠐΈදݱؒͷίαΠϯྨࣅΛ༻
ఏҊख๏ʛֶश
Similarity loss (Wang et al., 2019), which is a
metric-learning loss function that considers rela-
tive similarities between positive and negative pairs.
Let us denote the set of entities in the mini-batch
by B and the set of positive and negative samples
for the entity x0
i
2 B by Pi and Ni. We define the
cosine similarity of two entities x0
i
and x0
j
as Si,j,
resulting in a similarity matrix S 2 R|B| ⇥ |B|.
Based on Pi, Ni, and S, the following training
objectives are set:
LMS =
1
|B|
|B|
X
i=1
⇢
1
↵
log
⇥
1 +
X
k2Pi
e ↵(Sik )
⇤
+
1
log
⇥
1 +
X
k2Ni
e (Sik )
⇤
,
where ↵, are the temperature scales and is the
offset applied on S. For pair mining, we follow the
original paper (Wang et al., 2019).
Test Datasets & Evaluation Metric We evalu-
ated BIOCOM on three datasets for the biomedi-
cal entity normalization task: NCBI disease cor-
pus (NCBID) (Do˘
gan et al., 2014), BioCreative V
Chemical Disease Relation (BC5CDR) (Li et al.,
2016), and MedMentions (Mohan and Li, 2018).
Following previous studies (D’Souza and Ng, 2015;
Mondal et al., 2019), we used the accuracy as the
evaluation metric.
Given that BC5CDR and MedMentions contain
mentions whose concepts are not in MEDIC, these
were filtered out during the evaluation. We refer
to these as “BC5CDR-d” and “MedMentions-d”
respectively.
Model Details The contextual representation
for each entity x was obtained from PubMed-
BERT (Gu et al., 2020), which was trained
on a large number of PubMed abstracts using