correctly the first time good annotation teams are small – and should collaborate with the data scientist lots of high-value opportunities need specialist knowledge and expertise Why annotation tools need to be efﬁcient
the best possible way with what we know just like multi-label classification where examples can have more than one right answer update towards: wrong labels get 0 probability, rest is split proportionally
be applied to other non-NER tasks: dependency parsing, coreference resolution, relation extraction, summarization etc. structures we’re predicting are highly correlated annotating it all at once is super inefficient – binary supervision can be much better
more ideas quickly. Most ideas don’t work – but some succeed wildly. ... fewer projects will fail. Figure out what works before trying to scale it up. ... you can build entirely custom solutions and nobody can lock you in.