Google_QA__Solution___Review__1_.pdf

Transcript

1. Google QA Solution & Review
2. Agenda ・Model Architecture ・Post Processing ・Augmentation ・Other Tips & Trials
3. Model Architecture ・3 models in 1 model ・MLE Layer ・Ensemble of Bert/XLNet
4. Post Process ・Threshold Optimization ・Test N for range(200) & Select Best CV
5. Augmentation ・Back Translation ・TextBlob
6. +1. Flex Module ・Change MaxLen for Token selection Max Len == Max Len1 + Max Len2 == 512 - 3
7. +1. MLE Module ・Almost Same ・Ensemble is also well VS
8. +α. Bert Pretraining (MLM) ・Training on External Dataset
9. +α. Distillation ・Pseudo label for External Dataset
10. +α. Other Models ・Bert-Large, XLNet-Large ・RoBerta, RoBerta-Large ・ALBert, ALBert-Large ・GPT2, XLM, etc...
11. +α. Pseudo Labeling ・1 Opinion about PSeudo Labeling
12. +α. Draco PP V2 ・For 12 columns which (Max - Min) < 1 ・Invert when evaluate
13. +α. Post Process+1 ・for N in range(200) Optimize R ・Scipy, NN, etc...
14. Thanks !!