and baselines Goal: What makes a good paper? Refresher on baselines: https://www.quora.com/What-does- baseline-mean-in-machine-learning Pick simple metrics and baselines Minimally required metric targets?
set size and content similar? ✓Entire process described? ✓Pre-processing steps described completely? ✓Well-known methods? ✓Or: Complete descriptions of the methods? 3. Results ✔ Methodology ✔ Abstract & Introduction
✓German corpus ? Research documents rather than banking documents ✓Entire process described? ✓Seems to be complete ✓Pre-processing steps described completely? ✓Image conversion and scaling is described ? OCR tool / approach is not mentioned ✓Well-known methods? ✓Neural networks with descriptions of the conﬁguration
metrics? ✓Metrics appropriate for the problem and dataset? ✓Better than your baseline and metric targets? ✓Any published review of the results? ✓Improvements analyzed with suitable statistical tests? ✔ Results ✔ Methodology ✔ Abstract & Introduction
metric for classiﬁcation XMetrics appropriate for the dataset? Not suitable for the given imbalanced classes ✓Better than your baseline? Yes, by 0.25 over the baseline ? Better than the metrics target? They are close ? Any published review of the results? Not yet XImprovement analyzed with suitable statistical tests? No statistical analysis, and reported measurements are not comparable
research ﬁndings 2. Decide on your comparison criteria 3. Evaluate quality, relevance and reproducibility 4. Prioritize your chosen approaches 5. Prototype the best approaches Slides will be tweetet from @ellen_koenig
Kozin •Network icon by Gregor Cresner •Problem solving icon and razor blade icon by Vector Market •Bank icon by Stock image photo 2. Slide 6 •Document icon afredocreates.com/icons and ﬂaticons.com for thenounproject.com •Bar chart icon: pixabay.com •Touch icon by Jasfart for thenounproject.com •Target icon by Libby Ventura for thenounproject.com