A Summary of ECIR'18

Highlights of the 40th European Conference on Information Retrieval (ECIR '18)
Date: April 6, 2018
Venue: Stavanger, Norway. Symposium at the IAI group

Please cite, link to or credit this presentation when using it or part of it in your work.

#InformationRetrieval #IR

Transcript

1. A summary of ECIR'18 Darío Garigliotti April 6, 2018
2. A word cloud • Frequent terms: • Reproducibility • Interpretability • Accountability • Explainability
3. Points from the main panel Panel on predicting/explaining performance • What is performance? For whom? • Users like explainability, determinism... • Users: minimum explanation and "give me the results in SERP"; • Engineers: happy to deliver a non-black-box = detailed description of all decisions and settings • Usual vs novice user
4. Points from the main panel • Users hate failure, IR strong in failure analysis • Researcher intuitions not always explicitly reported (e.g., Carsten's paper) • Statistical vs practical significance • IR problems become complex, so the metrics involved (and their optimization) • It might need a breakthrough in UI, leading to new metrics • Different risks in academia vs industry • E.g., failure prediction ASAP
5. Points from the main panel • Over-optimizations, e.g., on user groups • current -e.g., Salesforce-; • future per-person customization? -and Edgar' keynote-) • Overall moving to a ML paradigm? Metamodels (i.e., not IR retrieval models, but models for learning models)? • Model the whole info need rather than single queries • No transparency => no ability for users to realize her complex task • Explainability for diversity and bias in complex needs vs how the system decided these results • IR goal: get info need => so maybe rather than guess, ask (CAIR)
6. Industry day panel • Again, explainability of, e.g., end-2-end DL models • So what is need for reproducibility? • Example: CAIR? community: surviving without problem definition, metric, or dataset • If we aim reproducibility, we can't stop working on developing datasets (vs decreasing/missing availability of large datasets from companies)
7. Gabriella Kazai keynote - Evaluation • History • INEX evaluation pipeline (Experts; altruism) • Offline metrics and crowdsourcing (Crowds; $) • Online metrics (Users; self-interest) • Crowdsourcing for offline evaluation • Potential: to access large source of expertise • Current: challenges in costs, time, quality • Factors: payment, task - guidelines, judge population...
8. Gabriella Kazai keynote - Evaluation • Future of IR • Pervasive • Personal • Conversational-oriented
9. Some papers • Reproducibility + Best paper awardee: • Gianmaria Silvello, Maristella Agosti, Riccardo Bucco, Giulio Busato, Giacomo Fornari, Andrea Langeli, Alberto Purpura, Giacomo Rocco and Alessandro Tezza. Statistical Stemmers: A Reproducibility Study • Reproducibility of NN approaches • Alexander Dür, Andreas Rauber and Peter Filzmoser. Reproducing a Neural Question Answering Architecture applied to the SQuAD Benchmark Dataset: Challenges and Lessons Learned • Hybrid embedding (+ related? to QA) • Daniel Cohen and W Bruce Croft. A Hybrid Embedding Approach to Noisy Answer Passage Retrieval
10. Some papers • Entities in topic modeling • Andreas Spitz and Michael Gertz. Entity-centric Topic Extraction and Exploration: A Network-based Approach • Original snippets • Martin Potthast, Wei-Fan Chen, Matthias Hagen and Benno Stein. A Plan for Ancillary Copyright: Original Snippets • Music recommendation • Kartik Gupta, Noveen Sachdeva and Vikram Pudi. Explicit Modelling of the Implicit Short Term User Preferences for Music Recommendation • Multilinguality? • Georgios Balikas, Charlotte Laclau, Ievgen Redko and Massih-Reza Amini. Cross-lingual Document Retrieval using Regularized Wasserstein Distance