09 API Interactions III.

Transcript

1. Anne Schuth (Blendle / University of Amsterdam, The Netherlands) Krisztian Balog (University of Stavanger, Norway) Tutorial at ECIR 2016 in Padua, Italy API Interactions III.
2. Outline for API interactions • Obtaining an API key and signing up for a site • Getting queries and candidate items • Generating and uploading rankings • Obtaining feedback and outcome
3. Feedback endpoint • GET /api/participant/feedback/(key)/ (qid)[/(runid)] • Returns feedback for this user on this query • runid is optional • qid can be “all”
4. Feedback endpoint { "feedback": [ {"qid": "S-q1", "runid": "baseline", "modified_time": "Sun, 27 Apr 2014 13:46:00 -0000", "doclist": [ { "clicked": false, "docid": "R-d89", "team": "participant" }, { "clicked": false, "docid": "R-d87", "team": "site" }, { "clicked": true, "docid": "R-d88", "team": "site" }, ...
5. Outcome endpoint • GET /api/participant/outcome/(key)/ (qid) • Returns outcome for this user on this query • Outcomes are aggregated per test period • qid can be “all”
6. Outcome endpoint { "outcomes": [ { "impressions": 661, "losses": 103, "outcome": 0.4690721649484536, "qid": "all", "site_id": "R", "test_period": { "end": "Sat, 16 May 2015 00:00:00 -0000", "name": "CLEF LL4IR Round #1", "start": "Fri, 01 May 2015 00:00:00 -0000" }, "ties": 467, "type": "test", "wins": 91 }, ... ] }
7. Exercise • Complete 03_feedback.py to obtain feedback and outcome for the queries from the run(s) you submitted in exercise #2
8. What to do with this feedback? • You figure it out… • (That might be your next paper)