DAT630 - Entity Retrieval II.

University of Stavanger, DAT630, 2016 Autumn

Transcript

1. DAT630
 Entity Retrieval II. Krisztian Balog | University of Stavanger 02/11/2016
2. Recap - Entities are meaningful units for organizing information - Used for enriching in search engine results - Knowledge bases store massive amounts of information about entities as RDF triples - Entities can be represented as documents for retrieval - Using document fields can preserve (some of) the underlying structure
3. So far… - Term-based retrieval models - Robust and effective, but ignore semantics - entity-specific properties (types, relationships, etc.) Text-only representation info need entity matching Abc Abc Abc
4. Incorporating semantics - working definition:
 semantics = references to meaningful structures - How to capture, represent, and use structure? - It concerns all components of the retrieval process! Text-only representation info need entity matching Abc Abc Abc Text+structure representation info need entity matching Abc Abc Abc
5. Spectrum of queries I need a list of female computer scientists who work on semantic search. human understanding machine understanding natural language keyword keyword++ structured language female computer scientists semantic search female semantic search
 <profession: computer scientist> SELECT ?p WHERE { ?p has-profession Computer_Scientist . ?p has-gender Female . ?p occurs-with "semantic search"}
6. Spectrum of queries I need a list of female computer scientists who work on semantic search. human understanding machine understanding natural language keyword keyword++ structured language female computer scientists semantic search female semantic search
 <profession: computer scientist> SELECT ?p WHERE { ?p has-profession Computer_Scientist . ?p has-gender Female . ?p occurs-with "semantic search"}
7. Scenario #1 - User provides keyword++ query search UI retrieval method entity search results suggestions, facets, etc.
8. Example keyword++ queries
10. Scenario #2 - Query understanding component constructs the keyword++ query (automatically) keyword query retrieval method entity search results keyword++ query query understanding
11. Entity Types
12. Interacting with types
 grouping results people companies jobs (more) people
13. Interacting with types
 filtering results
14. Interacting with types
 filtering results
15. Target type(s) are provided
 faceted search, form fill-in, etc.
16. Type-aware ranking query entity Olympic games target types Rio de Janeiro term-based similarity type-based similarity … … entity types
17. Challenges - Users are not familiar with the type system
18. Very many types…
 which are typically hierarchically organized
19. Sense of scale
20. In general, categorizing things can be hard - What is King Arthur? - Person / Royalty / British royalty - Person / Military person - Person / Fictional character
21. Which King Arthur?!
22. Considerations for type-aware ranking - Need to be able to handle the imperfections of the type system - Inconsistencies - Missing assignments - Granularity issues - Entities labeled with too general or too specific types - User input is to be treated as a hint, not as a strict filter
23. Type-aware retrieval #1 - Strict filtering model Type-based
 similarity Term-based
 similarity w stands for word 1 if the query and entity have some types in common, otherwise 0 P(q|e) = P(qw |e) · [types(q) \ types(e) 6= ;]
24. Type-aware retrieval #2 - Soft filtering model Type-based
 similarity Term-based
 similarity w stands for word Compare the type distribution of in the query against that of the entity Query types Entity types P(q|e) = P(qw |e) · P(qt |e)
25. Type-aware retrieval #3 - Interpolation model P(q|e) = (1 )P(qw |e) + P(qt |e) Type-based
 similarity Term-based
 similarity w stands for word Compare the type distribution of in the query against that of the entity Query types Entity types
26. Entity Relationships
27. Related entities
30. Searching for arbitrary relations* airlines that currently use Boeing 747 planes
 ORG Boeing 747 Members of The Beaux Arts Trio
 PER The Beaux Arts Trio What countries does Eurail operate in?
 LOC Eurail *given an input entity and target type
31. A typical pipeline Input (entity, target type, relation)
 Ranked list 
 of entities Candidate 
 entities Retrieving docs/snippets Query expansion ... Type filtering Deduplication Exploiting lists ...
32. Modeling related entity finding
    - Ranking entities of a given type (T) that stand in a required relation (R) with an input entity (E) - Three-component model p(e|E, T, R) / p(e|E) · p(T|e) · p(R|E, e) Context model Type filtering Co-occurrence model xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx
33. Ranking Entities without Ready-made Descriptions
34. Scenario - Entity descriptions are not readily available - Entity occurrences are annotated - manually - automatically (i.e., entity linking)
35. The basic idea
    Use documents to go from queries to entities Query-document association the document’s relevance Document-entity association how well the document characterises the entity e q xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx
36. Two principal approaches - Profile-based methods - Create a textual profile for entities, then rank them (by adapting document retrieval techniques) - Document-based methods - Indirect representation based on mentions identified in documents - First ranking documents (or snippets) and then aggregating evidence for associated entities
37. Profile-based methods q xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx e xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx e e
38. Document-based methods q xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx xx x xxxx x xxx xx xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx X e X X e e
39. Many possibilities in terms of modeling - Generative (probabilistic) models - Discriminative (probabilistic) models - Voting models - Graph-based models
40. Candidate models (“Model 1”)
    P(q|✓e ) = Y t2q P(t|✓e )n(t,q) Smoothing
 With collection-wide background model (1 )P(t|e) + P(t) X d P(t|d, e)P(d|e) Document-entity association Term-candidate 
 co-occurrence In a particular document. In the simplest case: P(t|d)
41. Document models (“Model 2”)
    P(q|e) = X d P(q|d, e)P(d|e) Document-entity association Document relevance How well document d supports the claim that e is relevant to q Y t2q P(t|d, e)n(t,q) Simplifying assumption 
 (t and e are conditionally independent given d) P(t|✓d )
42. Document-entity associations - Boolean (or set-based) approach - Weighted by the confidence in entity linking - Consider other entities mentioned in the document e q xxxx x xxx xx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxxx xxxxxx xx x xxx xx x xxxx xx xxx x xxxxx xx x xxx xx xxxx xx xxx xx x xxxxx xxx