Search as Communication: Lessons from a Personal Journey

The Etsy's Code as Craft presentation discusses lessons learned from my personal journey in search engineering. It covers insights from library science about treating search as an information-seeking context and communicating with users. It also discusses the importance of entity detection and how to leverage corpus features to improve extraction. I realized that queries vary in difficulty and systems need to recognize this and adapt accordingly. The key takeaway is that search should be treated as a communication problem rather than just a ranking task.

Transcript

1. Search  as  Communica/on:   Lessons  from  a  Personal  Journey   Daniel  Tunkelang   Head  of  Query  Understanding,  LinkedIn  
2. These  are  great  textbooks  on  informa/on  retrieval.  
3. Unfortunately,  I  never  read  them  in  school.   But  I  did  study  graphs  and  stuff.    
4. I  found  myself  developing  a  search  engine.  
5. And  the  next  thing  I  knew,  I  was  a  search  guy.  
6. So  what  did  I  learn  along  the  way?  
7. Search  isn't  a  ranking  problem.   It's  a  communica/on  problem.  
8. Outline   1.  Lessons  from  Library  Science     2.  Adventures  with  InformaAon  ExtracAon     3.  A  Moment  of  Clarity  
9. 1.  Lessons  from  Library  Science  
10. InformaAon  need   query   select  from  results   rank  using  IR  model   USER:   SYSTEM:   M-­‐idf   PageRank   A  birds-­‐eye  view  of  how  search  engines  work.  
11. Old  school  search:  ask  a  librarian.  
12. Search  lives  in  an  informa/on-­‐seeking  context.     [Pirolli  and  Card,  2005]  
13. vs.   Recognize  ambiguity  and  ask  for  clarifica/on.  
14. Clarify,  then  refine.   Computers   Books  
15. Faceted  search.  It’s  not  just  for  e-­‐commerce.  
16. Give  users  transparency,  guidance,  and  control.  
17. Take-­‐away  for  search  engine  developers:       Act  like  a  librarian.  Communicate  with  your  user.  
18. 2.  Adventures  with  Informa/on  Extrac/on  
19. String  matching  is  great  but  has  limits.  
20. 20   20 for i in [1..n]! s ← w1 w2 … wi ! if Pc (s) > 0! a ← new Segment()! a.segs ← {s}! a.prob ← Pc (s)! B[i] ← {a}! for j in [1..i-1]! for b in B[j]! s ← wj wj+1 … wi ! if Pc (s) > 0! a ← new Segment()! a.segs ← b.segs U {s}! a.prob ← b.prob * Pc (s)! B[i] ← B[i] U {a}! sort B[i] by prob! truncate B[i] to size k! People  search  for  en//es.  Recognize  them!  
21. Named  en/ty  recogni/on  is  free,  as  in  free  beer.  
22. Problem:  they  process  each  document  separately.   EnAty   DetecAon   System   Why  not  take  advantage  of  corpus  features?      
23. Give  your  documents  the  right  to  vote!   Use  a  high-­‐recall  method  to  collect  candidates.   •  e.g.,  all  Atle-­‐case  spans  of  words  other   than  single  word  beginning  a  sentence.     Process  each  document  separately.   •  Each  candidate  is  assigned  an  enAty  type,   or  no  type  at  all.     If  a  candidate  is  mostly  assigned  a  single  enAty   type,  extrapolate  to  all  its  occurrences.  
24. Looking  for  topics?  Use  idf,  and  its  cousin  ridf.   Inverse  document  frequency  (idf)   •  Too  low?  Probably  a  stop  word.   •  Too  high?  Could  be  noise.     Residual  inverse  document  frequency  (ridf)   •  Predict  idf  using  Poisson  model.   •  Difference  between  idf  and  predicted  idf.    “a  good  keyword  is  far  from  Poisson”            [Church  and  Gale,  1995]  
25. Terminology  extrac/on?  Try  data  recycling.  
26. Obtain  en//es  by  any  means  necessary.  
27. Take-­‐away  for  search  engine  developers:       En/ty  detec/on  is  crucial.  And  it  isn’t  that  hard.  
28. 3.  A  Moment  of  Clarity  
29. informaAon  Need   query   select  from  results   rank  using  IR  model   USER:   SYSTEM:   M-­‐idf   PageRank   Let’s  go  back  to  our  pigeons  for  a  moment.    
30. What  does  this  process  look  like  to  the  system?   vs.  
31. And  here’s  what  it  looks  like  to  the  user.   GOOD   NOT  SO  GOOD   But  can  the  system  tell  the  difference?  
32. User  experience  should  reflect  system  confidence.   vs.  
33. h^p://searchengineland.com/ge`ng-­‐organized-­‐paid-­‐search-­‐user-­‐intent-­‐the-­‐search-­‐funnel-­‐116312   Derived  from  [Jansen  et  al,  2007].   Searches  reflect  a  variety  of  informa/on  needs.  
34. 34   34 for i in [1..n]! s ← w1 w2 … wi ! if Pc (s) > 0! a ← new Segment()! a.segs ← {s}! a.prob ← Pc (s)! B[i] ← {a}! for j in [1..i-1]! for b in B[j]! s ← wj wj+1 … wi ! if Pc (s) > 0! a ← new Segment()! a.segs ← b.segs U {s}! a.prob ← b.prob * Pc (s)! B[i] ← B[i] U {a}! sort B[i] by prob! truncate B[i] to size k! We  can  segment  informa/on  need  from  the  query.  
35. We  can  learn  from  analyzing  user  behavior.  
36. And  we  can  look  at  our  relevance  scores.   Naviga/onal   Exploratory  
37. Claudia  Hauff,  Query  Difficulty  for  Digital  Libraries  [2009]   There  are  many  pre-­‐  and  post-­‐retrieval  signals.  
38. Take-­‐away  for  search  engine  developers:       Queries  vary  in  difficulty.  Recognize  and  adapt.  
39. Review   1.  Lessons  from  Library  Science   •  Act  like  a  librarian.  Communicate  with  users.     2.  Adventures  with  InformaAon  ExtracAon   •  EnAty  detecAon  is  crucial.  And  isn’t  that  hard.     3.  A  Moment  of  Clarity   •  Queries  vary  in  difficulty.  Recognize  and  adapt.  
40. Conclusion:   Read  the  textbooks.   But  treat  search  as  a  communica/on  problem.  
41. WE’RE  HIRING!   hbp://data.linkedin.com/search       Contact  me:   dtunkelang@linkedin.com   hbp://linkedin.com/in/dtunkelang