Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Computer Called "Watson"

James Thomas
September 17, 2012

A Computer Called "Watson"

LondonJS talk on the technology behind IBM Watson.

James Thomas

September 17, 2012
Tweet

More Decks by James Thomas

Other Decks in Technology

Transcript

  1. A Computer Called “Watson”
    James Thomas, IBM Emerging Tech
    COVER PHOTO
    Add cover photo
    Monday, 17 September 2012

    View full-size slide

  2. GARY KASPAROV VERSUS “DEEP BLUE”
    MAY 1997
    Monday, 17 September 2012

    View full-size slide

  3. BLUE GENE
    DECEMBER 1999
    Monday, 17 September 2012

    View full-size slide

  4. “Fast forward to 2004...”
    Monday, 17 September 2012

    View full-size slide

  5. JEOPARDY QUIZ SHOW
    Monday, 17 September 2012

    View full-size slide

  6. KEN JENNINGS
    CHAMPION OF CHAMPIONS
    Monday, 17 September 2012

    View full-size slide

  7. KEN JENNINGS
    CHAMPION OF CHAMPIONS
    KEN’S PERFORMANCE
    Monday, 17 September 2012

    View full-size slide

  8. What if we build a
    computer to play Jeopardy?
    Monday, 17 September 2012

    View full-size slide

  9. What if we build a
    computer to play Jeopardy?
    It’s way too hard
    Monday, 17 September 2012

    View full-size slide

  10. Jeopardy!
    Finite, mathematically
    defined search space
    Limited number of moves
    and states
    Symbols completed
    grounded in mathematical
    rules of the games
    Chess
    Enormous problem domain
    and search space
    Source material is inherently
    unstructured information
    Advanced natural language
    processing
    Monday, 17 September 2012

    View full-size slide

  11. BASELINE PERFORMANCE
    INITIAL EXPERIMENT
    Monday, 17 September 2012

    View full-size slide

  12. “Fast forward to 2007...”
    Monday, 17 September 2012

    View full-size slide

  13. DR. DAVID FERRUCCI - “WATSON” TEAM DIRECTOR
    2007 - 2011
    Monday, 17 September 2012

    View full-size slide

  14. SIMULATING JEOPARDY! MATCHES
    Monday, 17 September 2012

    View full-size slide

  15. BASELINE PERFORMANCE
    INITIAL EXPERIMENT
    Monday, 17 September 2012

    View full-size slide

  16. WATSON WAS READY
    FEBRUARY 2011
    Monday, 17 September 2012

    View full-size slide

  17. Watson Jeopardy! Challenge
    February 14-16 2011.
    Ken Jennings
    Brad Rutter
    Monday, 17 September 2012

    View full-size slide

  18. WATSON 1 - HUMANS 0
    FEBRUARY 16, 2011
    Monday, 17 September 2012

    View full-size slide

  19. “Its largest airport is named
    for a WWII hero; its second
    largest, for a WWII battle”
    Category: US Cities
    Monday, 17 September 2012

    View full-size slide

  20. “Its largest airport is named
    for a WWII hero; its second
    largest, for a WWII battle”
    Category: US Cities
    Watson: Toronto
    Monday, 17 September 2012

    View full-size slide

  21. How does it work?
    Monday, 17 September 2012

    View full-size slide

  22. WATSON ARCHITECTURE - DEEPQA
    MASSIVELY PARALLEL PROBABILISTIC EVIDENCE-BASED ARCHITECTURE
    Monday, 17 September 2012

    View full-size slide

  23. DEEPQA ARCHITECTURE PHASES
    Question Analysis
    Primary Search
    Candidate Answer
    Generation
    Evidence Retrieval
    Evidence Scoring
    Merging and Ranking
    Monday, 17 September 2012

    View full-size slide

  24. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    CAPITALS
    QUESTION:
    WHAT IS THE CAPITAL
    OF SAMOA?
    Monday, 17 September 2012

    View full-size slide

  25. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    POETS & POETRY
    QUESTION:
    HE WAS A BANK CLERK IN THE YUKON BEFORE HE
    PUBLISHED “SONGS OF SOURDOUGH” IN 1907.
    Monday, 17 September 2012

    View full-size slide

  26. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    POETS & POETRY
    QUESTION:
    HE WAS A BANK CLERK IN THE YUKON BEFORE HE
    PUBLISHED “SONGS OF SOURDOUGH” IN 1907.
    DISCOVER:
    FOCUS
    Monday, 17 September 2012

    View full-size slide

  27. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    POETS & POETRY
    QUESTION:
    HE WAS A BANK CLERK IN THE YUKON BEFORE HE
    PUBLISHED “SONGS OF SOURDOUGH” IN 1907.
    DISCOVER:
    LEXICAL ANSWER TYPE
    Monday, 17 September 2012

    View full-size slide

  28. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    POETS & POETRY
    QUESTION:
    HE WAS A BANK CLERK IN THE YUKON BEFORE HE
    PUBLISHED “SONGS OF SOURDOUGH” IN 1907.
    DISCOVER:
    RELATIONSHIPS
    Monday, 17 September 2012

    View full-size slide

  29. DEEPQA ARCHITECTURAL PHASES
    QUESTION ANALYSIS
    CATEGORY:
    POETS & POETRY
    QUESTION:
    HE WAS A BANK CLERK IN THE YUKON BEFORE HE
    PUBLISHED “SONGS OF SOURDOUGH” IN 1907.
    DISCOVER:
    QUESTION CLASS
    Monday, 17 September 2012

    View full-size slide

  30. DEEPQA ARCHITECTURAL PHASES
    HYPOTHESIS GENERATION
    Hypothesis generation
    consists of two phases...
    Monday, 17 September 2012

    View full-size slide

  31. DEEPQA ARCHITECTURAL PHASES
    HYPOTHESIS GENERATION
    Primary Search
    Monday, 17 September 2012

    View full-size slide

  32. DEEPQA ARCHITECTURAL PHASES
    HYPOTHESIS GENERATION
    Data Sources - Unstructured
    Wikipedia
    IMDb
    Encyclopaedias
    Dictionaries
    Thesauri
    Literary Works
    Newswire Articles
    Monday, 17 September 2012

    View full-size slide

  33. DEEPQA ARCHITECTURAL PHASES
    HYPOTHESIS GENERATION
    Data Sources - Structured
    DBpedia
    Wordnet
    YAGO
    Monday, 17 September 2012

    View full-size slide

  34. DEEPQA ARCHITECTURAL PHASES
    HYPOTHESIS GENERATION
    Candidate Answer Generation
    Monday, 17 September 2012

    View full-size slide

  35. DEEPQA ARCHITECTURAL PHASES
    EVIDENCE RETRIEVAL
    Watson then searches for
    additional evidence...
    Monday, 17 September 2012

    View full-size slide

  36. DEEPQA ARCHITECTURAL PHASES
    EVIDENCE SCORING
    Deep Scoring
    Monday, 17 September 2012

    View full-size slide

  37. DEEPQA ARCHITECTURAL PHASES
    EVIDENCE SCORING
    0
    0.25
    0.5
    0.75
    1
    Temporal Geospatial LAT Popularity
    Deep Scoring
    Christopher Columbus Vasco De Gamma
    Monday, 17 September 2012

    View full-size slide

  38. DEEPQA ARCHITECTURAL PHASES
    FINAL MERGING AND RANKING
    Answer Merging
    Monday, 17 September 2012

    View full-size slide

  39. DEEPQA ARCHITECTURAL PHASES
    FINAL MERGING AND RANKING
    Answer Merging
    Monday, 17 September 2012

    View full-size slide

  40. DEEPQA ARCHITECTURAL PHASES
    FINAL MERGING AND RANKING
    Answer Ranking
    ISAAC NEWTON [0.10, 0.45 ... 0.78, 0.12] ???
    ANSWER SCORES RANK
    EDMOND HALLEY [0.31, 0.52 ... 0.08, 0.72] ???
    WILHELM TEMPEL [0.21, 0.54 ... 0.18, 0.92] ???
    PETER SELLERS [0.14, 0.04 ... 0.25, 0.41] ???
    Monday, 17 September 2012

    View full-size slide

  41. DEEPQA ARCHITECTURAL PHASES
    FINAL MERGING AND RANKING
    Monday, 17 September 2012

    View full-size slide

  42. DEEPQA ARCHITECTURAL PHASES
    FINAL MERGING AND RANKING
    Answer Ranking
    ISAAC NEWTON [0.10 ... 0.12] 3
    ANSWER SCORES RANK
    EDMOND HALLEY [0.31 ... 0.72] 1
    WILHELM TEMPEL [0.21 ... 0.92] 2
    PETER SELLERS [0.14 ... 0.41] 4
    0.14
    CONFIDENCE
    0.78
    0.34
    0.04
    Monday, 17 September 2012

    View full-size slide

  43. DEEPQA PERFORMANCE
    SCALING CHALLENGES
    Need to answer within a few
    seconds....
    Monday, 17 September 2012

    View full-size slide

  44. DEEPQA PERFORMANCE
    SCALING CHALLENGES
    Need to answer within a few
    seconds....
    Early implementation took
    over two hours
    Monday, 17 September 2012

    View full-size slide

  45. SCALING WATSON
    Monday, 17 September 2012

    View full-size slide

  46. WATSON HARDWARE
    NINETY POWER 750 SERVERS
    2880 POWER7 PROCESSOR CORES
    16 TERABYTES OF RAM
    Monday, 17 September 2012

    View full-size slide

  47. WATSON 1 - HUMANS 0
    FEBRUARY 16, 2011
    Monday, 17 September 2012

    View full-size slide

  48. PUTTING WATSON TO WORK
    What’s next?
    Monday, 17 September 2012

    View full-size slide

  49. PUTTING WATSON TO WORK
    HEALTHCARE
    Monday, 17 September 2012

    View full-size slide

  50. QUESTIONS?
    Monday, 17 September 2012

    View full-size slide