A Computer Called "Watson"

Slide 1

Slide 1 text

A Computer Called “Watson” James Thomas, IBM Emerging Tech COVER PHOTO Add cover photo Monday, 17 September 2012

Slide 2

Slide 2 text

GARY KASPAROV VERSUS “DEEP BLUE” MAY 1997 Monday, 17 September 2012

Slide 3

Slide 3 text

BLUE GENE DECEMBER 1999 Monday, 17 September 2012

Slide 4

Slide 4 text

“Fast forward to 2004...” Monday, 17 September 2012

Slide 5

Slide 5 text

JEOPARDY QUIZ SHOW Monday, 17 September 2012

Slide 6

Slide 6 text

KEN JENNINGS CHAMPION OF CHAMPIONS Monday, 17 September 2012

Slide 7

Slide 7 text

KEN JENNINGS CHAMPION OF CHAMPIONS KEN’S PERFORMANCE Monday, 17 September 2012

Slide 8

Slide 8 text

What if we build a computer to play Jeopardy? Monday, 17 September 2012

Slide 9

Slide 9 text

What if we build a computer to play Jeopardy? It’s way too hard Monday, 17 September 2012

Slide 10

Slide 10 text

Jeopardy! Finite, mathematically deﬁned search space Limited number of moves and states Symbols completed grounded in mathematical rules of the games Chess Enormous problem domain and search space Source material is inherently unstructured information Advanced natural language processing Monday, 17 September 2012

Slide 11

Slide 11 text

BASELINE PERFORMANCE INITIAL EXPERIMENT Monday, 17 September 2012

Slide 12

Slide 12 text

“Fast forward to 2007...” Monday, 17 September 2012

Slide 13

Slide 13 text

DR. DAVID FERRUCCI - “WATSON” TEAM DIRECTOR 2007 - 2011 Monday, 17 September 2012

Slide 14

Slide 14 text

SIMULATING JEOPARDY! MATCHES Monday, 17 September 2012

Slide 15

Slide 15 text

BASELINE PERFORMANCE INITIAL EXPERIMENT Monday, 17 September 2012

Slide 16

Slide 16 text

WATSON WAS READY FEBRUARY 2011 Monday, 17 September 2012

Slide 17

Slide 17 text

Watson Jeopardy! Challenge February 14-16 2011. Ken Jennings Brad Rutter Monday, 17 September 2012

Slide 18

Slide 18 text

WATSON 1 - HUMANS 0 FEBRUARY 16, 2011 Monday, 17 September 2012

Slide 19

Slide 19 text

“Its largest airport is named for a WWII hero; its second largest, for a WWII battle” Category: US Cities Monday, 17 September 2012

Slide 20

Slide 20 text

“Its largest airport is named for a WWII hero; its second largest, for a WWII battle” Category: US Cities Watson: Toronto Monday, 17 September 2012

Slide 21

Slide 21 text

How does it work? Monday, 17 September 2012

Slide 22

Slide 22 text

WATSON ARCHITECTURE - DEEPQA MASSIVELY PARALLEL PROBABILISTIC EVIDENCE-BASED ARCHITECTURE Monday, 17 September 2012

Slide 23

Slide 23 text

DEEPQA ARCHITECTURE PHASES Question Analysis Primary Search Candidate Answer Generation Evidence Retrieval Evidence Scoring Merging and Ranking Monday, 17 September 2012

Slide 24

Slide 24 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: CAPITALS QUESTION: WHAT IS THE CAPITAL OF SAMOA? Monday, 17 September 2012

Slide 25

Slide 25 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: POETS & POETRY QUESTION: HE WAS A BANK CLERK IN THE YUKON BEFORE HE PUBLISHED “SONGS OF SOURDOUGH” IN 1907. Monday, 17 September 2012

Slide 26

Slide 26 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: POETS & POETRY QUESTION: HE WAS A BANK CLERK IN THE YUKON BEFORE HE PUBLISHED “SONGS OF SOURDOUGH” IN 1907. DISCOVER: FOCUS Monday, 17 September 2012

Slide 27

Slide 27 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: POETS & POETRY QUESTION: HE WAS A BANK CLERK IN THE YUKON BEFORE HE PUBLISHED “SONGS OF SOURDOUGH” IN 1907. DISCOVER: LEXICAL ANSWER TYPE Monday, 17 September 2012

Slide 28

Slide 28 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: POETS & POETRY QUESTION: HE WAS A BANK CLERK IN THE YUKON BEFORE HE PUBLISHED “SONGS OF SOURDOUGH” IN 1907. DISCOVER: RELATIONSHIPS Monday, 17 September 2012

Slide 29

Slide 29 text

DEEPQA ARCHITECTURAL PHASES QUESTION ANALYSIS CATEGORY: POETS & POETRY QUESTION: HE WAS A BANK CLERK IN THE YUKON BEFORE HE PUBLISHED “SONGS OF SOURDOUGH” IN 1907. DISCOVER: QUESTION CLASS Monday, 17 September 2012

Slide 30

Slide 30 text

DEEPQA ARCHITECTURAL PHASES HYPOTHESIS GENERATION Hypothesis generation consists of two phases... Monday, 17 September 2012

Slide 31

Slide 31 text

DEEPQA ARCHITECTURAL PHASES HYPOTHESIS GENERATION Primary Search Monday, 17 September 2012

Slide 32

Slide 32 text

DEEPQA ARCHITECTURAL PHASES HYPOTHESIS GENERATION Data Sources - Unstructured Wikipedia IMDb Encyclopaedias Dictionaries Thesauri Literary Works Newswire Articles Monday, 17 September 2012

Slide 33

Slide 33 text

DEEPQA ARCHITECTURAL PHASES HYPOTHESIS GENERATION Data Sources - Structured DBpedia Wordnet YAGO Monday, 17 September 2012

Slide 34

Slide 34 text

DEEPQA ARCHITECTURAL PHASES HYPOTHESIS GENERATION Candidate Answer Generation Monday, 17 September 2012

Slide 35

Slide 35 text

DEEPQA ARCHITECTURAL PHASES EVIDENCE RETRIEVAL Watson then searches for additional evidence... Monday, 17 September 2012

Slide 36

Slide 36 text

DEEPQA ARCHITECTURAL PHASES EVIDENCE SCORING Deep Scoring Monday, 17 September 2012

Slide 37

Slide 37 text

DEEPQA ARCHITECTURAL PHASES EVIDENCE SCORING 0 0.25 0.5 0.75 1 Temporal Geospatial LAT Popularity Deep Scoring Christopher Columbus Vasco De Gamma Monday, 17 September 2012

Slide 38

Slide 38 text

DEEPQA ARCHITECTURAL PHASES FINAL MERGING AND RANKING Answer Merging Monday, 17 September 2012

Slide 39

Slide 39 text

DEEPQA ARCHITECTURAL PHASES FINAL MERGING AND RANKING Answer Merging Monday, 17 September 2012

Slide 40

Slide 40 text

DEEPQA ARCHITECTURAL PHASES FINAL MERGING AND RANKING Answer Ranking ISAAC NEWTON [0.10, 0.45 ... 0.78, 0.12] ??? ANSWER SCORES RANK EDMOND HALLEY [0.31, 0.52 ... 0.08, 0.72] ??? WILHELM TEMPEL [0.21, 0.54 ... 0.18, 0.92] ??? PETER SELLERS [0.14, 0.04 ... 0.25, 0.41] ??? Monday, 17 September 2012

Slide 41

Slide 41 text

DEEPQA ARCHITECTURAL PHASES FINAL MERGING AND RANKING Monday, 17 September 2012

Slide 42

Slide 42 text

DEEPQA ARCHITECTURAL PHASES FINAL MERGING AND RANKING Answer Ranking ISAAC NEWTON [0.10 ... 0.12] 3 ANSWER SCORES RANK EDMOND HALLEY [0.31 ... 0.72] 1 WILHELM TEMPEL [0.21 ... 0.92] 2 PETER SELLERS [0.14 ... 0.41] 4 0.14 CONFIDENCE 0.78 0.34 0.04 Monday, 17 September 2012

Slide 43

Slide 43 text

DEEPQA PERFORMANCE SCALING CHALLENGES Need to answer within a few seconds.... Monday, 17 September 2012

Slide 44

Slide 44 text

DEEPQA PERFORMANCE SCALING CHALLENGES Need to answer within a few seconds.... Early implementation took over two hours Monday, 17 September 2012

Slide 45

Slide 45 text

SCALING WATSON Monday, 17 September 2012

Slide 46

Slide 46 text

WATSON HARDWARE NINETY POWER 750 SERVERS 2880 POWER7 PROCESSOR CORES 16 TERABYTES OF RAM Monday, 17 September 2012