Tweet
Tweet

Slide 1

Slide 1 text

® © 2014 MapR Technologies 1 ® © 2014 MapR Technologies áǎìDžéƓ— MahoutSSolrlįæDL ¬~£¯Ž3‚¨¯V

Slide 2

Slide 2 text

® © 2014 MapR Technologies 3 ´°W˜‹{ •  ¬~£¯Ž3‚¨¯SX? •  ¬~£¯Ž3‚¨¯l§33VģŨFhVX? •  ƅſ¬~£¯Ž3‚¨¯Wũ •  ege9¬~£¯Ž3‚¨¯VFhVX?

Slide 3

Slide 3 text

® © 2014 MapR Technologies 4 áǎìDžéƓ ¬~£¯Ž3‚¨¯VfaôÙU¬~£¯Ž3‚¨ ¯t¯ƒ¯WŽq¯SáƖlĜś

Slide 4

Slide 4 text

® © 2014 MapR Technologies 5 ® © 2014 MapR Technologies ƋŕVP9Q

Slide 5

Slide 5 text

® © 2014 MapR Technologies 6 ìDžéƓOQT:9:aW?

Slide 6

Slide 6 text

® © 2014 MapR Technologies 7 ìDžéƓo«ª…¢SJWáƖ A 1 A 2 ! " # $ T A 1 A 2 ! " # $= A 1 T A 2 T ! " % % # $ & & A 1 A 2 ! " # $ = A 1 T A 1 A 1 T A 2 AT 2 A 1 AT 2 A 2 ! " % % # $ & & r 1 r 2 ! " % % # $ & & = A 1 T A 1 A 1 T A 2 AT 2 A 1 AT 2 A 2 ! " % % # $ & & h 1 h 2 ! " % % # $ & & r 1 = A 1 T A 1 A 1 T A 2 ! " % # $ & h 1 h 2 ! " % % # $ & & O(κ k d + k3 d) = O(k2 d log n + k3 d) for small k, high quality O(κ d log k) or O(d log κ log k) for larger k, looser quality B<GA (;29DH>C I

Slide 7

Slide 7 text

® © 2014 MapR Technologies 8 ¬~£¯Ž3‚¨¯ •  Ž3ˆ: µ2W»ê4§335Soq¢SWq¯ˆ©{‚¨¯ –  Ž3ˆX¬~£¯Ž3‚¨¯¤Ž«WéƓVĬkih •  ťĂWƊīUaD@XŴ]D9q¯ˆ©{‚¨¯lģĘFhBS>àÐ •  %-0'(.2!>: ƚĸ.ƅſ.ÇŸœ3„W¬„©¯ßƮ.e„ oc¬ƒ¬‚3RWŪāʼnĶWģĘ

Slide 8

Slide 8 text

® © 2014 MapR Technologies 9 Google Maps: ¬„©¯W¬~£¯Ž3‚¨¯

Slide 9

Slide 9 text

® © 2014 MapR Technologies 10 Google Maps: {“­ƒ3ęºW¬~£¯Ž3‚¨¯

Slide 10

Slide 10 text

® © 2014 MapR Technologies 11 ® © 2014 MapR Technologies ¬~£¯Ž3‚¨¯WD@^ (]LX6I want a pony7)

Slide 11

Slide 11 text

® © 2014 MapR Technologies 12 I want a pony! <5"34$:=@ 79$#?$ AB68#,1*

Slide 12

Slide 12 text

® © 2014 MapR Technologies 13 ¹šXE`WŚò: ýD9Ž3ˆlĬOQ9]F=1

Slide 13

Slide 13 text

® © 2014 MapR Technologies 14 ¬~£¯Ž3‚¨¯ ƱƒW»êX.ŜµW»êl čơFhWlŬAQ@ih

Slide 14

Slide 14 text

® © 2014 MapR Technologies 15 ¬~£¯Ž3‚¨¯ oª„Xª¯SÒDŽlƲÉ Š¥3«…XÀŃĕlƲÉ oª„ Š¥3«…

Slide 15

Slide 15 text

® © 2014 MapR Technologies 16 ¬~£¯Ž3‚¨¯ oª„Xª¯SÒDŽlƲÉ Š¥3«…XÀŃĕlƲÉ žšXª¯lƲÉ oª„ žš Š¥3«…

Slide 16

Slide 16 text

® © 2014 MapR Technologies 17 ¬~£¯Ž3‚¨¯ žšXƌVţlưD>h? oª„ žš Š¥3«… ?

Slide 17

Slide 17 text

® © 2014 MapR Technologies 18 ¬~£¯Ž3‚¨¯ ÒDŽ.aNjm! oª„ žš Š¥3«…

Slide 18

Slide 18 text

® © 2014 MapR Technologies 19 ¬~£¯Ž3‚¨¯>TWe:V »kih=k=g]DL=1

Slide 19

Slide 19 text

® © 2014 MapR Technologies 20 SBjR.^mUSÈE@. žšaŸ’3>ưD9mRF

Slide 20

Slide 20 text

® © 2014 MapR Technologies 21 ¬~£¯Ž3‚¨¯ çÝ>Ÿ’3lƲÉDLfT:U h? oª„ žš Š¥3«… o£ªo ƌVţlo£ªoV

Slide 21

Slide 21 text

® © 2014 MapR Technologies 22 ¬~£¯Ž3‚¨¯ oª„ žš Š¥3«… o£ªo aDçÝ>Ÿ’3lƲÉDLf. Ÿ’3XƌWaWlčơFhWV ŌV×LU9... ?

Slide 22

Slide 22 text

® © 2014 MapR Technologies 23 ƘWĺŒŽ3ˆWòą •  ƈœVµøWoq¢XF[QSĺŒFh (ǁ>Ÿ’3>ưD9 S9:ĞĆXŌV×LU9…) –  ũrs«w¢y¦£¯.t¬œ3ˆ3Wƅſ •  ĤƷƟU¸úXĨŲ^>U9 –  =^JgWnjS=Ÿ’3We:Uƥ;GŴ]ihoq¢lģŨD L9WRU9őg •  ưD9WXũóÐUĺŒ –  BiBJ>žźų9ū^WċƂWĞĆƑR8g.¬~£¯Ž3‚¨¯ WėǀSUh

Slide 23

Slide 23 text

® © 2014 MapR Technologies 24 »ê=fŌV×PċƂlĚhVX •  ­|™nq«lĬOQ§33 x oq¢WNJƢ ª{„l Ģö –  ŝBWq¯ˆ©{‚¨¯WNJƢXçQWLJŀÐUĉ^¿kHSŊ[h SǍUŽ3ˆR8h •  oq¢ x oq¢WĺŒ ª{„VŁƁ •  ũóÐUĺŒlƭD.ŌV×Pq¯ƒ}3ˆ3lÅPAQq¯ ƒ}3ˆ3 ª{„lĢö –  ÊþǑÜŊLLRXTWĺŒ>ū^Wq¯ƒ}3ˆ3SDQĬ;h= lıülĈOQďňFhWVŬAVUh –  Apache Mahout WRowSimilarityJobXLLRlĬ:

Slide 24

Slide 24 text

® © 2014 MapR Technologies 25 Apache Mahout •  v3›¯‡3„ Apache ›­ƒs{http://mahout.apache.org •  Mahout version 0.9 X 2014²2ÁVªª3„ •  ìDžéƓWL`W„}3©š«Uo«ª…¢W©qš©ª –  Apache Hadoop Rê@aWa8iY.Hadoop ëñWaWa8h –  äC9„}3«RW^ê@aWa8h –  Ƈƣá»FhaWa8iY.‚3}¯‚¥«UaWa8h •  ÛğWaWlŭ_ –  {©„ˆª¯| & ÖãŋŠ –  ÃƆ –  ¬~£¯Ž3‚¨¯ –  Mahout Math ©qš©ª

Slide 25

Slide 25 text

® © 2014 MapR Technologies 26 ­|™nq« oª„ žš Š¥3«… oª„ žš Š¥3«… oª„

Slide 26

Slide 26 text

® © 2014 MapR Technologies 27 ­|™nq« u1 u3 u2 u1 u3 u2 u1 t1# t4# t3# t2# t3# t3# t1#

Slide 27

Slide 27 text

® © 2014 MapR Technologies 28 NJƢ ª{„: §33/oq¢ oª„ žš Š¥3«… ✔ ✔ ✔ ✔ ✔ ✔ ✔

Slide 28

Slide 28 text

® © 2014 MapR Technologies 29 ĺŒ ª{„: oq¢/oq¢ - 1# 2# 1# 1# 1# 1# 2# 1# SWV@?. 0# 0# 0# 0#

Slide 29

Slide 29 text

® © 2014 MapR Technologies 30 ĺŒ ª{„: oq¢/oq¢ LLR L?PESW";)&+U[… - 1# 2# 1# 1# 1# 1# 2# 1# 0# 0# 0# 0#

Slide 30

Slide 30 text

® © 2014 MapR Technologies 31 ĺŒ•q‘ª ª{„ 1# 1# not# not# 1#

Slide 31

Slide 31 text

® © 2014 MapR Technologies 32 ũólÅPAh A not A B 13 1000 not B 1000 100,000 A not A B 1 0 not B 0 2 A not A B 1 0 not B 0 10,000 A not A B 10 0 not B 0 100,000 JiKiWōŷ=fTmUĖł>ŐAh?

Slide 32

Slide 32 text

® © 2014 MapR Technologies 33 ũólÅPAh •  «3 LLR X< LLR lăæ A not A B 1 0 not B 0 2 A not A B 1 0 not B 0 10,000 A not A B 10 0 not B 0 100,000 1.95 4.52 14.3 JiKiWōŷ=fTmUĖł>ŐAh? A not A B 13 1000 not B 1000 100,000 0.90

Slide 33

Slide 33 text

® © 2014 MapR Technologies 34 ĺŒ ª{„ : LLR L?PESW";)&+U[ - 1# 2# 1# 1# 1# 1# 2# 1# 0# 0# 0# 0#

Slide 34

Slide 34 text

® © 2014 MapR Technologies 35 q¯ƒ}3ˆ3 ª{„: ũóÐUĺŒ ✔# ✔# MQ: 3% = "-40$65;/";)&+1! 90YJ …

Slide 35

Slide 35 text

® © 2014 MapR Technologies 36 T:cOQ§33V¬~£¯Ž3‚¨¯l ģĥFiYe9RDd:=?

Slide 36

Slide 36 text

® © 2014 MapR Technologies 37 ‚¯›«ClťĠ q“œ3‚¨¯: ¬~£¯Ž3‚¨¯‚„¢l áƖFhL`VĒƼŋŠlăæ

Slide 37

Slide 37 text

® © 2014 MapR Technologies 38 JWÎV.ĒƼt¯ƒ¯>T:ê@= ÅQ^]Dd:

Slide 38

Slide 38 text

® © 2014 MapR Technologies 39 Apache Solr/Apache Lucene •  Apache Solr/Lucene: v3›¯‡3„W–®™«UĒƼt¯ƒ¯ –  ƻǃ=PƩƵVq¯Ž‹{„CiLŽ3ˆZW{tª –  ™«y„.ÇåŽ3ˆ.ŎČÐVĊ^ķACiLŽ3ˆ •  Lucene –  ĒƼW~oìĪlģŨ –  Ŕ¬œ«©qš©ª •  Solr –  *$lƶ_œ3„W©‹–3 –  q¯ˆ3™s3„RŶûR?hL`.ãŹ>ħƗ

Slide 39

Slide 39 text

® © 2014 MapR Technologies 40 LucidWorks •  Solr/Lucene lœ3„VDLt¯ˆ3›©q…ĀA›©‹ ™u3¢ –  %"'lƶ_©‹–3 –  ™ª3W•3ƒ¨¯>&VķƧ •  LucidWorks X Solr WìĪXJW]]V.Solr WĦæřålħƗ VFhBSR.Lucene lƳƍVăæR?he:VDQ9h

Slide 40

Slide 40 text

® © 2014 MapR Technologies 41 Solr/Lucene/LucidWorks WÖIJ

Slide 41

Slide 41 text

® © 2014 MapR Technologies 42 y¦£¯S™p3«VÊFhĒƼ •  Ž3ˆXy¦£¯WĽ¿ •  y¦£¯X¹ƜÐVˆq«.Ƹ¾.y3®3.´İW ™p3«lĈP •  ¹ƜÐU‚¯ˆ‹{„ –  field:(୯ޠ1 ୯ޠ2) •  ]LX –  field:(୯ޠ1 OR ୯ޠ2) field:(“୯ޠ1 ୯ޠ2” ~ 5) field:(୯ޠ1 AND ୯ޠ 2)

Slide 42

Slide 42 text

® © 2014 MapR Technologies 43 q¯ƒ}3ˆ3 ª{„ ✔ id: t4 title: puppy desc: The sweetest little puppy ever. keywords: puppy, dog, pet indicators: (t1) ";)&+3/8%*1= Solr 0$65;/";)&+1 !90:'5;.(7;#;);D\KE X: ";)&+1!90 Solr ";.,%*0$65;/5+.+ OZYJ ";)&+R";.,%*NTF

Slide 43

Slide 43 text

® © 2014 MapR Technologies 44 áēWũ: ƅſ¬~£¯Ž3‚¨¯

Slide 44

Slide 44 text

® © 2014 MapR Technologies 45

Slide 45

Slide 45 text

® © 2014 MapR Technologies 46 §33W»ê: ƒ¥…WĔƴ6Take the A Train7l ƨ@

Slide 46

Slide 46 text

® © 2014 MapR Technologies 47 »êlœ3„V¬~£¯Ž3‚¨¯lģĥ

Slide 47

Slide 47 text

® © 2014 MapR Technologies 48 ƅſ¬~£¯‰3WÑĮlÅQ^]Dd:

Slide 48

Slide 48 text

® © 2014 MapR Technologies 49 ĒƼy¦£¯WŽ3ˆ†‹SUhſƴW£ˆ Ž3ˆ •  MusicBrainz WŽ3ˆ •  Ž3ˆVXo3p„ ID.MusicBrainz ID.o3p„Ĕ. |«3›/Ŝµ.¸Ŗ4ÇŢĔ5.õĵWe:UĞĆ>ŭ]ih

Slide 49

Slide 49 text

® © 2014 MapR Technologies 50 §33»êNJƢWũ: ŗƨ­|™nq« ”q©qDQ9h ÓÃR§33119 XţlDQ9]F=? ƒ¥…¡¦3ƒ‚¥¯ Duke EllingtonW o3p„ID ÌƦ qœ¯ ˆq› §33ID o3p„ID ©‹{ID

Slide 50

Slide 50 text

® © 2014 MapR Technologies 51 ¬~£¯‰3t¯ƒ¯WÑÓ 51#

Slide 51

Slide 51 text

® © 2014 MapR Technologies 52 ¬~£¯‰3t¯ƒ¯WÑÓ 52#

Slide 52

Slide 52 text

® © 2014 MapR Technologies 53 ƅſ¬~£¯Ž3‚¨¯W Lucene y¦£¯ •  ¬3’¯|CiL Mahout ¬~£¯‰3¤Ž«Wq¯ƒ}3 ˆ3 ª{„WŽ3ˆ>.o3p„Ž3ˆ†‹W y¦£¯¶W indicator ™p3«VťĂCiQ9h id 541902 mbid 983d4f8f-473e-4091-8394-415c105c4656 ame Charlie Winston area United Kingdom gender None Indicator_artists 997727,815,830794,59588,900,2591,1344,696268 id 1710 mbid 592a3b6d-c42b-4567-99c9-ecf63bd66499 ame Chck Berry area United States gender Male Indicator_artists 386685,875994,637954,3418,1344,789739,1460

Slide 53

Slide 53 text

® © 2014 MapR Technologies 54 v™©q¯éƓ oq¢ £ˆŽ3ˆ §33NJƢ ­|™nq« Mahout l ĬOLÃƹ ĒƼ {“­ƒ3 q¯ƒ}3ˆ3

Slide 54

Slide 54 text

® © 2014 MapR Technologies 55 MapR lĬOLªo«ˆq¢¬~£¯Ž3‚¨¯ oq¢ £ˆŽ3ˆ ­|™nq« Mahout l ĬOLÃƹ Web po ĒƼ {“­ƒ3 Pig Python ÔD9§33NJƢ ¬~£¯Ž3‚¨¯ èĿWŽ3ˆĄÉ èĿ èĿR+) %$ lĬæDQĄÉ &/)+

Slide 55

Slide 55 text

® © 2014 MapR Technologies 56 ‚¯›«V •  h l»OL§33(§33>cgígDL¼ŘWœ{«) •  Ah AT Ah ( ) AT A ( )h §33¶ĝW¬~£¯Ž3‚¨¯ (ŃĹX¼ŘZSƛFŁƁ) oq¢¶ĝW¬~£¯Ž3‚¨¯ (ƯĢWƙšlŁƔ) A X¼Řl§33ZŁƁ

Slide 56

Slide 56 text

® © 2014 MapR Technologies 57 o3y{Š¥·WijĐõ AT Ah ( ) AT A ( )h §33¶ĝW¬~£¯Ž3‚¨¯ oq¢¶ĝW¬~£¯Ž3‚¨¯ 1PàW÷ČRX.]Gªo«ˆq¢WČĴ>ľñ4ƺǒW¶5/ ¼ÎČĴXR?U9L`.ƈŮŅRŔű/ 2PàW÷ČRX.v™©q¯R4ƉÏV5¼ÎČĴ>ņĪ/ ÔD9§33Wœ{«(h)WäC9ČĴW^lªo«ˆq¢R »;Ye9/BWL`ńŽ>ƈœVe@Uh/

Slide 57

Slide 57 text

® © 2014 MapR Technologies 58 ® © 2014 MapR Technologies ege9¬~£¯Ž3‚¨¯VFhVX

Slide 58

Slide 58 text

® © 2014 MapR Technologies 59 «Š¤3‰«¬~£¯Ž3‚¨¯

Slide 59

Slide 59 text

® © 2014 MapR Technologies 60 «Š¤3‰«¬~£¯Ž3‚¨¯

Slide 60

Slide 60 text

® © 2014 MapR Technologies 61 ũ •  §33>{tª (A) lÉÙ –  (»êðÞ §33oq¢ {tª •  §33>—Žv (B) lŗƨ –  (»êðÞ §33oq¢ —Žv •  TA X{tªW¬~£¯Ž3‚¨¯lģĥ –  ,BNfl<ƭDMOLWRXU9RDd:=- •  TB X—ŽvW¬~£¯Ž3‚¨¯lģĥ –  ,BifWêĸ>ū?RXU9RF=-

Slide 61

Slide 61 text

® © 2014 MapR Technologies 62 ±¼USBj •  BTA X{tªVÊFhêĸlª~£¯ –  (8i.BiĒƼt¯ƒ¯EbU=OLOA –  9cŞ:U.~¯¯Œc£ˆŽ3ˆXÅQU9D

Slide 62

Slide 62 text

® © 2014 MapR Technologies 63 áēWũ •  {tª: ,Paco de Lucia- („q¯Wzˆª„) •  ƄùW£ˆŽ3ˆĒƼĖĭ: –  , %#'(&%- „q¯W~£Žpšĉě –  ƌX\SmTUD •  ¬~£¯Ž3‚¨¯œ3„ĒƼ: –  ™©£¯~zˆ3S‰¯€3 –  „q¯ş.{©‚‹{zˆ3 –  $"$>{©‚‹{U ™©£¯~Wª™lƏLj

Slide 63

Slide 63 text

® © 2014 MapR Technologies 64 aOSŮŅe@R?U9=?

Slide 64

Slide 64 text

® © 2014 MapR Technologies 65 ÊƽõVeg{­„¬~£¯Ž3‚¨¯lÆö AT A ( )h BT A ( )h v™©q¯©3’¯|lăæDL ƄùůW¬~£¯Ž3‚¨¯ {­„¬~£¯Ž3‚¨¯

Slide 65

Slide 65 text

® © 2014 MapR Technologies 66 A §33 ¼Ř

Slide 66

Slide 66 text

® © 2014 MapR Technologies 67 A 1 A 2 ! " # $ §33 ¼Ř ˆq›1 ¼Ř ˆq›2

Slide 67

Slide 67 text

® © 2014 MapR Technologies 68 A 1 A 2 ! " # $ T A 1 A 2 ! " # $= A 1 T A 2 T ! " % % # $ & & A 1 A 2 ! " # $ = A 1 T A 1 A 1 T A 2 AT 2 A 1 AT 2 A 2 ! " % % # $ & & r 1 r 2 ! " % % # $ & & = A 1 T A 1 A 1 T A 2 AT 2 A 1 AT 2 A 2 ! " % % # $ & & h 1 h 2 ! " % % # $ & & r 1 = A 1 T A 1 A 1 T A 2 ! " % # $ & h 1 h 2 ! " % % # $ & &

Slide 68

Slide 68 text

® © 2014 MapR Technologies 69 <]A: ŵ9Ë>ŏ9Ä¿

Slide 69

Slide 69 text

® © 2014 MapR Technologies 70 ïćWáƖÍWúáWòą •  NjdžWƷƟ → §33X‹›3ƒD=ÅU9 •  ĎŰõ → ƫÕÈEĖĭMSǏ?fiQD]: •  éƓűÜ → Jiÿ·éƓDU@UOQD]: •  ØDQïÍWǓƕUòąRXU9

Slide 70

Slide 70 text

® © 2014 MapR Technologies 71 ĖĭWŽpª¯| •  ¬~£¯Ž3‚¨¯ĖĭWƙšlÉiƐ;hL`VŽpª¯ |>Ĭkih –  ƙšWÉiƁ;X©¯‰¢V»: •  Žpª¯|Xv™©q¯õĪlŵâCHh •  Žpª¯|XÈÌVáēWõĪlegŏ@Fh–3™s{ Vġ9áƎ>8h

Slide 71

Slide 71 text

® © 2014 MapR Technologies 72 ĖĭWŽpª¯| •  ¬~£¯Ž3‚¨¯ĖĭWƙšlÉiƐ;hL`VŽpª¯ |>Ĭkih –  ƙšWÉiƁ;X©¯‰¢V»: •  Žpª¯|Xv™©q¯õĪlŵâCHh •  Žpª¯|XÈÌVáēWõĪlegŏ@Fh–3™s{ Vġ9áƎ>8h “ƌWTmUŁƔega±?UŞ9>¸h”

Slide 72

Slide 72 text

® © 2014 MapR Technologies 73 UIŽpª¯|>ľñ?

Slide 73

Slide 73 text

® © 2014 MapR Technologies 74 ‚¯›«UŽpª¯|o«ª…¢ •  ©¯{WÊþVxr„ÃƤlĂ;L¿ö„~o •  ƊīU¬œ«WÉiƐ;l»:“q…W„}3«lßƮ •  ƬÚXBW@f9 •  ]L … floor(t/T) l‚3SDQĬ: s = logr + N(0,logε) ε ∈ 1.5,3 [ ] Δr r ∝ε

Slide 74

Slide 74 text

® © 2014 MapR Technologies 75 ũ;Y… ε = 2 1 2 8 3 9 15 7 6 1 8 14 15 3 2 22 10 1 3 8 2 10 5 7 4 1 2 10 7 3 8 6 14 1 5 33 15 2 9 11 29 1 2 7 3 5 4 19 6 1 3 5 23 9 7 4 2 2 4 11 8 3 1 44 9 2 3 1 4 6 7 8 33 3 4 1 2 10 11 15 14 11 1 2 4 5 7 3 14 1 8 7 3 22 11 2 33

Slide 75

Slide 75 text

® © 2014 MapR Technologies 76 émMBS:# NjdžCihBSXŏ9BSM#

Slide 76

Slide 76 text

® © 2014 MapR Technologies 77 Q & A @mapr_japan maprjapan [email protected] <ò9¿kHXBNf]R MapR maprtech mapr-technologies