Kevin Bacon Number with Neo4j

グラフデータベースのNeo4jを使って、ケビン・ベーコン数の検証をしたLTのスライド

Transcript

1. Neo4j Ͱ έϏϯɾϕʔ ίϯ਺ʹ৮ΕͯΈΔ 2016.8.17 (Wed)
2. • ໊લ: Տٱอ ൟ࢙ (kawakubox) • ւ֎ࣄۀ෦ Eng • ! ϑϧϚϥιϯ׬૸໨ࢦͯ͠ ΦϑΟε to ࣗ୐ ͷ Extreme ୀ ࣾͳΜ͔ͰτϨʔχϯάதɻ
3. ಥવͰ͕͢ɺέϏϯɾϕ ʔίϯͬͯ஌ͬͯ·͢ ͔ʁ
5. ΞϝϦΧ߹ऺࠃͷആ༏ ओͳग़ԋ࡞඼ • ʰτϨϚʔζʱ • ʰΞϙϩ13ʱ • ʰX-MEN: ϑΝʔετɾδΣωϨʔγϣϯʱ etc etc ...
6. Ͱ͸ɺέϏϯɾϕʔίϯ ਺ͬͯ஌ͬͯ·͔͢ʁ
7. έϏϯɾϕʔίϯ਺ͱ͸ өըആ༏ͷڞԋؔ܎ͷʮڑ཭ʯΛέϰΟϯɾϕʔίϯΛى఺ʹ ܭଌ͞ΕΔ਺ ϕʔίϯຊਓ͕ʮϋϦ΢ουͷશһ͕ࣗ෼ͷڞԋऀ͔ɺڞԋऀ ͷڞԋऀͩʯͱ͍͏झࢫͷൃݴͨ͜͠ͱ͕͖͔͚ͬ — wikipedia
8. έϏϯɾϕʔίϯ਺ͷਤ Ͱඳ͘ͱ
10. ࿡࣍ͷִͨΓ ࿡࣍ͷִͨΓʢΖ͘͡ͷ΁ͩͨΓɺSix Degrees of Separationʣ ͱ͸ɺશͯͷਓ΍෺ࣄ͸6εςοϓҎ಺Ͱܨ͕͍ͬͯͯɺ༑ୡͷ ༑ୡ…Λհͯ͠ੈքதͷਓʑͱؒ઀తͳ஌Γ߹͍ʹͳΔ͜ͱ͕ Ͱ͖Δɺͱ͍͏Ծઆɻ — wikipedia • SNSͳͲͷԼ஍ʹͳ͍ͬͯΔԾઆ
11. ·͞ʹάϥϑDBʹͱͬͯ࠷దͷ࿩ͳͷͰɺNeo4jͷੑೳݕূ΋ ݉Ͷͯେ͖ΊͳσʔλΛ౤ೖͯ͠ΈΑ͏
12. σʔλϞσϧ
14. Node : Person • ϓϩύςΟ • person_id • name
15. Relation : KNOWS • ϓϩύςΟͳ͠
16. લఏ • 1ਓ͸࠷௿100ਓͱ஌Γ߹͍Ͱ͋Δ
17. 1,000,000ਓͷੈք σʔλ౤ೖ࣌ؒ(csv import) IMPORT DONE in 7m 6s 292ms. Imported: 1000000 nodes 200000000 relationships 2000000 properties
18. 1,000,000ਓͷੈք ϗοϓ਺ͷ෼෍ # ࢼߦճ਺ : 1,065ճ {0=>0, 1=>0, 2=>43, 3=>1021, 4=>1, 5=>0}
19. 10,000,000ਓͷੈք IMPORT DONE in 3h 5m 4s 706ms. Imported: 10000000 nodes 2000000000 relationships 20000000 properties
20. 10,000,000ਓͷੈք ϗοϓ਺ͷ෼෍ # ࢼߦճ਺ : 1,065ճ {0=>0, 1=>0, 2=>4, 3=>592, 4=>469, 5=>0}
21. ֬ೝ༻ͷCYPHERΫΤϦ MATCH (a:Person {person_id: {i}}), (b:Person {person_id: {j}}), p = shortestPath((a)-[*..6]->(b)) RETURN length(p) as len
22. CYPHERΫΤϦͷ຋༁ MATCH // person_id = i Λ࣋ͭϊʔυΛऔಘ͠ɺ a ͱ͢Δ (a:Person {person_id: {i}}), // person_id = j Λ࣋ͭϊʔυΛऔಘ͠ɺ b ͱ͢Δ (b:Person {person_id: {j}}), // a ͱ b ͷϗοϓ਺͕ 6 Ҏ಺ͷ΋ͷͰɺ࠷୹ܦ࿏Λऔಘ͢Δ p = shortestPath((a)-[*..6]->(b)) RETURN // ࠷୹ܦ࿏ͷ௕͞(len)Λฦ٫͢Δ length(p) as len
23. 126,000,000ਓͷੈքʢ೔ຊʣ -rw-r--r-- 1 kawakubox staff 21M 8 16 22:09 person_1000000.csv -rw-r--r-- 1 kawakubox staff 221M 8 17 01:31 person_10000000.csv -rw-r--r-- 1 kawakubox staff 2.6G 8 16 22:12 relation_1000000.csv -rw-r--r-- 1 kawakubox staff 2.9G 8 17 01:34 person_japan.csv -rw-r--r-- 1 kawakubox staff 29G 8 17 03:53 relation_10000000.csv ϦϨʔγϣϯ౤ೖ༻σʔλ͕ 10,000,000 ͷ࣌఺Ͱ 30G ͋ͬͨ ͷͰɺσʔλ࡞੒࣌఺Ͱஅ೦ɺɺɺ
24. ࠓճ΍ͬͯΈͯͷؾ෇͖ • ϊʔυ͕ 10,000,000 Ͱ΋ɺద੾ʹ index షΕ͹ɺprimary ݕ ࡧࣗମ͸ 0ms ͘Β͍Ͱฦͬͯ͘Δ • ϦϨʔγϣϯ͕ 2,000,000,000 ͘Β͍ʹͳΔͱɺ஗͘ͳͬͯ ͘Δ (5sec ͘Β͍) • ΍͹͍ΫΤϦ͸ɺCPUར༻཰ 300% ௒͑ͱ͔ฏؾͰ͢Δ • Enterprise൛࢖ͬͯͷΫϥελϦϯάͳΜ͔͸ࢼͯ͠Έͨ ͍
25. END