nroongaでつくる groonga 全文検索 APIサーバ

2012年4月18日 東京Node学園 5限目 Lightning Talk

Transcript

1. OSPPOHBͰͭ͘Δ
   HSPPOHBશจݕࡧ
   "1*αʔό
   औక໾$50ઃᒜ༸࣐
   ZPKJTIJEBSB!FOJTIJUFDIDPN
   !EBSBTIJ
   — nroonga: a library for building groonga powered nodes.
   ೥݄೔౦ژ/PEFֶԂݶ໨-JHIUOJOH5BML
   

   View Slide

2. ͩ͠ΒΑ͏͡
   ઃᒜ༸࣐
   

   View Slide

3. ๺ւಓ͸ࡳຈ͔Βདྷ·ͨ͠ɻ
   

   View Slide

4. ΦʔϓϯιʔεΧϯϑΝϨϯε)PLLBJEPʹڠࢍ͠·͢
   
   
   $44/JUFJO4FPVM 7PMʹͯ౰ࣾϝϯόʔ͕ൃද͠·͢
   
   
   

   View Slide

5. “সإͷͨ͑ͳ͍”
   

   View Slide

6. αΠϯΠϯ
   &OHMJTIc೔ຊޠ
   ೔ຊ3VCZձٞʹ͍ͭͯ νέοτߪೖɾ֤छ͓ਃࠐΈ ࢀՃऀ޲͚৘ใ εϙϯαʔ
   2011-07-19
   5XJUUFSSVCZLBJHJ
   RubyKaigi2008-2011
   ࣮ߦҕһ
   

   View Slide

7. Information:
   RubyKaigi Nikki (RSS)
   Tags:
   For Twitter: #sprk2012
   For Blogs, Flickr: sappororubykaigi2012
   Previous:
   Sapporo RubyKaigi03
   ˜4BQQPSP3VCZ,BJHJ5FBN
   

   View Slide

8. OSPPOHBͰͭ͘Δ
   HSPPOHBશจݕࡧ
   "1*αʔό
   औక໾$50ઃᒜ༸࣐
   ZPKJTIJEBSB!FOJTIJUFDIDPN
   !EBSBTIJ
   — nroonga: a library for building groonga powered nodes.
   ೥݄೔౦ژOPEFֶԂݶ໨-JHIUOJOH5BML
   

   View Slide

9. An open-source fulltext search engine and column store
   

   View Slide

10. An open-source fulltext search engine and column store
    ߴ଎ͳશจݕࡧ
    ΧϥϜετΞ
    ಈతࡧҾߏங (ϦΞϧλΠϜݕࡧ!)
    ߴ଎ͳϑΝηοτݕࡧ
    Ң౓ܦ౓ݕࡧ ...
    

    View Slide

11. What’s nroonga?
    

    View Slide

12. meets
    Node.js is an official trademark of Joyent. This image is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
    

    View Slide

13. ͨͱ͑͹
    

    View Slide

14. ݕࡧόοΫΤϯυͰ
    ෳࡶͳूܭॲཧΛ͍ͨ͠ͱ͖
    

    View Slide

15. AMQP ܦ༝Ͱ
    Redis Pub/Sub ܦ༝Ͱ
    ZeroMQ ܦ༝Ͱ
    WebSocket ܦ༝Ͱ
    ...Ͱ
    ΠϯσΫε͍ͨ͠ͱ͖
    

    View Slide

16. ݕࡧAPIαʔόʹ
    ೝূΛ͔͚͍ͨͱ͖
    

    View Slide

17. ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIΛఏڙ͢Δ
    ݕࡧϊʔυΛͭ͘ΕΔͱḿΔ
    

    View Slide

18. ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIΛఏڙ͢Δ
    ݕࡧϊʔυΛͭ͘ΕΔͱḿΔ
    

    View Slide

19. ݕࡧϊʔυ?
    

    View Slide

20. ݕࡧnode!
    

    View Slide

21. meets
    Node.js is an official trademark of Joyent. This image is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.”
    

    View Slide

22. A library for building groonga powered nodes.
    

    View Slide

23. var nroonga = require('nroonga');
    var db = new nroonga.Database('database');
    // Synchronous
    console.log(db.commandSync('status'));
    // Asynchronous
    db.command('status', function(error, data) {
    console.log(data);
    });
    ೚ҙͷ groonga ίϚϯυΛ࣮ߦ
    database Λ༻ҙ
    

    View Slide

24. db.commandSync('table_create', {
    name: 'tweets',
    flags: 'TABLE_HASH_KEY',
    key_type: 'ShortText',
    });
    db.commandSync('column_create', {
    table: 'tweets',
    name: 'text',
    flags: 'COLUMN_SCALAR',
    type: 'ShortText'
    });
    ྫ) table ͱ column Λ࡞੒
    

    View Slide

25. commit 55ba20ffa7
    Latest commit to the master branch
    do not repat links
    darashi authored 19 days ago
    Tags Downloads
    nroonga
    nroonga /
    / nroonga
    nroonga
    Files Commits Branches
    13 1
    Admin
    Admin Unwatch
    Unwatch Fork
    Fork Pull Request
    Pull Request
    Code
    Code Network
    Network Pull Requests
    Pull Requests Issues
    Issues Wiki
    Wiki Stats & Graphs
    Stats & Graphs
    0
    0 1
    1 2
    2
    SSH
    SSH HTTP
    HTTP Git Read-Only
    Git Read-Only git@github.com:nroonga/nroonga.git Read+Write access
    A library for building groonga powered nodes — Read more
    http://nroonga.github.com
     Clone in Mac
     Clone in Mac ZIP
    ZIP
    6
    master
    master
    branch:
    branch: 1
    history
    history
    name
    name age
    age message
    message
    examples 3 months ago extract examples/server.coffee as nroonga-httpd [darashi]
    lib 3 months ago use messagepack as output format [darashi]
    license 4 months ago add license [darashi]
    src 3 months ago use messagepack as output format [darashi]
    test 5 months ago load data with one command call [darashi]
    nroonga /
    Explore
    Explore Gist
    Gist Blog
    Blog Help
    Help darashi
    darashi
    Search…
    2
    https://github.com/nroonga/nroonga
    

    View Slide

26. ඇಉظAPIΛ࢖͑͹
    ಉ࣌ʹΫΤϦ͕૸Δ
    

    View Slide

27. ௚ྻ࣮ߦ෦෼΋
    cluster Λ࢖͏ͱ
    ϚϧνίΞͰಈ࡞
    

    View Slide

28. groonga ͷ࢖͍ํ͸
    http://groonga.org/docs/
    ͳͲΛࢀর͍ͩ͘͞
    

    View Slide

29. nroonga ͕
    αϙʔτ͍ͯ͠Δͷ͸
    “ΫΤϦAPI” ͷΈ
    

    View Slide

30. εϐʔυͰ͸
    groonga αʔόʹ͸ఢΘͳ͍
    ><
    

    View Slide

31. Ұํ...
    

    View Slide

32. ଞͷ෦෼ͷ
    ΦʔόϔουʹΑͬͯ
    nroonga ͷΦʔόϔου͕
    ແࢹग़དྷΔ৔߹΋͋Δ
    

    View Slide

33. English | Japanese | mobile
    buzztter
    Snapshoot the tweets!
    iPhone ΁ͷ push Λࢼݧӡ༻தͰ͢ɻ boxcar ΛΠϯετʔϧ͠ɺbuzztter Λ௥Ճ͍ͯͩ͘͠͞ ( PC / iPhone )ɻ
    ࠶Քಇ ޷͖ͳͷΛҰਓબͿΜ͡Ό #͜ͷ #΄͍͠ύϯπڣΜͰ ݪൃ Ֆݟ શһ
    ϑΥϩʔ͢Δ *ƅ˜ƅ* ਓRT #1൪ಡΈͨ͘ͳ͍ϥϊϕλΠτϧߟ... ίφϯ RTͨ͠ਓ ਓશһ
    ϑΥϩʔ #RTͨ͠ 4݄ ૬༿ ͜ͷը૾ ͨΒRT ύϯμ ´Д` Λߋ৽͠·ͨ͠ ͨ·͝ य़य़य़य़य़य़य़य़य़य़
    य़य़य़य़य़य़य़... ;Θ;Θ Ψϯό ఆظ ࢟ͷ ϗϞ ෱ౡ Ί͠ ಈ෺ wwwwww ΊͬͪΌ ´ɾωɾʆ ؾܰʹ ʀ´Д ༳
    Ε ஥ྑ͘ ͷ΍ͭ ·͔͡ ៉ྷ ָ͔ͬͨ͠ ʈωʈ ՄѪ͍ ֦ࢄر๬ ༏͍͠ وํ ͔࣌Β ϗϫΠτ தଜ ΍͹͍ ͝൧ ઌഐ
    ΄Μ· ࢥͬͯΔ ʄƅ ͘Μͷ ^q^ ƅ∀ƅ ૬ޓ λά ͓લ ͲΕ όΠτ ༦൧ ձ͍͍ͨ ը૾Λ ·Ͱʹ ͩΑʔ Πέϝϯ
    ݕࡧ
    ͜Μͳݕࡧ͸͍͔͕Ͱ͔͢ʁ ࠶Քಇ OR ޷͖ͳͷΛҰਓબͿΜ͡Ό OR #͜ͷ
    buzz-phraseΛؚΉ࠷৽ͷൃݴ
    mutuki_rei ΪϧΨϝογϡͷࠇϏΩχɾɾɾ^̿^ɹ#΄͍͠ύϯπڣΜͰtlΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    kuroroMMC @hosakanobuto ඇৗʹྑ͍จষͰ͕͢ɺޡࣈΛݟ͚ͭ·ͨ͠ɻʮ2017೥ͷദ࡚מӋݪൃࣄނʯͰ͸ͳ͘
    ʮ2007೥ͷദ࡚מӋݪൃࣄނʯͰ͸ͳ͍Ͱ͠ΐ͏͔ɻ (1෼Ҏ಺લ)
    _lOth_ ΞϝϦΧŦťͷࠃضฑϘΫαʔ͓ͺΜͭ #΄͍͠ύϯπڣΜͰtlΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    sweet138 ೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻ #΄͍͠ύϯ
    πڣΜͰTLΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    uchan1umi RT @yuta_okrock: ONE OK ROCK#͜ͷόϯυ͕޷͖ͳϑΥϩϫʔ͞Μ͕RTͯ͜͠ΕΛݟͨϑΥϩϫʔͷ
    ϑΥϩϫʔ͞Μ͕ϑΥϩʔͯͦ͠ͷϑΥϩϫʔͷϑΥϩϫʔ͞Μ΋RTͯͯ͘͠͠ΕΔͱ͍͏ແݶεύΠϥϧ͕ى͜Γ
    ϑΥϩϫʔ͞Μ͕Ұؾʹ૿͑Δ" (1෼Ҏ಺લ)
    hisana0601 ʲఆظʳΠφζϚ(ແҹ/GO)ɼബࡩَɼBLEACHɼۜࠢɼ͏ͨϓϦɼBRAVE10ɼ·ͲϚΪɼϋΨϨϯɼε
    λεΧɼېॻ໨࿥ɼ͚͍͓Μ...etc ϑΥϩʔͤͯ͞௖͖·͢(o^^o) ग़དྷΕ͹૬ޓͰ #͜ͷதʹҰͭͰ΋޷͖ͳΞχϝ͕
    AKB48 ߚനର߅Վ
    ߹ઓ [DVD]
    AKB48
    ৽඼ ˇ3,650
    ϕετϓϥΠε
    ˇ2,968
    ڥքઢ্ͷϗϥΠ
    κϯ ʤHorizon on...
    ઒্ູ, ෱ࢁ५,...
    ৽඼ ˇ5,236
    ϕετϓϥΠε
    ˇ5,236
    Ṝ͍ΑΕ! χϟϧࢠ
    ͞Μ 3
    ௕ᖒ߶, Ѩ੅Ղ...
    ৽඼ ˇ5,439
    ϕετϓϥΠε
    ˇ5,439
    Ӊ஦ઓ؋ϠϚτ
    2199
    ग़ᔹ༟, ੁੜོ...
    ৽඼ ˇ5,236
    ϕετϓϥΠε
    ˇ5,236
    

    View Slide

34. Web frontend
    w/Rails
    searcher node
    indexer node
    Twitter
    HTTP streaming
    crawler node
    Redis pub/sub
    HTTP API
    Buzztter Search Architecture Outline
    groonga DB
    (shared)
    Internal API server
    nroonga
    nroonga
    

    View Slide

35. “ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIͷ
    ݕࡧαʔό͕΄͍͠”
    

    View Slide

36. A library for building groonga powered nodes.
    

    View Slide

37. OSPPOHBͰͭ͘Δ
    HSPPOHBશจݕࡧ
    "1*αʔό
    औక໾$50ઃᒜ༸࣐
    ZPKJTIJEBSB!FOJTIJUFDIDPN
    !EBSBTIJ
    — nroonga: a library for building groonga powered nodes.
    ೥݄೔౦ژOPEFֶԂݶ໨-JHIUOJOH5BML
    

    View Slide