Upgrade to Pro — share decks privately, control downloads, hide ads and more …

nroongaでつくる groonga 全文検索 APIサーバ

nroongaでつくる groonga 全文検索 APIサーバ

2012年4月18日 東京Node学園 5限目 Lightning Talk

Yoji Shidara

April 18, 2012
Tweet

More Decks by Yoji Shidara

Other Decks in Technology

Transcript

  1. OSPPOHBͰͭ͘Δ
    HSPPOHBશจݕࡧ
    "1*αʔό
    औక໾$50ઃᒜ༸࣐
    ZPKJTIJEBSB!FOJTIJUFDIDPN
    !EBSBTIJ
    — nroonga: a library for building groonga powered nodes.
    ೥݄೔౦ژ/PEFֶԂݶ໨-JHIUOJOH5BML

    View Slide

  2. ͩ͠ΒΑ͏͡
    ઃᒜ༸࣐

    View Slide

  3. ๺ւಓ͸ࡳຈ͔Βདྷ·ͨ͠ɻ

    View Slide

  4. ΦʔϓϯιʔεΧϯϑΝϨϯε)PLLBJEPʹڠࢍ͠·͢


    $44/JUFJO4FPVM 7PMʹͯ౰ࣾϝϯόʔ͕ൃද͠·͢


    View Slide

  5. “সإͷͨ͑ͳ͍”

    View Slide

  6. αΠϯΠϯ
    &OHMJTIc೔ຊޠ
    ೔ຊ3VCZձٞʹ͍ͭͯ νέοτߪೖɾ֤छ͓ਃࠐΈ ࢀՃऀ޲͚৘ใ εϙϯαʔ
    2011-07-19
    5XJUUFSSVCZLBJHJ
    RubyKaigi2008-2011
    ࣮ߦҕһ

    View Slide

  7. Information:
    RubyKaigi Nikki (RSS)
    Tags:
    For Twitter: #sprk2012
    For Blogs, Flickr: sappororubykaigi2012
    Previous:
    Sapporo RubyKaigi03
    ˜4BQQPSP3VCZ,BJHJ5FBN

    View Slide

  8. OSPPOHBͰͭ͘Δ
    HSPPOHBશจݕࡧ
    "1*αʔό
    औక໾$50ઃᒜ༸࣐
    ZPKJTIJEBSB!FOJTIJUFDIDPN
    !EBSBTIJ
    — nroonga: a library for building groonga powered nodes.
    ೥݄೔౦ژOPEFֶԂݶ໨-JHIUOJOH5BML

    View Slide

  9. An open-source fulltext search engine and column store

    View Slide

  10. An open-source fulltext search engine and column store
    ߴ଎ͳશจݕࡧ
    ΧϥϜετΞ
    ಈతࡧҾߏங (ϦΞϧλΠϜݕࡧ!)
    ߴ଎ͳϑΝηοτݕࡧ
    Ң౓ܦ౓ݕࡧ ...

    View Slide

  11. What’s nroonga?

    View Slide

  12. meets
    Node.js is an official trademark of Joyent. This image is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.

    View Slide

  13. ͨͱ͑͹

    View Slide

  14. ݕࡧόοΫΤϯυͰ
    ෳࡶͳूܭॲཧΛ͍ͨ͠ͱ͖

    View Slide

  15. AMQP ܦ༝Ͱ
    Redis Pub/Sub ܦ༝Ͱ
    ZeroMQ ܦ༝Ͱ
    WebSocket ܦ༝Ͱ
    ...Ͱ
    ΠϯσΫε͍ͨ͠ͱ͖

    View Slide

  16. ݕࡧAPIαʔόʹ
    ೝূΛ͔͚͍ͨͱ͖

    View Slide

  17. ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIΛఏڙ͢Δ
    ݕࡧϊʔυΛͭ͘ΕΔͱḿΔ

    View Slide

  18. ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIΛఏڙ͢Δ
    ݕࡧϊʔυΛͭ͘ΕΔͱḿΔ

    View Slide

  19. ݕࡧϊʔυ?

    View Slide

  20. ݕࡧnode!

    View Slide

  21. meets
    Node.js is an official trademark of Joyent. This image is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.”

    View Slide

  22. A library for building groonga powered nodes.

    View Slide

  23. var nroonga = require('nroonga');
    var db = new nroonga.Database('database');
    // Synchronous
    console.log(db.commandSync('status'));
    // Asynchronous
    db.command('status', function(error, data) {
    console.log(data);
    });
    ೚ҙͷ groonga ίϚϯυΛ࣮ߦ
    database Λ༻ҙ

    View Slide

  24. db.commandSync('table_create', {
    name: 'tweets',
    flags: 'TABLE_HASH_KEY',
    key_type: 'ShortText',
    });
    db.commandSync('column_create', {
    table: 'tweets',
    name: 'text',
    flags: 'COLUMN_SCALAR',
    type: 'ShortText'
    });
    ྫ) table ͱ column Λ࡞੒

    View Slide

  25. commit 55ba20ffa7
    Latest commit to the master branch
    do not repat links
    darashi authored 19 days ago
    Tags Downloads
    nroonga
    nroonga /
    / nroonga
    nroonga
    Files Commits Branches
    13 1
    Admin
    Admin Unwatch
    Unwatch Fork
    Fork Pull Request
    Pull Request
    Code
    Code Network
    Network Pull Requests
    Pull Requests Issues
    Issues Wiki
    Wiki Stats & Graphs
    Stats & Graphs
    0
    0 1
    1 2
    2
    SSH
    SSH HTTP
    HTTP Git Read-Only
    Git Read-Only [email protected]:nroonga/nroonga.git Read+Write access
    A library for building groonga powered nodes — Read more
    http://nroonga.github.com
     Clone in Mac
     Clone in Mac ZIP
    ZIP
    6
    master
    master
    branch:
    branch: 1
    history
    history
    name
    name age
    age message
    message
    examples 3 months ago extract examples/server.coffee as nroonga-httpd [darashi]
    lib 3 months ago use messagepack as output format [darashi]
    license 4 months ago add license [darashi]
    src 3 months ago use messagepack as output format [darashi]
    test 5 months ago load data with one command call [darashi]
    nroonga /
    Explore
    Explore Gist
    Gist Blog
    Blog Help
    Help darashi
    darashi
    Search…
    2
    https://github.com/nroonga/nroonga

    View Slide

  26. ඇಉظAPIΛ࢖͑͹
    ಉ࣌ʹΫΤϦ͕૸Δ

    View Slide

  27. ௚ྻ࣮ߦ෦෼΋
    cluster Λ࢖͏ͱ
    ϚϧνίΞͰಈ࡞

    View Slide

  28. groonga ͷ࢖͍ํ͸
    http://groonga.org/docs/
    ͳͲΛࢀর͍ͩ͘͞

    View Slide

  29. nroonga ͕
    αϙʔτ͍ͯ͠Δͷ͸
    “ΫΤϦAPI” ͷΈ

    View Slide

  30. εϐʔυͰ͸
    groonga αʔόʹ͸ఢΘͳ͍
    ><

    View Slide

  31. Ұํ...

    View Slide

  32. ଞͷ෦෼ͷ
    ΦʔόϔουʹΑͬͯ
    nroonga ͷΦʔόϔου͕
    ແࢹग़དྷΔ৔߹΋͋Δ

    View Slide

  33. English | Japanese | mobile
    buzztter
    Snapshoot the tweets!
    iPhone ΁ͷ push Λࢼݧӡ༻தͰ͢ɻ boxcar ΛΠϯετʔϧ͠ɺbuzztter Λ௥Ճ͍ͯͩ͘͠͞ ( PC / iPhone )ɻ
    ࠶Քಇ ޷͖ͳͷΛҰਓબͿΜ͡Ό #͜ͷ #΄͍͠ύϯπڣΜͰ ݪൃ Ֆݟ શһ
    ϑΥϩʔ͢Δ *ƅ˜ƅ* ਓRT #1൪ಡΈͨ͘ͳ͍ϥϊϕλΠτϧߟ... ίφϯ RTͨ͠ਓ ਓશһ
    ϑΥϩʔ #RTͨ͠ 4݄ ૬༿ ͜ͷը૾ ͨΒRT ύϯμ ´Д` Λߋ৽͠·ͨ͠ ͨ·͝ य़य़य़य़य़य़य़य़य़य़
    य़य़य़य़य़य़य़... ;Θ;Θ Ψϯό ఆظ ࢟ͷ ϗϞ ෱ౡ Ί͠ ಈ෺ wwwwww ΊͬͪΌ ´ɾωɾʆ ؾܰʹ ʀ´Д ༳
    Ε ஥ྑ͘ ͷ΍ͭ ·͔͡ ៉ྷ ָ͔ͬͨ͠ ʈωʈ ՄѪ͍ ֦ࢄر๬ ༏͍͠ وํ ͔࣌Β ϗϫΠτ தଜ ΍͹͍ ͝൧ ઌഐ
    ΄Μ· ࢥͬͯΔ ʄƅ ͘Μͷ ^q^ ƅ∀ƅ ૬ޓ λά ͓લ ͲΕ όΠτ ༦൧ ձ͍͍ͨ ը૾Λ ·Ͱʹ ͩΑʔ Πέϝϯ
    ݕࡧ
    ͜Μͳݕࡧ͸͍͔͕Ͱ͔͢ʁ ࠶Քಇ OR ޷͖ͳͷΛҰਓબͿΜ͡Ό OR #͜ͷ
    buzz-phraseΛؚΉ࠷৽ͷൃݴ
    mutuki_rei ΪϧΨϝογϡͷࠇϏΩχɾɾɾ^̿^ɹ#΄͍͠ύϯπڣΜͰtlΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    kuroroMMC @hosakanobuto ඇৗʹྑ͍จষͰ͕͢ɺޡࣈΛݟ͚ͭ·ͨ͠ɻʮ2017೥ͷദ࡚מӋݪൃࣄނʯͰ͸ͳ͘
    ʮ2007೥ͷദ࡚מӋݪൃࣄނʯͰ͸ͳ͍Ͱ͠ΐ͏͔ɻ (1෼Ҏ಺લ)
    _lOth_ ΞϝϦΧŦťͷࠃضฑϘΫαʔ͓ͺΜͭ #΄͍͠ύϯπڣΜͰtlΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    sweet138 ೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻೆӢͷύϯπɻ #΄͍͠ύϯ
    πڣΜͰTLΛύϯπͰຒΊਚͦ͘͏ͥ (1෼Ҏ಺લ)
    uchan1umi RT @yuta_okrock: ONE OK ROCK#͜ͷόϯυ͕޷͖ͳϑΥϩϫʔ͞Μ͕RTͯ͜͠ΕΛݟͨϑΥϩϫʔͷ
    ϑΥϩϫʔ͞Μ͕ϑΥϩʔͯͦ͠ͷϑΥϩϫʔͷϑΥϩϫʔ͞Μ΋RTͯͯ͘͠͠ΕΔͱ͍͏ແݶεύΠϥϧ͕ى͜Γ
    ϑΥϩϫʔ͞Μ͕Ұؾʹ૿͑Δ" (1෼Ҏ಺લ)
    hisana0601 ʲఆظʳΠφζϚ(ແҹ/GO)ɼബࡩَɼBLEACHɼۜࠢɼ͏ͨϓϦɼBRAVE10ɼ·ͲϚΪɼϋΨϨϯɼε
    λεΧɼېॻ໨࿥ɼ͚͍͓Μ...etc ϑΥϩʔͤͯ͞௖͖·͢(o^^o) ग़དྷΕ͹૬ޓͰ #͜ͷதʹҰͭͰ΋޷͖ͳΞχϝ͕
    AKB48 ߚനର߅Վ
    ߹ઓ [DVD]
    AKB48
    ৽඼ ˇ3,650
    ϕετϓϥΠε
    ˇ2,968
    ڥքઢ্ͷϗϥΠ
    κϯ ʤHorizon on...
    ઒্ູ, ෱ࢁ५,...
    ৽඼ ˇ5,236
    ϕετϓϥΠε
    ˇ5,236
    Ṝ͍ΑΕ! χϟϧࢠ
    ͞Μ 3
    ௕ᖒ߶, Ѩ੅Ղ...
    ৽඼ ˇ5,439
    ϕετϓϥΠε
    ˇ5,439
    Ӊ஦ઓ؋ϠϚτ
    2199
    ग़ᔹ༟, ੁੜོ...
    ৽඼ ˇ5,236
    ϕετϓϥΠε
    ˇ5,236

    View Slide

  34. Web frontend
    w/Rails
    searcher node
    indexer node
    Twitter
    HTTP streaming
    crawler node
    Redis pub/sub
    HTTP API
    Buzztter Search Architecture Outline
    groonga DB
    (shared)
    Internal API server
    nroonga
    nroonga

    View Slide

  35. “ΞϓϦέʔγϣϯʹಛԽͨ͠
    γϯϓϧͳAPIͷ
    ݕࡧαʔό͕΄͍͠”

    View Slide

  36. A library for building groonga powered nodes.

    View Slide

  37. OSPPOHBͰͭ͘Δ
    HSPPOHBશจݕࡧ
    "1*αʔό
    औక໾$50ઃᒜ༸࣐
    ZPKJTIJEBSB!FOJTIJUFDIDPN
    !EBSBTIJ
    — nroonga: a library for building groonga powered nodes.
    ೥݄೔౦ژOPEFֶԂݶ໨-JHIUOJOH5BML

    View Slide