Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ikyu_hack_camp_#1

kentana20
August 10, 2014

 ikyu_hack_camp_#1

開発合宿#1 @ 日光はじめのいっぽ
Node.js でスクレイピングバッチ+Webアプリを作った

kentana20

August 10, 2014
Tweet

More Decks by kentana20

Other Decks in Technology

Transcript

  1. ։ൃ߹॓#1
    ͸͡Ίͷ͍ͬΆ 2014.08.08
    @kentana20

    View Slide

  2. Agenda
    ϨΪϡϨʔγϣϯʢ֬ೝʣ
    ࠾༻ٕͨ͠ज़
    ࠓճͷ࡞඼ίϯηϓτ
    ࡞ͬͨ΋ͷ
    Batch
    Web
    Node.jsॴײ
    ·ͱΊ

    View Slide

  3. ϨΪϡϨʔγϣϯ
    ͋·Γ৮͍ͬͯͳ͍ݴޠɾϑϨʔϜϫʔΫΛ৮Δ
    ࣾ಺ͷҰ෦αʔϏεΛ૝ఆͨ͠ҎԼͷॲཧϓϩ
    άϥϜΛ࡞Δ
    Batch:εΫϨΠϐϯάॲཧʴDBొ࿥
    Web:Batchॲཧͯ͠औಘͨ͠σʔλͷදࣔ

    View Slide

  4. ࠾༻ٕͨ͠ज़
    Node.js
    Batch/Webͱ΋ʹNode.js
    Node.js͸npmύοέʔδ͕๛෋ͰεΫϨΠϐϯάɺόον
    ॲཧʢcronʣɺDBΫϥΠΞϯτɺWebϑϨʔϜϫʔΫͱଟ
    ࠼ͳϥΠϒϥϦ܈͕ັྗ
    Redis
    ΠϯϝϞϦܕNoSQL

    View Slide

  5. ࠓճͷ࡞඼ίϯηϓτ
    ʮඒঁʯΛ୊ࡐʹͨ͠ΞϓϦ
    1ധ2೔Ͱ௕࣌ؒͷ࡞ۀ͕༧૝͞ΕͨͷͰɺπϥ
    ͘ͳ͍୊ࡐΛબ୒
    ਖ਼ৗʹऔಘɾදࣔ͞Εͨͱ͖ʹ༊΍͞ΕΔ
    ʮඒঁྐྵʯ͔Βඒঁը૾ͱϓϩϑΟʔϧΛεΫϨ
    ΠϐϯάͰऔಘ

    View Slide

  6. ࡞ͬͨ΋ͷ
    Batch
    `cheerio-httpcli` ͱ͍͏npmΛ࠾༻ʢ΋΍΋΍ΤϯδχΞblog࠷
    ߴʣ
    σʔλͷอଘ͸RedisΛ࠾༻ʢطʹOS Xʹೖ͔ͬͯͨΒʣ
    jQueryϥΠΫʹεΫϨΠϐϯάॲཧ͕ॻ͚Δ͙͢Ε΋ͷ
    CasperJSΛ࢖͍͔͕ͨͬͨɺNode.jsͰಈ࡞ͤ͞Δͷʹ
    SpookyJS͕ඞཁͱ͍͏͜ͱΛࣦ೦͍ͯͯ͠λΠϜϩε

    View Slide

  7. ࡞ͬͨ΋ͷ
    Web
    `Express` ͱ͍͏WebϑϨʔϜϫʔΫΛ࠾༻
    ΠϚυΩͷWebϑϨʔϜϫʔΫͰɺejsͱ͍͏Rails
    Ͱ͍͏ͱ͜ΖͷerbతͳwebςϯϓϨʔτ͕࢖͑
    ͯεοͱೖΕͨ
    ͕࣌ؒͳͯ͘ϓϨʔϯա͗ΔUIʹͳͬͨ

    View Slide

  8. ΍Γ͔ͨͬͨ͜ͱɾ৺࢒Γ
    Q (ඇಉظ৐Γ͜ͳͤͳ͔ͬͨʣ
    ඒঁͷϓϩϑΟʔϧΛ΋ͬͱऔಘ͔ͨͬͨ͠ʢliཁૉ͕ՄมͰid/
    class྆ํͳͯ͘”൴ࢯͷ༗ແ”͕औΕͳ͔ͬͨ͜ͱ͕৺࢒Γʣ
    WebϖʔδΛඒঁʹ;͞Θ͘͠ɺ΋ͬͱ՚΍͔ͳUIʹ͔ͨͬͨ͠
    ʢBatchͷ࣮૷ʹֻ͚࣌ؒ͗ͯ͢ɺWebͷ࣮૷ʹ2࣌ؒ͘Β͍͔͠ͱ
    Εͳ͔ͬͨɻ࣌ؒ഑෼ॏཁʣ
    SpookyJSͰσʔλεΫϨΠϐϯά͢Δͷ͸ָͦ͠͏ͳͷͰɺͦͬͪ
    ʹτϥΠͯ͠Έ͍ͨ

    View Slide

  9. Node.jsॴײ
    JavascriptϥΫʢ͚Ͳ΍ͬͺΓه๏޷͖͡Όͳ͍ʣ
    JavascriptͰ͢΂ͯ׬݁Ͱ͖ΔͷͰɺ؀ڥߏஙؚΊͯಋೖίε
    τɾֶशίετ௿͍
    npm͕๛෋ͳͷͰɺΠϚυΩͷWebΞϓϦͰ΍͍ͬͯΔ͜ͱ͸
    ͍͍ͩͨΧόʔͰ͖Δ
    γϯάϧεϨουɾඇಉظΛ৐Γ͜ͳ͢׮༰ͳਫ਼ਆ͕ඞཁ
    झຯͷΞϓϦͳΒશવީิʹೖͬͯ͘Δ͕B2C޲͚ͷ͋Δఔ౓
    ͷτϥϑΟοΫ͕༧૝͞ΕΔαʔϏεӡ༻͸ະ஌਺

    View Slide

  10. ։ൃ߹॓ॴײ
    ूதͯ͠ɺීஈͷੜ׆͔Β཭Εͯ։ൃʹ޲͖߹͏࣌ؒ
    ͸࠷ߴ
    ࠓճ͸͓ࢼ͠ʴ৽ݴޠ΁ͷτϥΠͩͬͨͷͰ1ധ2೔Ͱ
    ΋ྑ͔͕ͬͨɺ࢓ࣄͰ࣮ࡍʹ։ൃ߹॓͢ΔͳΒ2~3ധ
    ͘Β͍ͯ͠ɺΨοπϦ։ൃ͢Ε͹ɺ͔ͳΓॆ࣮ͨ͠੒
    ՌΛग़ͤͦ͏
    ఆظతʹ։࠵͍ͨ͠ʢ೥2ճͱ͔ʣ

    View Slide

  11. Enjoy Coding!

    View Slide