Slide 1

Slide 1 text

W8LT#04 ࣗՈ੡ΦλΫιϦϡʔγϣϯͷ঺հ @tondol 2015-07-29 (Wed.)

Slide 2

Slide 2 text

Self Introduction @tondol ܭࢉ޻ֶઐ߈M2 ˓˓ݚڀࣨ ˠ௕໺ˠେࡕˠ޿ౡˠ໊ݹ԰ˠ ˠʢΠϚίίʣˠࡳຈˠઋ୆ˠ •  ୭͔50ສԁ͘Ε •  ࠓظͷϥϒϥΠϒʂ

Slide 3

Slide 3 text

Self Introduction @tondol ܭࢉ޻ֶઐ߈M2 ˓˓ݚڀࣨ ˠ௕໺ˠେࡕˠ޿ౡˠ໊ݹ԰ˠ ˠʢΠϚίίʣˠࡳຈˠઋ୆ˠ •  ୭͔50ສԁ͘Ε •  ࠓظͷϥϒϥΠϒʂ

Slide 4

Slide 4 text

എܠ •  ΦλΫ͸ଟ๩ – ΠϕϯτࢀՃ – ΞϧόΠτ – ݚڀ – W8LTͷࢿྉ࡞੒ •  ࡏ୐ίϯςϯπ – ΞχϝɾϥδΦɾχίੜ౳ – ϦΞϧλΠϜʹফԽͰ͖ͳ͍͜ͱ΋͋Δ – ظؒݶఆίϯςϯπͷݫ͠͞

Slide 5

Slide 5 text

Archive everything you want

Slide 6

Slide 6 text

͜ͷੈʹଘࡏ͢Δ͢΂ ͯͷظؒݶఆίϯςϯ πΛΞʔΧΠϒ͍ͨ͠

Slide 7

Slide 7 text

ఏҊ •  NicoAnime –  χίχίͷΞχϝνϟϯωϧΛొ࿥ –  ֤Τϐιʔυ͕ࣗಈΞʔΧΠϒ͞ΕΔ •  NicoKako –  ಛఆͷχίχίੜ์ૹΛొ࿥ –  λΠϜγϑτػೳͰμ΢ϯϩʔυ •  koeradi –  ಛఆͷωοτϥδΦΛ࿥Ի •  ֤αʔϏεͷن໿తʹ͸……ʁ

Slide 8

Slide 8 text

NicoAnime

Slide 9

Slide 9 text

֓ཁ •  Ϣʔβʔ͕νϟϯωϧΛొ࿥ – ֤Ϋʔϧͷॳճ഑৴ظؒʹ΍Δ •  ఆظతʹcrawler͕ొ࿥ಈըΛ֬ೝ – ߋ৽෼͕͋Ε͹DLϦετʹೖΕΔ •  ఆظతʹdownloader͕ಈըΛDL – DLϦετʹ͋Δ΋ͷΛΞʔΧΠϒ͢Δ •  Ϣʔβʔͷૢ࡞͸΢ΣϒͰ׬݁ – ಈըͷ࠶ੜɾϩʔΧϧ΁ͷDL΋Մೳ

Slide 10

Slide 10 text

σϞ

Slide 11

Slide 11 text

ߏ੒ crawler (Ruby) downloader (Ruby) Niconico Live DB (MySQL) frontend (PHP) file system crontab nginx, php-fpm

Slide 12

Slide 12 text

ٕज़తৄࡉ(1) •  Niconico΁ͷϩάΠϯ – secure.nicovideo.jp/secure/login – ্هʹIDͱύεϫʔυΛPOST͢Δ – Ϩεϙϯεͷset-cookieϔομ͔Β user_session஋Λऔಘ͠هԱ͢Δ •  getflv/getthumbinfo – ಈըϝλσʔλΛऔಘ͢ΔAPI – user_sessionΛcookieͰૹ৴ͭͭ͠get – GETύϥϝʔλ/XMLϑΥʔϚοτ

Slide 13

Slide 13 text

ٕज़తৄࡉ(2) •  watch – user_sessionΛcookieͰૹ৴ͭͭ͠GET – Ϩεϙϯεͷset-cookieϔομ͔Β history஋Λऔಘ͠هԱ͢Δ •  μ΢ϯϩʔυ – user_session, historyΛcookieͰૹ৴ͭͭ͠ getflvͰऔಘͨ͠ಈըURLΛGET

Slide 14

Slide 14 text

ิ଍ •  Niconico APIʹؔ͢Δৄࡉ – https://github.com/tondol/NicoVideo •  ࿈ଓ࠶ੜ੍ݶͱ͔ը࣭ͱ͔ ϓϨϛΞϜ͡Όͳ͍ͱΩπ͍ͱࢥ͏ – ݄500ԁ͸ޡࠩ

Slide 15

Slide 15 text

NicoKako

Slide 16

Slide 16 text

֓ཁ •  Ϣʔβʔ͕χίχίੜ์ૹΛొ࿥ – λΠϜγϑτՄೳظؒʹ΍Δ – DLϦετʹ௥Ճ͞ΕΔ •  ఆظతʹdownloader͕ಈըΛDL – DLϦετʹ͋Δ΋ͷΛΞʔΧΠϒ͢Δ •  Ϣʔβʔͷૢ࡞͸΢ΣϒͰ׬݁ – ಈըͷ࠶ੜɾϩʔΧϧ΁ͷDL΋Մೳ – λΠϜγϑτ༧໿؅ཧػೳ΋

Slide 17

Slide 17 text

σϞ

Slide 18

Slide 18 text

ߏ੒ downloader (Ruby) Niconico Live DB (MySQL) frontend (PHP) file system crontab nginx, php-fpm

Slide 19

Slide 19 text

ٕज़తৄࡉ(1) •  Niconico΁ͷϩάΠϯ – লུ •  getplayerstatus – χίੜͷϝλσʔλΛऔಘ͢ΔAPI – user_sessionΛcookieͰૹ৴ͭͭ͠GET – XMLϑΥʔϚοτ – λΠϜγϑτDLʹඞཁͳύϥϝʔλͳͲ

Slide 20

Slide 20 text

ٕज़తৄࡉ(2) •  λΠϜγϑτDL –  rtmpeϓϩτίϧͰͷ഑৴ –  getplayerstatusͷϨεϙϯεΛݩʹDLʹඞཁͳ ύϥϝʔλΛߏ੒͢Δ –  ύϥϝʔλΛrtmpdumpʹ౉͢ –  DLঢ়گ͕100%ʹͳΔ·ͰϨδϡʔϜ •  ஫ҙ఺ –  ظݶ੾Εͷੜ์ૹ͸DLෆՄ –  ՝ۚࡁΈͳΒ༗ྉੜ์ૹ΋ಉ͡ํ๏ͰDLՄೳ

Slide 21

Slide 21 text

ิ଍ •  λΠϜγϑτDLʹؔ͢Δৄࡉ – https://gist.github.com/tondol/7039259 •  λΠϜγϑτ༧໿ͷ্ݶͱ͔λΠϛϯά ͱ͔ϓϨϛΞϜ͡Όͳ͍ͱΩπ͍ͱࢥ͏ – ݄500ԁ͸ޡࠩ

Slide 22

Slide 22 text

koeradi

Slide 23

Slide 23 text

֓ཁ •  ωοτϥδΦΛDL͢ΔεΫϦϓτ܊ – ରԠہɿ ௒A&GɾڹɾϦεϥδɾradikoɾϫϩοϓ – ؆қWeb UIΛඋ͑Δ •  ࿥Ի͍ͨ͠์ૹຖʹcrontabʹهड़ – ִि഑৴ͱ͔͸໘౗͔ͩΒඇରԠ – ݱࡏ14ϓϩάϥϜ/ि΄ͲΞʔΧΠϒӡ༻த

Slide 24

Slide 24 text

σϞ

Slide 25

Slide 25 text

ߏ੒ downloader(s) (Ruby/bash) net radio server(s) frontend (PHP) file system crontab nginx, php-fpm

Slide 26

Slide 26 text

ٕज़తৄࡉ •  ࢖༻πʔϧ – rtmpdump/ffmpeg/swftools – rtmpdump: rtmpϓϩτίϧʹΑΔDL – ffmpeg: HLSϓϩτίϧʹΑΔDL, ม׵ – swftools: radikoͷpartialkeyऔಘ༻ •  εΫϦϓτ – RubyͩͬͨΓbashͩͬͨΓ – ઃఆϑΝΠϧ͸dotenvʹهड़

Slide 27

Slide 27 text

ิ଍ •  radikoʹؔ͢Δৄࡉ – http://kyoshiaki.hatenablog.com/entry/ 2014/05/04/184748 •  ஍ํͷϥδΦΛฉ͘ͳΒ radikoϓϨϛΞϜΛܖ໿͠·͠ΐ͏ – ݄300ԁ͸ޡࠩ

Slide 28

Slide 28 text

NicoAnime NicoKako koeradi

Slide 29

Slide 29 text

Q and A?