2016/07/03開催のF#談話室(23)の発表資料です。
F# SCRAPINGF#ͰεΫϨΠϐϯάΛͯ͠Έͨʂcallmekohei2016/07/03 Sun
View Slide
Overview• ࣗݾհ• εΫϨΠϐϯάͷ͖͔͚ͬ• HTMLͷ2ͭͷऔಘํ๏• HTML͔ΒσʔλʔΛͱΓͩͯ͠ΈΔ
callmkoheiVBA3 yearsF#9 monthsͷ໊લϨΦɻΑΖ͘͠Ͷʂ
εΫϨΠϐϯάͷ͖͔͚ͬ
͜Μͳײ͡Ͱϩτ̓ͷ༧ଌΛ͍ͨ͠
ϩτ̓ͷબ൪߸ΛϗʔϜϖʔδΑΓೖख͍ͨ͠
ϗʔϜϖʔδHTMLιʔεͰͰ͖͍ͯΔ
ϗʔϜϖʔδͷσʔλʔΛऔಘ͢ΔʹHTMLιʔε͕ඞཁ
Ͳ͏ͬͯHTMLιʔεΛऔಘ͢Δ͔
HTMLιʔεΛऔಘ͢Δ̎ͭͷํ๏
ͦͷ̍System.NetΛ͔ͭ͏
ͦͷ̎PhantomJS Λ͔ͭ͏
جຊతʹSystem.NetͷΈͰେৎʢͩͱࢥ;ɻɻɻʣ͜͜ߟ͑Ͳ͜Ζ
Ͳ͏্ͯ͠ख͘औಘͰ͖ͳ͍߹PhantomJSΛͬͯΈΔ
ͨͩPhantomJS͍ͷͰ͢
αʔόʔͷଓൺֱSystem.Net: 2sPhantomJS: 7s࣮ߦ࣌ؒࢀߟ
͜͜Ͱ࣮ࡍʹαʔόʔʹͭͳ͛ͯΈΔdemo
HTMLιʔε͔ΒσʔλʔΛͱΓͩͯ͠ΈΔ
͜͜ͰศརͳϥΠϒϥϦFSharp.Data
FSharp Data ͱʁThe F# Data library implementseverything you need to access datain your F# applications and scripts.CSV, HTML, JSON and XMLʹରͯ͠ͷศརϥΠϒϥϦhttp://fsharp.github.io/FSharp.Data/
a tag
͜͜Ͱa tag ΛͱΓͩͯ͠ΈΔdemo
table tag
͜͜Ͱtable tag ΛͱΓͩͯ͠ΈΔdemo
εΫϨΠϐϯά͢Δͱ͖ʹศརͩͱࢥ͏ؔΛ·ͱΊͯΈͨ
SCRAPINGfshttps://github.com/callmekohei/SCRAPINGfs
͞Βʹɺɺ
FSharp.DataHtmlDocument.Load͍͚ͯͳ͍ͱࢥͬͯΔͱ͜Ζ
ϒϥβʔ͡Όͳ͍ͱ͔͡ΕΔจࣈίʔυ͕͏·͘ॲཧ͞Εͳ͍Http Utilities Λ͏͜ͱͰղܾʂ
Formॲཧ͕Ͱ͖ͳ͍
Formॲཧ͕Ͱ͖ͳ͍Http Utilities Λ͏͜ͱͰͪΐͬͱղܾʂ
Formॲཧ͕Ͱ͖ͳ͍System.Net Λ͏͜ͱͰղܾʂ
؆୯ͳൺֱද
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠