WebView からデータをスクレイピングする

React Native 祭での LT に使った資料です。
https://reactnative-matsuri.com/ja

Transcript

1. WebView ͔Β σʔλΛεΫϨΠϐϯά͢Δ Tatsunori Nakano
2. ࣗݾ঺հ • Tatsunori Nakano • ࡳຈग़਎ → ౦ژ • ฏ೔͸౎಺ͷ Web ܥͷձࣾͰ React Λॻ͍͍ͯ·͢ɻ • ࠓ೥͔Β React Native ৮Γ࢝Ίɺि຤ʹΞϓϦΛ࡞ͬͨΓ͍ͯ͠·͢ɻ
3. ࿩͢͜ͱ • ࡞੒ͨ͠ΞϓϦͷ঺հ • WebView ͱΞϓϦͷσʔλͷ΍ΓऔΓʹ͍ͭͯ
4. ϋΠϥΠτɾϚωʔδϟʔ ɾKindle ͷϋΠϥΠτΛΠϯϙʔτ ɾෳ਺ͷຊͷϋΠϥΠτΛΞϓϦ಺ͰҰݩ؅ཧ ɾϥϯμϜදࣔ΍ϦϚΠϯυػೳͰهԱΛఆண
5. CSV ΤΫεϙʔτ PDF ΤΫεϙʔτ ݩʑ Kindle ʹ͸ΤΫεϙʔτػೳ͕උΘ͍ͬͯΔ ͕ɺΞϓϦͰ͸ѻ͍ʹ͍͘😥
6. Kindle ͷϋΠϥΠτ͸ϒϥ΢β͔ΒΞΫηεͰ͖Δ WebView Ͱ։͍ͯϋΠϥΠτΛ௚઀औಘ͢Δ💡
7. WebView ͰʮϝϞͱϋΠϥΠτʯΛ։͘ ຊͷҰཡ৘ใΛऔಘͯ͠දࣔ͢Δ બ୒ͨ͠ຊͷϋΠϥΠτΛऔಘ͢Δ
8. document.querySelectorAll("a.a-text-normal").forEach((a,index) => console.log(index,a.text)) ※ ຊͷλΠτϧҰཡΛऔಘ
9. injectJavaScript() • WebView ্Ͱ JavaScript ͷίʔυΛ࣮ߦ͢Δ • Ҿ਺ʹ࣮ߦ͍ͨ͠ JavaScript ͷίʔυΛ string Ͱ౉͢ • WebView ʹ ref Λઃఆ͠ɺͦͷ ref ʹର࣮ͯ͠ߦ͢Δ // ྫɿഎܠΛ੨͘͢Δ webviewRef.current.injectJavaScript("document.body.style.backgroundColor = 'blue'")
10. window.ReactNativeWebView.postMessage() • WebView ͔ΒΞϓϦଆ΁ϝοηʔδΛ౤͛Δ • Ҿ਺ʹ౤͍͛ͨϝοηʔδΛ string Ͱ౉͢ // ྫɿΞϓϦଆʹ "Hello" ͱ͍͏จࣈྻΛ౤͛Δ window.ReactNativeWebView.postMessage("Hello!")
11. onMessage • WebView ͷϓϩύςΟ • postMessage() ʹΑͬͯඈ͹͞ΕͨϝοηʔδΛड͚औΔ // ྫɿඈΜͰ͖ͨϝοηʔδΛ alert Ͱදࣔ͢Δ return ( <View style={{ flex: 1 }}> <WebView source={{ html }} onMessage={(event) => { alert(event.nativeEvent.data); }} /> </View> );
12. ɾinjectJavaScript() ɾwindow.ReactNativeWebView.postMessage() ɾonMessage ͜ͷ 3 ͭΛ૊Έ߹Θͤͯ WebView ͔Β৘ใΛऔಘ͠·͢
13. document.querySelectorAll("a.a-text-normal").forEach((a,index) => console.log(index,a.text)) ※ ຊͷλΠτϧҰཡΛऔಘ
14. document.querySelectorAll("a.a-text-normal").forEach((a,index) => console.log(index,a.text)) document.querySelectorAll("a.a-text-normal").forEach((a,index) => window.ReactNativeWebView.postMessage([index,a.text])) const getTitleList = 'document.querySelectorAll("a.a-text-normal").forEach((a,index) => window.ReactNativeWebView.postMessage([index,a.text]))' // WebView ͷ ref const webviewRef = useRef<WebView>(null!); // JavaScript ࣮ߦ༻ͷؔ਺ const runJS = (code: string) => { webviewRef.current.injectJavaScript(code); }; return ( <SafeAreaView> <WebView source={{ uri: 'https://read.amazon.co.jp/notebook' }} onNavigationStateChange={() => runJS(getTitleList)} onMessage={(event) => { console.log(event.nativeEvent.data) }} ref={webviewRef} /> </SafeAreaView> );
15. string Λ Ճ޻ͯ͠ը໘ʹඳը͢Δ
16. // WebView ಺Ͱ࣮ߦͤ͞Δ JavaScript ͷίʔυ const codes = { getTitleList: 'document.querySelectorAll("a.a-text-normal").forEach((a,i) => window.ReactNativeWebView.postMessage([i,a.text]))', getAsinList: 'document.querySelectorAll("#kp-notebook-library>div").forEach((div, i)=> window.ReactNativeWebView.postMessage([i,div.id,"asin"]))', click: 'document.querySelectorAll("a.a-text-normal")[number].click()', loadingCheck: 'window.ReactNativeWebView.postMessage(document.querySelector("div#kp-notebook-spinner.kp-spinner").style.display)', getTitle: 'window.ReactNativeWebView.postMessage(document.querySelector("#annotation-scroller div.a-column.a-span-last > h3").textContent)', getImg: 'window.ReactNativeWebView.postMessage(document.querySelector("#annotation-scroller div.a-column.a-span3 > a img").src)', getLocation:'window.ReactNativeWebView.postMessage([[...[...document.querySelectorAll("#kp-notebook-annotations > div > div > div > input")].filter(input => {return input.parentNode.nextElementSibling.contains(input.parentNode.nextElementSibling.querySelector(".a-alert-inline")) === false})].map(input => input.value)])', getText: 'window.ReactNativeWebView.postMessage([...document.querySelectorAll("#highlight")].map(span => span.textContent + "#*#--*#*"))', }; ຊͷλΠτϧऔಘ༻ɺΫϦοΫ༻ɺϋΠϥΠτऔಘ༻ͳͲͷ JavaScript ͷίʔυΛ·ͱΊͨ Object Λ༻ҙ͓͖ͯ͠ɺ ୺຤ͷૢ࡞ʹԠͯ͡ injectJavaScript() Ͱ࣮ߦ
17. ·ͱΊ • injectJavaScript() • window.ReactNativeWebView.postMessage() • onMessage ͜ͷ 3ͭΛ૊Έ߹Θͤͯ WebView ͔Β৘ใΛऔಘ͠·͢
18. ͋Γ͕ͱ͏͍͟͝·ͨ͠ @centerfield77