Androidでスクレイピングした話 / Talk of scraping with Android

Transcript

1. @futabooo Connehito Marché #1ʙAndroidࢢʙ #ConnehitoMarche AndroidͰεΫϨΠϐϯάͨ͠࿩
2. 2 About me  4FOJPS
   &OHJOFFS
   
   4DSVN
   .BTUFS
   BU
   FVSFLB
   *OD
    +BWB
   ,PUMJO
   (PMBOH
   5ZQF4DSJQU
   "OHVMBS+4
    'BOUBTZ&BSUI
   ;FSP
   T$3:FE
   4QMBUPPO
    GVUBCPPP
   ɹɹɹೋ઒ ོߒʢTakahiro Futagawaʣ
3. 3 About eureka  ैۀһ਺
    ໊ʢฏۉ೥ྸࡀʣ
    ࣄۀ಺༰
    ࣗࣾαʔϏεاըɾ։ൃɾӡӦ
    1BJST
    $PVQMFT ձࣾ֓ཁ
4. 4 About eureka
5. ࠓ೔ͷ࿩
6. https://droidkaigi.jp/2018/proposals
7. Έͳ͞Μ͸ຊΛಡΜͩ͋ͱͲ͏͠·͔͢ʁ
8. ಡΜͩ಺༰Λه࿥͓͖ͯͨ͘͠ͳΓ·ͤΜ͔ʁ
9. https://bookmeter.com/
10. 10 ಡॻϝʔλʔ  ಡΜͩຊΛײ૝ͱͱ΋ʹه࿥Ͱ͖Δ
     ಡΜͩྔΛ؅ཧ͢Δ͜ͱ͕Ͱ͖Δ
11. AndroidΞϓϦ͕ͳ͍
12. ͳ͚Ε͹ͭ͘Ε͹͍͍
13. ࡞ͬͨ
14. ࣮૷ͷ঺հ
15. 15 ࣮૷ͷ঺հ  KIZKTPVQ
     +BWB
    )5.-
    1BSTFS
    XJUI
    CFTU
    PG
    %0.
    $44
    BOE
    KRVFSZ
16. ϩάΠϯը໘
17. interface LoginService { @GET("/login") fun get(): Observable<ResponseBody> @FormUrlEncoded @POST("/login") fun login( @Field("session[email_address]") mail: String, @Field("session[password]") password: String, @Field("authenticity_token") authenticityToken: String): Observable<ResponseBody> }
18. retrofit.create(LoginService::class.java).get() .flatMap { val reader = BufferedReader(InputStreamReader(it.byteStream())) val result = reader.readLines().filter(String::isNotBlank).toList() val token = Jsoup.parse(result.toString()) .select("form input[name=authenticity_token]").attr("value") } .subscribeBy( onNext = { val reader = BufferedReader(InputStreamReader(it.byteStream())) val result = reader.readLines().filter(String::isNotBlank).toList() val alert = Jsoup.parse(result.toString()) .select(“div.container li.bm-flash-item—alert").isNotEmpty() if (alert) { Timber.i("E-mail or Password is wrong") contract.failureLogin() return@subscribeBy } }, onError = { Timber.e(it, it.message) } )
19. ར༻ͨ͠πʔϧ
20. postman, Charles, Chrome dev tool
21. ϋϚͬͨͱ͜Ζ
22. ಥવͷ࢓༷มߋ❗
23. 23 ಥવͷ࢓༷มߋ  ௕ظٳΈ໌͚ʹΞϓϦʹϩάΠϯͰ͖ͳ͘ͳͬͨ
     ৽͍͠ηΩϡϦςΟରࡦͰ3FRVFTUʹ5PLFO͕ඞཁʹͳ͍ͬͯͨ
     εΫϨΠϐϯάͳͷͰ਌੾ͳΤϥʔ͕ฦ͞ΕΔΘ͚Ͱ΋ͳ͍
     ௐࠪʹ̍ि͔͚ؒͯमਖ਼
24. ·ͱΊ
25. 25 ·ͱΊ  "OESPJEΞϓϦ͕ແ͍͚Ͳ࢖͍͍ͨαʔϏε͕͋ͬͨΒ࡞ͬͪΌ͑͹ྑ͍
     ͱ͸͍͑εΫϨΠϐϯά͸ಥવͷ࢓༷มߋͰ͙͢ࢮ͵
     ΞϓϦͷίʔυ͸͸HJUIVCެ։ͯ͠·͢
     GVUBCPPP#PPL-JGF
26. Credit: NASA Earth Observatory/NOAA NGDC Thank you :)