Androidでスクレイピングした話 / Talk of scraping with Android

Transcript

1. @futabooo Connehito Marché #1ʙAndroidࢢʙ #ConnehitoMarche AndroidͰεΫϨΠϐϯάͨ͠࿩

2. 2 About me  4FOJPS
   &OHJOFFS
   
   4DSVN
   .BTUFS
   BU
   FVSFLB
   *OD
    +BWB
   ,PUMJO
   (PMBOH

   
   5ZQF4DSJQU
   "OHVMBS+4
    'BOUBTZ&BSUI
   ;FSP
   T$3:FE
   4QMBUPPO
    GVUBCPPP
   ɹɹɹೋ઒ ོߒʢTakahiro Futagawaʣ

3. 3 About eureka  ैۀһ਺
    ໊ʢฏۉ೥ྸࡀʣ
    ࣄۀ಺༰
   

   ࣗࣾαʔϏεاըɾ։ൃɾӡӦ
    1BJST
    $PVQMFT ձࣾ֓ཁ

4. 4 About eureka

5. ࠓ೔ͷ࿩

6. https://droidkaigi.jp/2018/proposals

7. Έͳ͞Μ͸ຊΛಡΜͩ͋ͱͲ͏͠·͔͢ʁ

8. ಡΜͩ಺༰Λه࿥͓͖ͯͨ͘͠ͳΓ·ͤΜ͔ʁ

9. https://bookmeter.com/

10. 10 ಡॻϝʔλʔ  ಡΜͩຊΛײ૝ͱͱ΋ʹه࿥Ͱ͖Δ
     ಡΜͩྔΛ؅ཧ͢Δ͜ͱ͕Ͱ͖Δ

11. AndroidΞϓϦ͕ͳ͍

12. ͳ͚Ε͹ͭ͘Ε͹͍͍

13. ࡞ͬͨ

14. ࣮૷ͷ঺հ

15. 15 ࣮૷ͷ঺հ  KIZKTPVQ
     +BWB
    )5.-
    1BSTFS
    XJUI
    CFTU
    PG
    %0.
    $44
    BOE
    KRVFSZ

16. ϩάΠϯը໘

17. interface LoginService { @GET("/login") fun get(): Observable<ResponseBody> @FormUrlEncoded @POST("/login") fun

    login( @Field("session[email_address]") mail: String, @Field("session[password]") password: String, @Field("authenticity_token") authenticityToken: String): Observable<ResponseBody> }

18. retrofit.create(LoginService::class.java).get() .flatMap { val reader = BufferedReader(InputStreamReader(it.byteStream())) val result =

    reader.readLines().filter(String::isNotBlank).toList() val token = Jsoup.parse(result.toString()) .select("form input[name=authenticity_token]").attr("value") } .subscribeBy( onNext = { val reader = BufferedReader(InputStreamReader(it.byteStream())) val result = reader.readLines().filter(String::isNotBlank).toList() val alert = Jsoup.parse(result.toString()) .select(“div.container li.bm-flash-item—alert").isNotEmpty() if (alert) { Timber.i("E-mail or Password is wrong") contract.failureLogin() return@subscribeBy } }, onError = { Timber.e(it, it.message) } )

19. ར༻ͨ͠πʔϧ

20. postman, Charles, Chrome dev tool

21. ϋϚͬͨͱ͜Ζ

22. ಥવͷ࢓༷มߋ❗

23. 23 ಥવͷ࢓༷มߋ  ௕ظٳΈ໌͚ʹΞϓϦʹϩάΠϯͰ͖ͳ͘ͳͬͨ
     ৽͍͠ηΩϡϦςΟରࡦͰ3FRVFTUʹ5PLFO͕ඞཁʹͳ͍ͬͯͨ
     εΫϨΠϐϯάͳͷͰ਌੾ͳΤϥʔ͕ฦ͞ΕΔΘ͚Ͱ΋ͳ͍
     ௐࠪʹ̍ि͔͚ؒͯमਖ਼

24. ·ͱΊ

25. 25 ·ͱΊ  "OESPJEΞϓϦ͕ແ͍͚Ͳ࢖͍͍ͨαʔϏε͕͋ͬͨΒ࡞ͬͪΌ͑͹ྑ͍
     ͱ͸͍͑εΫϨΠϐϯά͸ಥવͷ࢓༷มߋͰ͙͢ࢮ͵
     ΞϓϦͷίʔυ͸͸HJUIVCެ։ͯ͠·͢
     GVUBCPPP#PPL-JGF

26. Credit: NASA Earth Observatory/NOAA NGDC Thank you :)