Androidでスクレイピングした話 / Talk of scraping with Android

Transcript

1. @futabooo
   Connehito Marché #1ʙAndroidࢢʙ #ConnehitoMarche
   AndroidͰεΫϨΠϐϯάͨ͠࿩
   

   View Slide

2. 2
   About me
   4FOJPS&OHJOFFS4DSVN.BTUFSBUFVSFLB *OD
   +BWB ,PUMJO (PMBOH 5ZQF4DSJQU "OHVMBS+4
   'BOUBTZ&BSUI;FSP T$3:FE 4QMBUPPO
   GVUBCPPP
   ɹɹɹೋ઒ ོߒʢTakahiro Futagawaʣ
   

   View Slide

3. 3
   About eureka
   ैۀһ਺
   ໊ʢฏۉ೥ྸࡀʣ
   ࣄۀ಺༰
   ࣗࣾαʔϏεاըɾ։ൃɾӡӦ
   1BJST
   $PVQMFT
   ձࣾ֓ཁ
   

   View Slide

4. 4
   About eureka
   

   View Slide

5. ࠓ೔ͷ࿩
   

   View Slide

6. https://droidkaigi.jp/2018/proposals
   

   View Slide

7. Έͳ͞Μ͸ຊΛಡΜͩ͋ͱͲ͏͠·͔͢ʁ
   

   View Slide

8. ಡΜͩ಺༰Λه࿥͓͖ͯͨ͘͠ͳΓ·ͤΜ͔ʁ
   

   View Slide

9. https://bookmeter.com/
   

   View Slide

10. 10
    ಡॻϝʔλʔ
    ಡΜͩຊΛײ૝ͱͱ΋ʹه࿥Ͱ͖Δ
    ಡΜͩྔΛ؅ཧ͢Δ͜ͱ͕Ͱ͖Δ
    

    View Slide

11. AndroidΞϓϦ͕ͳ͍
    

    View Slide

12. ͳ͚Ε͹ͭ͘Ε͹͍͍
    

    View Slide

13. ࡞ͬͨ
    

    View Slide

14. ࣮૷ͷ঺հ
    

    View Slide

15. 15
    ࣮૷ͷ঺հ
    KIZKTPVQ
    +BWB)5.-1BSTFS XJUICFTUPG%0. $44 BOEKRVFSZ
    

    View Slide

16. ϩάΠϯը໘
    

    View Slide

17. interface LoginService {
    @GET("/login") fun get(): Observable
    @FormUrlEncoded @POST("/login") fun login(
    @Field("session[email_address]") mail: String,
    @Field("session[password]") password: String,
    @Field("authenticity_token") authenticityToken: String): Observable
    }
    

    View Slide

18. retrofit.create(LoginService::class.java).get()
    .flatMap {
    val reader = BufferedReader(InputStreamReader(it.byteStream()))
    val result = reader.readLines().filter(String::isNotBlank).toList()
    val token = Jsoup.parse(result.toString())
    .select("form input[name=authenticity_token]").attr("value")
    }
    .subscribeBy(
    onNext = {
    val reader = BufferedReader(InputStreamReader(it.byteStream()))
    val result = reader.readLines().filter(String::isNotBlank).toList()
    val alert = Jsoup.parse(result.toString())
    .select(“div.container li.bm-flash-item—alert").isNotEmpty()
    if (alert) {
    Timber.i("E-mail or Password is wrong")
    contract.failureLogin()
    return@subscribeBy
    }
    },
    onError = { Timber.e(it, it.message) }
    )
    

    View Slide

19. ར༻ͨ͠πʔϧ
    

    View Slide

20. postman, Charles, Chrome dev tool
    

    View Slide

21. ϋϚͬͨͱ͜Ζ
    

    View Slide

22. ಥવͷ࢓༷มߋ❗
    

    View Slide

23. 23
    ಥવͷ࢓༷มߋ
    ௕ظٳΈ໌͚ʹΞϓϦʹϩάΠϯͰ͖ͳ͘ͳͬͨ
    ৽͍͠ηΩϡϦςΟରࡦͰ3FRVFTUʹ5PLFO͕ඞཁʹͳ͍ͬͯͨ
    εΫϨΠϐϯάͳͷͰ਌੾ͳΤϥʔ͕ฦ͞ΕΔΘ͚Ͱ΋ͳ͍
    ௐࠪʹ̍ि͔͚ؒͯमਖ਼
    

    View Slide

24. ·ͱΊ
    

    View Slide

25. 25
    ·ͱΊ
    "OESPJEΞϓϦ͕ແ͍͚Ͳ࢖͍͍ͨαʔϏε͕͋ͬͨΒ࡞ͬͪΌ͑͹ྑ͍
    ͱ͸͍͑εΫϨΠϐϯά͸ಥવͷ࢓༷มߋͰ͙͢ࢮ͵
    ΞϓϦͷίʔυ͸͸HJUIVCެ։ͯ͠·͢
    GVUBCPPP#PPL-JGF
    

    View Slide

26. Credit: NASA Earth Observatory/NOAA NGDC
    Thank you :)
    

    View Slide