$30 off During Our Annual Pro Sale. View Details »

Androidでスクレイピングした話 / Talk of scraping with Android

futabooo
February 01, 2018

Androidでスクレイピングした話 / Talk of scraping with Android

futabooo

February 01, 2018
Tweet

More Decks by futabooo

Other Decks in Technology

Transcript

  1. @futabooo
    Connehito Marché #1ʙAndroidࢢʙ #ConnehitoMarche
    AndroidͰεΫϨΠϐϯάͨ͠࿩

    View Slide

  2. 2
    About me
    4FOJPS&OHJOFFS4DSVN.BTUFSBUFVSFLB *OD
    +BWB ,PUMJO (PMBOH 5ZQF4DSJQU "OHVMBS+4
    'BOUBTZ&BSUI;FSP T$3:FE 4QMBUPPO
    GVUBCPPP
    ɹɹɹೋ઒ ོߒʢTakahiro Futagawaʣ

    View Slide

  3. 3
    About eureka
    ैۀһ਺
    ໊ʢฏۉ೥ྸࡀʣ
    ࣄۀ಺༰
    ࣗࣾαʔϏεاըɾ։ൃɾӡӦ
    1BJST
    $PVQMFT
    ձࣾ֓ཁ

    View Slide

  4. 4
    About eureka

    View Slide

  5. ࠓ೔ͷ࿩

    View Slide

  6. https://droidkaigi.jp/2018/proposals

    View Slide

  7. Έͳ͞Μ͸ຊΛಡΜͩ͋ͱͲ͏͠·͔͢ʁ

    View Slide

  8. ಡΜͩ಺༰Λه࿥͓͖ͯͨ͘͠ͳΓ·ͤΜ͔ʁ

    View Slide

  9. https://bookmeter.com/

    View Slide

  10. 10
    ಡॻϝʔλʔ
    ಡΜͩຊΛײ૝ͱͱ΋ʹه࿥Ͱ͖Δ
    ಡΜͩྔΛ؅ཧ͢Δ͜ͱ͕Ͱ͖Δ

    View Slide

  11. AndroidΞϓϦ͕ͳ͍

    View Slide

  12. ͳ͚Ε͹ͭ͘Ε͹͍͍

    View Slide

  13. ࡞ͬͨ

    View Slide

  14. ࣮૷ͷ঺հ

    View Slide

  15. 15
    ࣮૷ͷ঺հ
    KIZKTPVQ
    +BWB)5.-1BSTFS XJUICFTUPG%0. $44 BOEKRVFSZ

    View Slide

  16. ϩάΠϯը໘

    View Slide

  17. interface LoginService {
    @GET("/login") fun get(): Observable
    @FormUrlEncoded @POST("/login") fun login(
    @Field("session[email_address]") mail: String,
    @Field("session[password]") password: String,
    @Field("authenticity_token") authenticityToken: String): Observable
    }

    View Slide

  18. retrofit.create(LoginService::class.java).get()
    .flatMap {
    val reader = BufferedReader(InputStreamReader(it.byteStream()))
    val result = reader.readLines().filter(String::isNotBlank).toList()
    val token = Jsoup.parse(result.toString())
    .select("form input[name=authenticity_token]").attr("value")
    }
    .subscribeBy(
    onNext = {
    val reader = BufferedReader(InputStreamReader(it.byteStream()))
    val result = reader.readLines().filter(String::isNotBlank).toList()
    val alert = Jsoup.parse(result.toString())
    .select(“div.container li.bm-flash-item—alert").isNotEmpty()
    if (alert) {
    Timber.i("E-mail or Password is wrong")
    contract.failureLogin()
    return@subscribeBy
    }
    },
    onError = { Timber.e(it, it.message) }
    )

    View Slide

  19. ར༻ͨ͠πʔϧ

    View Slide

  20. postman, Charles, Chrome dev tool

    View Slide

  21. ϋϚͬͨͱ͜Ζ

    View Slide

  22. ಥવͷ࢓༷มߋ❗

    View Slide

  23. 23
    ಥવͷ࢓༷มߋ
    ௕ظٳΈ໌͚ʹΞϓϦʹϩάΠϯͰ͖ͳ͘ͳͬͨ
    ৽͍͠ηΩϡϦςΟରࡦͰ3FRVFTUʹ5PLFO͕ඞཁʹͳ͍ͬͯͨ
    εΫϨΠϐϯάͳͷͰ਌੾ͳΤϥʔ͕ฦ͞ΕΔΘ͚Ͱ΋ͳ͍
    ௐࠪʹ̍ि͔͚ؒͯमਖ਼

    View Slide

  24. ·ͱΊ

    View Slide

  25. 25
    ·ͱΊ
    "OESPJEΞϓϦ͕ແ͍͚Ͳ࢖͍͍ͨαʔϏε͕͋ͬͨΒ࡞ͬͪΌ͑͹ྑ͍
    ͱ͸͍͑εΫϨΠϐϯά͸ಥવͷ࢓༷มߋͰ͙͢ࢮ͵
    ΞϓϦͷίʔυ͸͸HJUIVCެ։ͯ͠·͢
    GVUBCPPP#PPL-JGF

    View Slide

  26. Credit: NASA Earth Observatory/NOAA NGDC
    Thank you :)

    View Slide