かわいいを支援するAppleハック，そして💓で振り返るシンデレラ5th / Apple hack assisting kawaii & Replay Cinderella Girls 5th LIVE with heartbeats

IM@S Engineer Talks - IM@Study 2nd Anniv Event : Brand New Meetup!! 2018-07-07
https://imas.connpass.com/event/90996/

YouTube Archive
https://youtu.be/UvN8M3QzE08?t=3747

Blog
https://banjun.hatenablog.com/entry/2018/07/13/233019

Transcript

1. ͔Θ͍͍Λࢧԉ͢Δ"QQMFϋοΫ

   ͦͯ͠ͰৼΓฦΔγϯσϨϥUI
   ͹Μ͡ΎΜ!CBOKVO
   *.!4&OHJOFFS5BMLT
   

   View Slide

2. View Slide

3. View Slide

4. View Slide

5. View Slide

6. -PWF-JWFSʜϥΠϒϑΥτ
   1IPUP4UVEJP1MBZFSʜϥΠτχ
   ϯάΩϟϓνϟ(16ΫϩϚΩʔ
   5PVDI#BSͰӎ݄ͭΜͭΜ
   J04Ќͷ͋Ε͜Ε
   
   
   

   View Slide

7. ϥΠϒϑΥτ
   -PWF-JWFS
   σϨες.7
   ϥΠϒϑΥτ

   ੾Γग़͠ੜ੒
   a͗Ύͬ
   

   View Slide

8. ϑΥʔϚοτղੳϋοΫ

   ͕ඞཁΒ͍͠
   J04
   ϥΠϒϑΥτ GQT
   
   J04
   J04
   GQT
   
   J04
   GQT
   
   GQTΫϥογϡमਖ਼
   ॎ.7ൃݟ ॎ.7ෆՄ Ұ෦
   ެࣜॎ.7
   
   -PWF-JWFS
   88%$
   ͋Γ͢ݟͤΔ
   88%$
   GQT
   
   ͋Γ͢ݟͤΔ
   4."35
   ॎ.7ෆՄ Ұ෦
   
   

   View Slide

9. View Slide

10. ϥΠτχϯάΩϟϓνϟΫϩϚΩʔ
    1IPUP4UVEJP1MBZFS
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. ٶ৓
    %BZ%BZ
    ੴ઒
    %BZ%BZ
    େࡕ
    %BZ%BZ
    

    View Slide

15. View Slide

16. ੩Ԭ
    %BZ%BZ
    ນு
    %BZ%BZ
    ෱Ԭ
    %BZ%BZ
    

    View Slide

17. View Slide

18. 44"
    %BZ%BZ
    

    View Slide

19. https://support.apple.com/ja-jp/ht207934
    ʮͦͷଞʯΛ͔ͭ͏
    J1IPOFిݯΦϑ
    γΞλʔϞʔυ
    ʮϥΠϒʯ͸ͳ͍
    αΠϨϯτ
    ಈ͍ͯ΋ը໘఺͔ͳ͍
    ϫʔΫΞ΢τ
    

    View Slide

20. https://speakerdeck.com/nnsnodnb/wotakuniyun-dong-hanan-sii
    ؔ࿈ࢿྉ
    ޿ΊΑ͏ϥΠϒه࿥
    

    View Slide

21. Ұ೥ޙ
    

    View Slide

22. %FNP
    
    

    View Slide

23. ˣ#%؍͍ͯΔࢲͷ
    
    ϛϥΫϧςϨύγʔ
    0SBOHF4BQQIJSF
    #&:0/%5)&45"3-*()5
    ؾ͍͍࣋ͪΑͶҰ౳৆ʂ
    ७৘.JEOJHIU఻આ
    ࠶ੜ࣌ࠁͷ౰࣌ͷˣ
    αϚΧχ
    

    View Slide

24. View Slide

25. 
    ͜͜Λͭͬͯ͘

    ϥΠϒϥϦʹ͍ͨ͠
    

    View Slide

26. View Slide

27. View Slide

28. CGWindowListCreateImage
    CGWindowListCopyWindowInfo
    TargetWindow(
    appName: "Mac Blu-ray Player",
    windowTitle: "Mac Blu-ray Player")?
    .capture(relativeBounds: rect)
    Ͱݕࡧ
    ͰΩϟϓνϟ
    $PSF(SBQIJDT
    ͭ·ΓˣΈ͍ͨͷ͕࡞ΕΔ
    

    View Slide

29. ςΩετϥϕϧͷը૾
    จࣈ෼ղ
    ਺ࣈೝࣝ
    ࠶ੜ࣌ࠁ
    YYඵ
    
    ʜ
    ʜ
    

    View Slide

30. $PSF(SBQIJDT
    7JTJPO5FYU%FUFDUJPO
    $PSF.-5VSJ$SFBUF
    ͨ͠ࢉ
    
    YYඵ
    
    ʜ
    ʜ
    
    
    
    

    View Slide

31. 7JTJPO5FYU%FUFDUJPO
    (>= iOS 11, macOS 10.13)
    https://github.com/hollisliu/iOS-Vision-Text-Detection-Demo
    ը૾
    
    ը૾ͷʮจࣈͬΆ͍ʯ෦෼ݕ஌
    ͳΜͷจࣈ͔͸෼͔Βͳ͍
    

    View Slide

32. $PSF.-
    (>= iOS 11, macOS 10.13)
    ػցֶशࡁΈϞσϧΛ͔ͭͬͯɺ

    J04ͱNBD04Ͱɺਪ࿦͚ͩΛ͢Δ
    5FOTPS'MPXͳͲ͔Β$PSF.-ͷϞσϧ
    ϑΝΠϧϑΥʔϚοτ΁ม׵Մೳ
    ./*45ֶशࡁΈͷϞσϧͳͲ͸ೖख༰қ
    https://github.com/likedan/Awesome-CoreML-Models
    

    View Slide

33. $PSF.-
    (>= iOS 11, macOS 10.13)
    ɹɹ
    ./*45

    ֶशࡁΈͷϞσϧ
    

    View Slide

34. View Slide

35. 012838 012639
    ઐ༻Ϟσϧʹ

    ஔ͖׵͑
    ͜Ε͸࡞Δ
    

    View Slide

36. ,FSBTͰ$//ॻ͍ͯ./*45ͬΆ͘
    ֶश͢Ε͹͍͍͔ͳ
    ͋ͷͶɺ

    "QQMF͕5VSJΛങऩͨ͠Α͒
    CBOKVO
    CBOKVO
    ੈքதͷΓΜ͝ʹ

    ৄ͍͠৘ใے
    ͦ͏͔5VSJ$SFBUFͰ΍Ζ͏
    

    View Slide

37. https://github.com/apple/turicreate
    5VSJ$SFBUF
    ΄΅θϩίʔυͰը૾෼ྨΛֶश
    લஈ͕ResNetPSSqueezeNet ͷసҠֶश
    ֶश͸΍͍ɾ܇࿅ը૾গ͠Ͱ͍͍ ຕʙ
    
    ࠓޙ
    .PKBWF͔Β͸Create MLͱ͍͏໊લͰ
    9DPEF্͔Β(6*ͷΈͰֶशͰ͖Δ
    

    View Slide

38. View Slide

39. ˞͢͜͠ηί͍࿩
    ࣮ࡍ͸܇࿅σʔλͷ͕WBMJEBUJPOʹ࢖ΘΕΔ
    ͷͰɺਖ਼ղσʔλ͕ݮΓ͗͢ͳ͍Α͏ʹௐ੔
    ࠓճ൚Խೳྗ͸΄ͱΜͲཁΒͳ͍ͷͰϑΝΠ
    ϧίϐʔͱ͔ʜ
    

    View Slide

40. View Slide

41. %0,*%0,*3IZUIN
    )FBSU7PJDF

    J04
    
    )FBSU7PJDF

    NBD04
    
    .VMUJQFFS

    $POOFDUJWJUZ

    ૹ৴
    

    View Slide

42. ࠓޙ΍͍͖͍ͬͯͨ
    ϥΠϒͱϫʔΫΞ΢τͷ։࢝ऴྃ࣌ࠁ߹Θͤ
    ηοτϦετͱͷॏͶ͋Θͤ
    ˢϫʔΫΞ΢τه࿥ͷ޻෉ɺ͔ͳ͊ʜ
    

    View Slide

43. View Slide

44. .!(*$ˑ
    ੴ઒ େࡕ
    Ξλϙϯ
    ΦϨαϑΝ
    #54
    :15 Ԧࢠ
    Ԧࢠ
    ઈରಛݖ
    ྫྷഴ
    ηΫΪϧ
    ''
    8(0
    αϚΧχ
    ੨ۭ৭ସ͑
    5)&*%0-.!45&3$*/%&3&--"(*3-4UI-*7&50634FSFOEJQJUZ1BSBEFΑΓ
    

    View Slide

45. https://nrstudy.connpass.com
    ͓͠Βͤ ۚ
    ༦ํʙ
    ΋͘΋͘ձ΍Γ·͢✨
    

    View Slide

46. Ұ೥தɺΛ͓͍͘͠
    

    View Slide