$30 off During Our Annual Pro Sale. View Details »

UIのブラックボックスを探る/iOSDC23

noppefoxwolf
September 02, 2023

 UIのブラックボックスを探る/iOSDC23

noppefoxwolf

September 02, 2023
Tweet

More Decks by noppefoxwolf

Other Decks in Programming

Transcript

  1. UIͷϒϥοΫϘοΫεΛ୳Δ
    iOSDC23 day1 Track B noppe
    1

    View Slide

  2. noppe @noppefoxwolf
    • DeNA Co., Ltd.
    • Pococha
    • Indie app
    • vear
    • Editormode
    • Nigh;ox DAWN for mastodon
    2

    View Slide

  3. ೴಺ϓϨϏϡʔ@DeNAϒʔε
    3

    View Slide

  4. ࠓ೔࿩͢͜ͱ
    4

    View Slide

  5. UIͷϒϥοΫϘοΫεͱ͸
    5

    View Slide

  6. UIͷϒϥοΫϘοΫε = ҉໧ͷ࢓

    6

    View Slide

  7. ΧελϜUIͷඞཁੑ
    • ಛఆͷૢ࡞Λ࠷దԽ͍ͨ͠
    • ΧϨϯμʔɾը૾ϏϡʔΞ
    • ΞϓϦͷΞΠσϯςΟςΟΛߏஙͨ͠
    ͍
    • طଘͷUIͷϦσβΠϯ
    7

    View Slide

  8. ΧελϜUIͷ඼࣭
    • Apple੡ͷUIͱൺ΂Δͱѱ͘ͳΓ͕ͪ
    • όϥϯε͕ѱ͍
    • ৮Γ৺஍͕ѱ͍
    • ࢖͍ʹ͍͘
    • ...etc
    8

    View Slide

  9. ΧελϜφϏήʔγϣϯόʔͷྫ
    9

    View Slide

  10. 10

    View Slide

  11. Bad UI͕ग़དྷͯ͠·͏ཧ

    • γεςϜͷ͓࡞๏Λແࢹͯ͠͠·͏͔
    Β
    • ΧελϜUI͸γεςϜUIͷ্Ͱ࡞Δ
    ΂͖ʢԫʣ
    • ͓࡞๏Λແࢹͯ͠UIΛ࡞Δʢ੨ʣ
    ͱɺҧ࿨ײ͕ൃੜ
    11

    View Slide

  12. ͓࡞๏Λ஌Δʹ͸
    12

    View Slide

  13. ͓࡞๏Λ஌Δʹ͸1
    • Human Interface Guidelines
    • جຊతͳ͓࡞๏͸͜Ε
    • ϑΥϯταΠζɾϘλϯαΠζͳͲ
    ͷϕετϓϥΫςΟε
    • UIKit, Swi8UI΋͜Εʹ४ڌ͍ͯ͠Δ
    13

    View Slide

  14. ͓࡞๏Λ஌Δʹ͸2
    • γεςϜίϯϙʔωϯτΛ෼ੳ͢Δ
    • HIGʹॻ͔Ε͍ͯͳ͍͓࡞๏
    • ඼࣭ͷཧ૝஋
    • γεςϜίϯϙʔωϯτ
    • Apple੡ͷΞϓϦ
    • UIKit΍Swi/UIͷUIίϯϙʔωϯτ
    14

    View Slide

  15. UIΛ෼ੳ͢Δ
    15

    View Slide

  16. UIΛ෼ੳ͢Δ؍఺
    • ϏδϡΞϧ!
    • ΠϯλϥΫγϣϯ"
    16

    View Slide

  17. ϏδϡΞϧ
    17

    View Slide

  18. ϏδϡΞϧ
    • αΠζɾ৭ɾܗͳͲ໨ʹݟ͑Δ΋ͷ
    • HIGʹॻ͔Ε͍ͯΔ΋ͷ΋͋Δ
    18

    View Slide

  19. ϏδϡΞϧͷద༻ྫ
    19

    View Slide

  20. αΠζ
    • ΞΠίϯɾ༨നɾจࣈαΠζɾؙ֯
    • εΫγϣΛࡱӨ͠ଌΔ
    • Sketch΍PhotoshopͳͲͷRulerػೳ
    • จࣈαΠζͷ৔߹͸ɺεΫγϣΛ൒ಁ
    ໌ʹͯ͠ॏͶͯൺֱ
    • ๏ଇੑΛݟ͚ͭΒΕΔͱGood
    20

    View Slide

  21. αΠζͱΞΫηγϏϦςΟ
    • ΞΫηγϏϦςΟ΍ʮ֦େදࣔʯઃఆʹΑͬͯɺαΠζ͕มԽ
    ͢ΔͷͰ஫ҙ
    • Dynamic Type͸࠷େ࠷খ஋͕ܾ·͍ͬͯΔ৔߹΋͋Δ
    21

    View Slide

  22. Χϥʔ
    • ಛఆͷҙຯΛ࣋ͭΧϥʔʹ஫໨
    • ΧϥʔϐοΧʔΛར༻͢Δ
    • Digital Color Meter, Sip, ColorSlurp
    • ΧϥʔεϖʔεΛ߹Θ͓ͤͯ͘
    22

    View Slide

  23. Χϥʔͷྫ
    • ϔϧεέΞͰ࢖ΘΕ͍ͯΔ৭Λ࢖͏ࣄ
    ͰɺHealthKitͱ࿈ܞ͢ΔϘλϯͰ͋Δ
    ͜ͱΛೝࣝ͠΍͘͢ͳΔ
    • Ϣʔβʔʹ৭ͰҙຯΛ఻͑Δ
    • ஍ਤͷւͷ੨৭
    • εΠονͷ྘৭
    23

    View Slide

  24. μʔΫϞʔυͱΧϥʔ
    • ৚݅ʹΑͬͯ৭͕มΘΔ͜ͱ͕͋Δ
    • μʔΫϞʔυ
    • ৭൓స
    • Smart invert
    • ΞΫηϯτΧϥʔͷઃఆ
    24

    View Slide

  25. ୯৭Ͱͳ͍έʔε
    ୯৭Ͱແ͍৔߹͸৭ΛऔΓग़ͤͳ͍ͷͰ஫ҙ
    ޿Ίʹ৭ΛऔΓग़ͯ͠ɺϜϥ͕ͳ͍͔͔֬ΊΔͷ΋ྑ͍Ͱ͠ΐ͏
    • ϒϥʔ, άϥσʔγϣϯ, ಁա౓
    25

    View Slide

  26. Ξχϝʔγϣϯ
    • dura&on, curve, &ming
    • iOS Simulator
    • Slow Anima&ons
    • εΫϦʔϯϨίʔυΛίϚૹΓͯ֬͠

    • QuickTime Player, iina
    26

    View Slide

  27. ߴ౓ͳϏδϡΞϧͷ෼ੳ
    27

    View Slide

  28. View Hierachy Debugger
    • ௚઀αΠζΛ֬ೝͰ͖Δ
    • Xcode
    • h(ps:/
    /lookin.work
    • OSS
    • ίϯιʔϧػೳ͕༏ल
    • h(ps:/
    /revealapp.com
    • Paid App
    • UIViewͷߏ଄΋ݟΕΔ
    28

    View Slide

  29. Apple Map UI-Component
    Clone Challenge
    • AppleͷMapΞϓϦΛ࠶ݱͯ͠ΈΔ
    • ӈ্ͷϘλϯͷͱ͜Ζ
    • RevealΛ࢖ͬͯɺUIΛ෼ੳ͢Δ
    29

    View Slide

  30. Revealͷ࢓૊Έ
    • ΞϓϦͷதͰಈ࡞͢ΔαʔόʔΛཱͪ
    ্͛ͯɺmacͷΞϓϦͱ௨৴͢Δ
    • ΞϓϦʹRevealServer.frameworkΛ૊
    ΈࠐΉඞཁ͕͋Δ
    • Lookin΋ಉ༷ͷ࢓༷
    • ࣗલͷΞϓϦͰ͔͠ಈ࡞Ͱ͖ͳ͍
    30

    View Slide

  31. Reveal runs anywhere
    • ΞϓϦىಈதʹ֎͔ΒRevealServerΛload͢Δ
    • fridaΛ࢖͏ͱ؆୯
    31

    View Slide

  32. Frida
    h"ps:/
    /frida.re
    • ൚༻తͳϦαʔνπʔϧ
    • εΫϦϓτͷΠϯδΣΫτ
    • ϝιουίʔϧͷ؂ࢹ
    • rootʹΠϯετʔϧՄೳ
    • iPhoneͷroot͕ඞཁ
    32

    View Slide

  33. Frida script
    loader.js
    let loaderPath = "~/RevealServer.framework/RevealServer"
    Module.load(loaderPath)
    33

    View Slide

  34. Load framework using Frida
    # 1. frida-psͰUSBͰ઀ଓͨ͠iPhone͔ΒϓϩηεIDΛ୳͢
    $ frida-ps --usb | grep Map
    2537 Maps
    # 2. fridaͰPIDʹରͯ͠εΫϦϓτΛ࣮ߦ͢Δ
    $ frida --usb --load loader.js --attach-pid 2537
    34

    View Slide

  35. 35

    View Slide

  36. 36

    View Slide

  37. 37

    View Slide

  38. 38

    View Slide

  39. 39

    View Slide

  40. Ϋϥε໊͔ΒϔομʔΛ
    ୳͢
    • ktool, DyldExtractor
    • h1ps:/
    /github.com/0cyn/sdk-builder
    • APIઃܭΛֶͿ
    • Frida΍LLDBͰϝιουΛ࣮ߦ࣮ͯ͠ݧ
    ͢Δ
    40

    View Slide

  41. UIΛ෼ੳ͢Δ؍఺
    • ϏδϡΞϧ!
    • ΠϯλϥΫγϣϯ"
    41

    View Slide

  42. ΠϯλϥΫγϣϯ
    42

    View Slide

  43. ΠϯλϥΫγϣϯͰେࣄͳ͜ͱ
    ౰ͨΓલͷૢ࡞Λ͔ͬ͠Γఏڙ͢Δ
    • ࣗ෼ʹͱͬͯͷ౰ͨΓલͰ͸ͳ͘ɺϢʔβʔʹͱͬͯͷ౰ͨΓ
    લΛ࣮૷͢Δ
    • δΣενϟʔʹΑΔೖྗ
    • ΩʔϘʔυ΍ϙΠϯλʔσόΠεʹΑΔೖྗ
    • ϑΟʔυόοΫ
    43

    View Slide

  44. δΣενϟʔ
    • Ϗϡʔ͕ͲΜͳδΣενϟʔʹରԠ͍ͯ͠Δ͔֬ೝ͢Δ
    • ෳ਺ͷδΣενϟʔΛಉ࣌ʹ࢖͏͜ͱ΋͋Δ
    • ઌೖ؍Λࣺͯͯ৮ͬͯΈΔ
    • ݟಀ͕ͪ͠ͳ΋ͷΛ͍͔ͭ͘঺հ
    44

    View Slide

  45. Case1. ิॿδΣενϟʔ
    • λοϓͰOn,Offͷ੾Γସ͑
    • ύϯͰ΋ิॿతʹ੾Γସ͕͑Ͱ͖Δ
    45

    View Slide

  46. Case2. ෆՄࢹτϦΨʔ
    • ͋Δఔ౓ݟͨ໨Ͱ࢖͍ํ͕෼͔Δ΋ͷ
    ͕΄ͱΜͲ
    • ͔͠͠ɺෆՄࢹͷτϦΨʔ΋͋Δ
    • γΣΠΫɺϚϧνλοϓɺ3Dλον
    ͳͲ
    46

    View Slide

  47. Case3. ᮢ஋
    • ࣮ࡍͷݟͨ໨ΑΓ΋ζϨͨҐஔʹ൑ఆ
    ͕͋Δ৔߹΋͋Δ
    47

    View Slide

  48. ֎෦σόΠε͔Βͷೖྗ
    • ϋʔυ΢ΣΞΩʔϘʔυ
    • γϣʔτΧοτʹରԠ͍ͯ͠Δ͔
    • ϙΠϯλʔσόΠε
    • ϙΠϯλ͕ϗόʔͨ࣌͠ͷڍಈ
    48

    View Slide

  49. ϑΟʔυόοΫ
    • ϏδϡΞϧϑΟʔυόοΫ
    • α΢ϯυ
    • ϋϓςΟΫε
    • ϘΠεΦʔόʔ
    49

    View Slide

  50. α΢ϯυ
    • ૹ৴Իɺ௨஌ԻɺΫϦοΫԻͳͲ
    • Իͷ௕͞ɺԻͷେ͖͕͞ࢀߟʹͳΔ
    • ipaΛղౚͯ͠ɺaifϑΝΠϧΛऔΓग़͢
    50

    View Slide

  51. ϋϓςΟΫε
    • ίϯςΩετϝχϡʔɺPull to Refresh
    • ෇͚ͯͳ͍ͱ͜ΖΛҙࣝ͢Δ
    • λϒͷҠಈͳͲ
    51

    View Slide

  52. ϘΠεΦʔόʔ
    • ઃఆ → ΞΫηγϏϦςΟ → ϘΠεΦʔόʔ
    • ϘΠεΦʔόʔ͕ಡΈ্͛ΔจষΛ֬ೝ͢Δ
    • HIG΋ࢀর2
    2 h$ps:/
    /developer.apple.com/jp/design/human-interface-guidelines/accessibility#VoiceOver
    52

    View Slide

  53. ΧελϜUIΛ࡞Δ্Ͱͷώϯτ
    53

    View Slide

  54. UIͷղऍ͸Ұக͢Δͱ͸
    ݶΒͳ͍
    • ࢀߟʹͨ͠UI΍৭ʹ͸ઃܭࢥ૝͕͋Δ
    • ໌֬ͳҙਤͷ্Ͱ࣮૷͞Ε͍ͯΔ
    • ҆қʹਅࣅͯ͠΋ɺຊདྷͷࢥ૝ͱ͸ҟ
    ͳΔ࢖͍ํʹͳΔ͜ͱ΋͋Δ
    • ΞϓϦ಺௨஌ʹAirPodsͷΑ͏ͳ
    HUDΛ࢖͏ͱɺϢʔβʔ͸צҧ͍͠
    ͳ͍͔ʁ
    54

    View Slide

  55. ࣮૷ʹ࠷దͳUI΍APIΛબͿ
    • UIKitʹطʹ͋Δ΋ͷΛ࠶࣮૷͠ͳ͍
    • εΫϩʔϧͷ׳ੑΛ࣮૷͠ͳ͍ɻUIScrollViewΛ࢖͏
    • ϘλϯͰྑ͍ͳΒTapGestureͰ࣮૷͠ͳ͍
    55

    View Slide

  56. σβΠϯͷ௥ैίετ
    • OSͷΞοϓσʔτʹΑͬͯσβΠϯ͕
    มΘΔ
    • ͔࣮ͤͬ͘૷ͨ͠UI͕ݹष͘ݟ͑Δ͜
    ͱ΋
    • σβΠϯͷ௥ैίετ͕ൃੜ͢Δ
    56

    View Slide

  57. Φϑʹग़དྷΔ͜ͱ΋๨Εͣʹ
    • ΞΫηγϏϦςΟͰແޮʹग़དྷΔ͜ͱ΋
    • Shake to Undo
    • Reduce Mo1on
    • ࣗલͰ࣮૷ͨ͠Βɺ੾Γସ͑΋ߟྀ͢Δ͜ͱ
    57

    View Slide

  58. ݱ࣮తʹग़དྷͳ͍͜ͱ΋͋Δ
    • ΧελϜUI͸඼࣭͕མͪΔ͜ͱΛલఏʹ
    • τϨʔυΦϑΛҙࣝͯ͠ɺແବʹΧελϜUI࡞Βͳ͍ͷ͕ίπ
    • Ұճݟͨ໨ͷྑ͍΋ͷΛ࡞͔ͬͯΒ͸໭ͤͳ͍
    • ΞϓϦͷίΞόϦϡʔʹߜ࣮ͬͯ૷ͯ͠Έͯ͸
    58

    View Slide

  59. See also
    • h#ps:/
    /github.com/scenee/Floa6ngPanel
    • h#ps:/
    /github.com/iDevelopper/PBPopupController
    • h#ps:/
    /github.com/omaralbeik/Drops
    • h#ps:/
    /github.com/SvenTiigi/WhatsNewKit
    • h#ps:/
    /github.com/usagimaru/Floa6ngSwitch
    • h#ps:/
    /github.com/noppefoxwolf/ColorPicker
    59

    View Slide

  60. ͓ΘΓ
    60

    View Slide