TextKit 2 時代の iOS のキーボードとテキスト入力と表示のすべて

iOS は一見、テキスト入力や表示のような基本的実装はとても簡単に思えますが、実際には一度はその動作に頭を悩ましたことがあるでしょう。

iOSDC 2020 では「iOS のキーボードと文字入力のすべて」と題してこれらの問題について詳細に検討しました。
そして、この3年間で iOS は大きく変化し、SwiftUI が広く使われるようになり、全く新しいAPI、TextKit 2 が登場しました。
TextKit 2 は大きく変化しおり、互換性に関して注意すべき点が多くあります。
このセッションでは、過去の事例と比較して、多くのユーザが使う iOS アプリにおけるリッチテキストの実装などを踏まえて、
キーボードの挙動や SwiftUI、そして TextKit 2 によって変わったテキスト処理について実装例を用いて検討していきます。

対象とする方: iOS アプリ開発の経験がある中・上級者

Transcript

1. TextKit 2 ࣌୅ͷ iOS ͷΩʔϘʔυͱςΩετೖྗ
   ͱදࣔͷ͢΂ͯ
   @niw
   
   
   9/2/2023 — Tokyo, Japan
   
   
   iOSDC Japan 2023
   

   View Slide

2. Yoshimasa Niwa
   @niw
   

   View Slide

3. ࣌͸ 2020 ೥…
   

   View Slide

4. iOS ͷΩʔϘʔυͱจࣈೖྗͷ͢΂ͯ
   @niw
   
   
   09/21/2020 — Online
   
   
   iOSDC Japan 2020
   

   View Slide

5. iOS ͷςΩετදࣔͷ͢΂ͯ
   @niw
   
   
   9/2/2023 — Tokyo, Japan
   
   
   iOSDC Japan 2023
   TextKit 2 ରԠ൛
   New!
   

   View Slide

6. TextKit 1
   

   View Slide

7. ಛ௃
   TextKit 1
   ݹ͔͘Βଓ͘ NSLayoutManager Λத৺ͱͨ͠ΫϥεͰߏ
   ੒͞ΕΔ API
   
   
   ΋ͱ΋ͱ TextKit 1 ͳͲͱ͸ݺ͹Ε͍ͯͳ͔͕ͬͨɺTextKit 2
   ͷ API ͱ۠ผ͢ΔͨΊʹͦ͏ݺ͹ΕΔΑ͏ʹͳͬͨ
   

   View Slide

8. NSTextStorage
   NSLayoutManager
   NSTextContainer
   UITextView Ϗϡʔ
   ίϯτϩʔϥʔ
   Ϟσϧ
   

   View Slide

9. let textStorage = NSTextStorage()
   
   
   let layoutManager = NSLayoutManager()
   
   
   textStorage.addLayoutManager(layoutManager)
   
   
   let textContainer = NSTextContainer()
   
   
   layoutManager.addTextContainer(textContainer)
   
   
   let textView = UITextView(
   

   frame: view.bounds,
   

   textContainer: textContainer
   

   )
   

   View Slide

10. ಛ௃
    TextKit 1
    ૢ࡞ͷ࠷খ୯Ґ͸จࣈͷཁૉ
    Ͱ͋ΔάϦϑ
    
    
    Ϟσϧʹ૬౰͢Δจࣈྻͷൣ
    ғͱରԠ͢ΔάϦϑͷૢ࡞͕
    Մೳ
    ͋
    A
    ಟ
    

    View Slide

11. let layoutManager = textView.layoutManager
    
    
    let glyphRange = layoutManager.glyphRange(
    
    
    forCharacterRange: NSRange(
    

    location: 0, length: 1
    

    ),
    
    
    actualCharacterRange: nil
    
    
    )
    
    
    let rect = layoutManager.boundingRect(
    
    
    forGlyphRange: glyphRange,
    
    
    in: textView.textContainer
    
    
    )
    

    View Slide

12. ར༻ྫ
    

    View Slide

13. U+3042 U+3044 U+3046 U+3048 U+304A
    

    View Slide

14. U+0C85 U+0C95 U+0CCD U+0C9F
    U+0CCB U+0CAC U+0CB0 U+0CCD
    

    View Slide

15. Meet TextKit 2 (WWDC 21)
    

    https://developer.apple.com/videos/play/wwdc2021/10061/
    

    View Slide

16. ͱ͍͏͔…
    

    View Slide

17. View Slide

18. View Slide

19. ·ͱΊ
    TextKit 1
    จࣈྻͷൣғͱϏϡʔͷۣܗ͕άϦϑ୯ҐͰରԠ෇ΒΕΔ
    
    
    จࣈྻͷൣғͱάϦϑͷൣғ͕ରԠ͚ͮΒΕͳ͍ݴޠ͕ଘࡏ
    ͍ͯ͠Δ
    
    
    ਖ਼֬ʹಈ࡞͠ͳ͍έʔε͕͋Δ
    

    View Slide

20. TextKit 2
    

    View Slide

21. ಛ௃
    TextKit 2
    NSTextLayoutManager Λத৺ʹ৽͘͠௥Ճ͞Εͨ API Ͱɺ
    TextKit 1ͱ͸ಠཱͯ͠ಈ࡞͢Δ
    
    
    ΑΓਖ਼֬ʹಈ࡞͢ΔΑ͏ʹͳͬͨ
    
    
    ෆมͷ஋ΫϥεΛ࢖͏
    

    View Slide

22. NSTextStorage
    NSLayoutManager
    NSTextContainer
    UITextView Ϗϡʔ
    Ϟσϧ
    ίϯτϩʔϥʔ
    

    View Slide

23. NSTextContentStorage
    NSTextLayoutManager
    NSTextContainer
    UITextView Ϗϡʔ
    Ϟσϧ
    ίϯτϩʔϥʔ
    

    View Slide

24. let textContentStorage = NSTextContentStorage()
    
    
    let textLayoutManager = NSTextLayoutManager()
    
    
    textContentStorage
    

    .addTextLayoutManager(textLayoutManager)
    
    
    let textContainer = NSTextContainer()
    
    
    textLayoutManager.textContainer = textContainer
    
    
    let textView = UITextView(
    

    frame: view.bounds,
    

    textContainer: textContainer
    
    
    )
    

    View Slide

25. จࣈྻͱϨΠΞ΢τ
    

    View Slide

26. ಛ௃
    TextKit 2
    ߴ౓ʹந৅Խ͞ΕͨΠϯλʔϑΣΠε
    
    
    TextKit 1 ͷૢ࡞ͷ࠷খ୯ҐͰ͋ͬͨάϦϑ͸Ӆṭ͞ΕɺάϦ
    ϑͷ৘ใ΍ͦͷۣܗͳͲ͸Ұ੾खʹೖΒͳ͘ͳͬͨ
    
    
    TextKit 2 ͷૢ࡞ͷ࠷খ୯Ґ͸ߦ
    

    View Slide

27. NSTextStorage

    NSAttributedString
    NSTextContentStorage
    NSTextLayoutFragment
    NSTextLineFragment
    

    View Slide

28. TextKit 2
    จࣈྻͱϨΠΞ΢τ
    ೚ҙͷจࣈྻͷൣғ͔ΒɺߦΑΓࡉ͔͍ը໘্ͷۣܗΛٻΊ
    Δ͜ͱ͸Ͱ͖ͳ͍
    ը໘্ͷ೚ҙͷҐஔ͔Βจࣈྻ্ͷൣғʹରԠ͢Δ͔Ͳ͏͔
    ͱ͍ͬͨώοτςετ͸ಠࣗͷΞτϦϏϡʔτΛ࢖͏͜ͱͰ
    ͋Δఔ౓Մೳ
    

    View Slide

29. จࣈྻ͔ΒϨΠΞ΢τ
    

    View Slide

30. TextKit 2
    จࣈྻ͔ΒϨΠΞ΢τ
    NSTextLayoutManager ͷ
    enumerateTextLayoutFragments(from:options:u
    sing) ͰจࣈྻͷҐஔ͔Β NSTextLayoutFragment Λ
    Ұཡ
    
    
    textLineFragments Ͱ NSTextLineFragment ΛҰཡ
    
    
    NSTextLineFragment ͷ typographicBounds Λ࢖͏
    

    View Slide

31. όά͕͋Γ·͢
    typographicBounds
    typographicBounds ͸ layoutFragmentFrame ͷ
    origin ͔Βͷ࠲ඪͷ͸ͣ
    
    
    iOS 17 Ͱ͸ͦͷ௨Γʹͳ͍ͬͯΔ
    
    
    iOS 16 Ͱ͸ x ํ޲͚ͩ 0.0 ͔Βͷ࠲ඪʹͳ͍ͬͯΔ
    

    View Slide

32. ޗഐ͸ೣͰ͋Δɻ໊લ͸·ͩແ
    ͍ɻ
    

    Ͳ͜ͰੜΕ͔ͨͱΜͱݟ౰͕ͭ
    ͔͵ɻԿͰ΋ബ҉͍͡Ί͡Ί͠
    ͨॴͰχϟʔχϟʔٽ͍͍ͯͨ
    ࣄ͚ͩ͸هԱ͍ͯ͠Δɻ
    Text Container
    layoutFragmentFrame
    typographicBounds
    

    View Slide

33. if #available(iOS 16.0, *) {
    
    
    let textLayoutManager = textView.textLayoutManager!
    
    
    let textContentStorage = textLayoutManager.textContentManager
    
    
    textLayoutManager.enumerateTextLayoutFragments(
    

    from: textContentStorage?.documentRange.location
    

    ) { layoutFragment in
    
    
    for lineFragment in layoutFragment.textLineFragments {
    
    
    let bounds: CGRect
    
    
    if #available(iOS 17.0, *) {
    
    
    bounds = lineFragment.typographicBounds
    
    
    } else {
    
    
    bounds = lineFragment.typographicBounds.offsetBy(
    
    
    dx: -layoutFragment.layoutFragmentFrame.minX,
    
    
    dy: 0.0
    
    
    )
    
    
    }
    
    
    // Use `bounds`...
    
    
    }
    
    
    return true
    
    
    }
    
    
    

    View Slide

34. ϨΠΞ΢τ͔Βจࣈྻ
    

    View Slide

35. TextKit 2
    ϨΠΞ΢τ͔Βจࣈྻ
    NSTextLayoutManager ͷ textLayoutFragment(for:)
    Λ࢖ͬͯ NSLayoutFragment ΛऔಘɺlineFragments Λ
    ࢖ͬͯͦΕͧΕͷ typographicBounds Λώοτςετ
    NSLineFragment ͷ characterIndex(for:) Λ࢖ͬͯจ
    ࣈྻ಺ͷҐஔΛऔಘ
    
    
    attributes(at:effectiveRange:)Ͱจࣈྻ্ͷӨڹൣ
    ғΛऔಘ
    

    View Slide

36. όά͕͋Γ·͢
    typographicBounds
    typographicBounds ͸ layoutFragmentFrame ͷ
    origin ͔Βͷ࠲ඪͷ͸ͣ
    
    
    iOS 17 Ͱ͸ͦͷ௨Γʹͳ͍ͬͯΔ
    
    
    iOS 16 Ͱ͸ x ํ޲͚ͩ 0.0 ͔Βͷ࠲ඪʹͳ͍ͬͯΔ
    

    View Slide

37. let attributedText = NSMutableAttributedString(
    

    "ޗഐ͸ೣͰ͋Δɻ໊લ͸·ͩແ͍"
    

    )
    
    
    let userNameAttributeKey = NSAttributedString.Key(
    

    rawValue: "UserName"
    

    )
    

    attributedText.addAttribute(
    

    userNameAttributeKey,
    

    value: "@neko",
    

    range: NSRange(location: 0, length: 2)
    

    )
    

    

    textView.attributedText = attributedText
    

    

    ...
    

    View Slide

38. let pointInTextView = // ...
    
    
    let pointInTextContainer = CGPoint(
    
    
    x: pointInTextView.x - textView.textContainerInset.left,
    
    
    y: pointInTextView.y - textView.textContainerInset.top
    
    
    )
    
    
    

    if let layoutFragment = textView.textLayoutManager?
    

    .textLayoutFragment(
    

    for: pointInTextContainer
    

    ) {
    
    
    let pointInLayoutFragment = CGPoint(
    
    
    x: pointInTextContainer.x -
    

    layoutFragment.layoutFragmentFrame.minX,
    
    
    y: pointInTextContainer.y -
    

    layoutFragment.layoutFragmentFrame.minY
    
    
    )
    
    
    ...
    

    View Slide

39. ...
    
    
    for lineFragment in layoutFragment.textLineFragments {
    
    
    let typographicBounds: CGRect
    
    
    if #available(iOS 17.0, *) {
    
    
    typographicBounds = lineFragment.typographicBounds
    
    
    } else {
    
    
    typographicBounds =
    

    lineFragment.typographicBounds.offsetBy(
    
    
    dx: -layoutFragment.layoutFragmentFrame.minX,
    
    
    dy: 0.0
    
    
    )
    
    
    }
    
    
    ...
    

    View Slide

40. ...
    

    if typographicBounds.contains(pointInLayoutFragment) {
    
    
    let pointInLineFragment = CGPoint(
    
    
    x: pointInLayoutFragment.x - typographicBounds.minX,
    
    
    y: pointInLayoutFragment.y - typographicBounds.minY
    
    
    )
    
    
    let characterIndex = lineFragment.characterIndex(
    

    for: pointInLineFragment
    

    )
    
    
    var characterRange =
    

    NSRange(location: NSNotFound, length: 0)
    
    
    let attributes = textView.attributedText.attributes(
    

    at: characterIndex,
    

    effectiveRange: &characterRange
    

    )
    
    
    if let userName = attributes[userNameAttributeKey] {
    
    
    // Use `userName` and `characterRange` ...
    
    
    }
    
    
    }
    
    
    }
    
    
    }
    

    View Slide

41. ͭΒΈ͕ଟ͍ͷͰ͕͢…
    

    View Slide

42. ศརʹͳͬͨػೳ
    

    View Slide

43. TextKit 2 ͷศརʹͳͬͨػೳ
    NSTextAttachmentViewProvider
    NSTextAttachmentViewProvider Λ࢖ͬͯ೚ҙͷϏϡʔΛ؆୯
    ʹจதʹຒΊࠐΊΔΑ͏ʹͳͬͨ
    
    
    TextKit 1 Ͱ͸ը૾͚͚ͩͩͰɺ೚ҙͷϏϡʔ͸ؤுͬͯࣗ෼
    ͰϨΠΞ΢τ͢Δඞཁ͕͋ͬͨ
    

    View Slide

44. View Slide

45. final class PlayerView: UIView {
    
    
    // ...
    
    
    override init(frame: CGRect) {
    
    
    // ...
    
    
    let player = AVQueuePlayer(playerItem: playerItem)
    
    
    playerLayer = AVPlayerLayer(player: player)
    
    
    playerLayer.videoGravity = .resizeAspectFill
    
    
    super.init(frame: frame)
    
    
    layer.addSublayer(playerLayer)
    
    
    player.play()
    
    
    }
    
    
    

    // ...
    
    
    }
    

    View Slide

46. final class AttachmentViewProvider: NSTextAttachmentViewProvider {
    
    
    override func loadView() {
    
    
    view = PlayerView()
    
    
    }
    
    
    override func attachmentBounds(
    
    
    for attributes: [NSAttributedString.Key : Any],
    
    
    location: NSTextLocation,
    
    
    textContainer: NSTextContainer?,
    
    
    proposedLineFragment: CGRect,
    
    
    position: CGPoint
    
    
    ) -> CGRect {
    
    
    CGRect(
    

    x: 0.0,
    

    y: 0.0,
    

    width: proposedLineFragment.width,
    

    height: 100.0
    

    )
    
    
    }
    
    
    }
    

    View Slide

47. final class Attachment: NSTextAttachment {
    
    
    override func viewProvider(
    
    
    for parentView: UIView?,
    
    
    location: NSTextLocation,
    
    
    textContainer: NSTextContainer?
    
    
    ) -> NSTextAttachmentViewProvider? {
    
    
    let viewProvider = AttachmentViewProvider(
    
    
    textAttachment: self,
    
    
    parentView: parentView,
    
    
    textLayoutManager: textContainer?.textLayoutManager,
    
    
    location: location
    
    
    )
    
    
    viewProvider.tracksTextAttachmentViewBounds = true
    
    
    return viewProvider
    
    
    }
    
    
    }
    

    View Slide

48. let attributedText = NSMutableAttributedString()
    
    
    

    attributedText.append(NSAttributedString(
    

    string: "ޗഐ͸ೣͰ͋Δɻ",
    

    attributes: attributes
    

    ))
    
    
    attributedText.append(NSAttributedString(
    

    attachment: Attachment()
    

    ))
    
    
    attributedText.append(NSAttributedString(
    

    string: "໊લ͸·ͩແ͍",
    
    
    attributes: attributes
    

    ))
    

    View Slide

49. ஫ҙ఺
    NSTextAttachmentViewProvider
    NSAttributedString ʹΑͬͯɺຒΊࠐΉϏϡʔΛؚΊ
    ͯϦιʔε͕ॴ༗͞ΕΔ
    
    
    ϝϞϦϦʔΫ͕ى͜Βͳ͍Α͏ʹɺ॥؀ࢀরʹ஫ҙ
    

    View Slide

50. ޓ׵ੑ
    

    View Slide

51. ରԠ͢Δ Ϗϡʔ
    iOS ͷόʔδϣϯ
    iOS 14 iOS 15 iOS 16 Ҏ߱
    UITextField
    􀁣
    􀁣
    UITextView
    􀁣
    

    View Slide

52. UITextView
    ࣗಈϑΥʔϧόοΫ
    iOS 15 ͸ TextKit 1 ͚ͩ
    
    
    iOS 16 Ҏ߱͸ TextKit 2 ͕σϑΥϧτ
    ͔͠͠ɺTextKit 1 ͷ API ʹͪΐͬͱͰ΋৮ΕΔͱ TextKit
    1 ʹ੾ΓସΘͬͯ͠·͏
    
    
    Ұ౓ TextKit 1 ʹͳΔͱɺTextKit 2 ʹ͸໭Βͳ͍
    

    View Slide

53. TextKit 1 ͚ͩΛ࢖͏৔߹
    TextKit Λ࢖͏
    ॳظԽ࣌ʹ TextKit 1 Λ࢖͏Α͏ʹ͢Δ
    
    
    ࣮ߦ࣌ʹίετͷ͔͔ΔࣗಈϑΥʔϧόοΫΛىͤ͜͞ͳ͍
    

    View Slide

54. TextKit 2 Λ࢖͏৔߹
    TextKit Λ࢖͏
    TextKit 2 ͚ͩΛ࢖͏৔߹
    
    
    UITextView Λ೿ੜͤͯ͞ɺTextKit 1 ͷ API Λ௵͢
    

    _UITextViewEnablingCompatibilityMode Ͱ֬ೝ
    ࣮ߦ࣌ʹ൑அ͢Δ৔߹
    #available(iOS 16.0, *) Ͱ iOS 16 Λ࣮ߦ࣌ʹࣝผ
    

    View Slide

55. // TextKit 1 ͚ͩΛ࢖͏৔߹
    
    
    let textView = UITextView(usingTextLayoutManager: false)
    
    
    // TextKit 2 ͚ͩΛ࢖͏৔߹
    
    
    final class TextView: UITextView {
    
    
    override var layoutManager: NSLayoutManager {
    
    
    fatalError("ͩΊͰ͢")
    
    
    }
    
    
    }
    

    View Slide

56. // ৚݅ʹΑͬͯ੾Γସ͑Δ৔߹
    
    
    let textView: UITextView
    
    
    if #available(iOS 16.0, *) {
    
    
    textView = UITextView(usingTextLayoutManager: true)
    

    } else {
    

    textView = UITextView()
    
    
    }
    

    final class TextView: UITextView {
    
    
    override var layoutManager: NSLayoutManager {
    
    
    if #available(iOS 16.0, *) {
    
    
    fatalError("ͩΊͰ͢")
    
    
    } else {
    
    
    super.layoutManager
    
    
    }
    
    
    }
    
    
    }
    

    View Slide

57. ·ͱΊ
    TextKit 2
    UITextView Ͱ iOS 16 Ҏ߱Ͱ TextKit 2 ͕࢖͑ΔΑ͏ʹͳͬͨ
    
    
    ࠷খ୯Ґ͕ߦͳͷͰɺͦΕҎԼͷૢ࡞͕ඞཁͳ৔߹͸޻෉͢
    Δඞཁ͕͋Δ
    Ґஔܭࢉ͕ iOS 16 Ͱ͸όάͬͯΔ
    TextKit 1 ʹࣗಈϑΥʔϧόοΫ͢ΔͷͰڍಈΛ؅ཧ͢Δ
    

    View Slide

58. SwiftUI
    

    View Slide

59. Text, TextEditor
    SwiftUI
    ඇৗʹ൥ࡶͳ AttributedString ͕ Text ϏϡʔͰ࢖͑Δ͕ػ
    ೳ͕ݶΒΕΔ
    
    
    NSTextAttachment ͸࢖͑ͳ͍
    TextKit ͷ API ΁ͷΞΫηε͸ͳ͍
    

    View Slide

60. ·ͱΊ
    SwiftUI
    TextKit ʹ͸ରԠͯ͠ͳ͍
    Text Ϗϡʔ΍ TextEditor Ϗϡʔ͸ػೳ͕͔ͳΓݶΒΕΔ
    
    
    TextKit ͷػೳ͕ඞཁͳ৔߹͸ૉ௚ʹ UITextView ΛຒΊࠐΉ
    

    View Slide

61. ͱ͍͏͜ͱͰ…
    

    View Slide

62. ·ͱΊ
    TextKit 1 ͸จࣈྻ͔ΒάϦϑ୯ҐͰૢ࡞Մೳɻͨͩ͠ਖ਼͘͠
    ಈ࡞͠ͳ͍έʔε͕͋Δ
    
    
    TextKit 2 ͸ศརͳػೳ͕௥Ճ͞Ε͍ͯΔ͕ɺจࣈྻ͔Βߦ୯
    ҐͰͷૢ࡞ʹݶΒΕΔɻtypographicBounds ͸όάͬͯΔ
    SwiftUI ͷText, TextEditor Ϗϡʔ͸ػೳ͕ݶΒΕΔ
    

    View Slide

63. ·ͱΊ
    ͲΕ΋׬ᘳͰ͸ͳ͍
    ΞϓϦέʔγϣϯͷ໨త΍ඞཁͳػೳʹԠͯ͡ɺTextKit 1,
    TextKit 2, SwiftUI ΛબͿ
    

    View Slide