Slide 1

Slide 1 text

Unpacking Compose Multiplatform Accessibility Colin Marsch - @colinmarsch

Slide 2

Slide 2 text

Hey, I’m Colin Marsch Android Engineer @ Cash App

Slide 3

Slide 3 text

Talk Overview • Compose Multiplatform overview • UI construction • Accessibility handling • Testing

Slide 4

Slide 4 text

What is Compose Multiplatform?

Slide 5

Slide 5 text

Kotlin Multiplatform Shared Logic Compose Multiplatform Shared UI

Slide 6

Slide 6 text

var showImage by remember { mutableStateOf(false) } Column { Button(onClick = { showImage = !showImage }) { Text("Toggle image") } AnimatedVisibility(showImage) { Image("Compose Multiplatform Logo") } }

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

How does Compose Multiplatform build UI?

Slide 9

Slide 9 text

On Android it’s just Compose UI

Slide 10

Slide 10 text

class MainActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContent { App() } } }

Slide 11

Slide 11 text

class MainActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContent { App() } } }

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

What about iOS?

Slide 14

Slide 14 text

fun MainViewController() : UIViewController = ComposeUIViewController { App() }

Slide 15

Slide 15 text

Skiko

Slide 16

Slide 16 text

Canvas Implementation Pros • Identical UI between platforms • UI won’t change during OS version updates • Less platform speci fi c code Cons • Possible performance overhead • Native UI interoperability complexity • Accessibility complexity

Slide 17

Slide 17 text

Native iOS UI Interoperability

Slide 18

Slide 18 text

Column { Text("How to use UIKitView inside Compose Multiplatform") UIKitView( factory = { MKMapView() }, modifier = Modifier.border(2.dp, Color.Blue).size(300.dp), ) }

Slide 19

Slide 19 text

Column { Text("How to use UIKitView inside Compose Multiplatform") UIKitView( factory = { MKMapView() }, modifier = Modifier.border(2.dp, Color.Blue).size(300.dp), ) }

Slide 20

Slide 20 text

You can also use SwiftUI within Compose Multiplatform

Slide 21

Slide 21 text

How is Compose Multiplatform related to accessibility?

Slide 22

Slide 22 text

Android Accessibility Tree

Slide 23

Slide 23 text

AccessibilityNodeInfo

Slide 24

Slide 24 text

AccessibilityNodeInfo.getContentDescription()

Slide 25

Slide 25 text

AccessibilityNodeInfo.getContentDescription() AccessibilityNodeInfo.isSelected()

Slide 26

Slide 26 text

AccessibilityNodeInfo.getContentDescription() AccessibilityNodeInfo.isSelected() AccessibilityNodeInfo.isImportantForAccessibility()

Slide 27

Slide 27 text

SemanticsNode

Slide 28

Slide 28 text

SemanticsNode.config.get(SemanticsProperties.ContentDescription)

Slide 29

Slide 29 text

SemanticsNode.config.get(SemanticsProperties.ContentDescription) SemanticsNode.config.get(SemanticsProperties.Selected)

Slide 30

Slide 30 text

SemanticsNode.config.get(SemanticsProperties.ContentDescription) SemanticsNode.config.get(SemanticsProperties.Selected) SemanticsNode.config.get(SemanticsProperties.Heading)

Slide 31

Slide 31 text

SemanticsNode -> AccessibilityNodeInfo

Slide 32

Slide 32 text

Compose Multiplatform == Compose UI

Slide 33

Slide 33 text

iOS Accessibility Tree

Slide 34

Slide 34 text

UIAccessibilityElement

Slide 35

Slide 35 text

UIAccessibilityElement.accessibilityLabel

Slide 36

Slide 36 text

UIAccessibilityElement.accessibilityLabel UIAccessibilityElement.accessibilityHint

Slide 37

Slide 37 text

UIAccessibilityElement.accessibilityLabel UIAccessibilityElement.accessibilityHint UIAccessibilityElement.accessibilityTraits.contains(.selected)

Slide 38

Slide 38 text

SemanticsNodes -> UIAccessibilityElement*

Slide 39

Slide 39 text

SemanticsNodes -> UIAccessibilityElement* * Lazily

Slide 40

Slide 40 text

Mapping iOS properties

Slide 41

Slide 41 text

override fun accessibilityLabel(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityLabel) { val contentDescription = config.getOrNull(SemanticsProperties.ContentDescription) if (contentDescription != null) { contentDescription } else { val editableText = config.getOrNull(SemanticsProperties.EditableText)?.text editableText ?: config.getOrNull(SemanticsProperties.Text) ?.joinToString("\n") { it.text } } }

Slide 42

Slide 42 text

override fun accessibilityLabel(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityLabel) { val contentDescription = config.getOrNull(SemanticsProperties.ContentDescription) if (contentDescription != null) { contentDescription } else { val editableText = config.getOrNull(SemanticsProperties.EditableText)?.text editableText ?: config.getOrNull(SemanticsProperties.Text) ?.joinToString("\n") { it.text } } }

Slide 43

Slide 43 text

override fun accessibilityLabel(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityLabel) { val contentDescription = config.getOrNull(SemanticsProperties.ContentDescription) if (contentDescription != null) { contentDescription } else { val editableText = config.getOrNull(SemanticsProperties.EditableText)?.text editableText ?: config.getOrNull(SemanticsProperties.Text) ?.joinToString("\n") { it.text } } }

Slide 44

Slide 44 text

override fun accessibilityLabel(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityLabel) { val contentDescription = config.getOrNull(SemanticsProperties.ContentDescription) if (contentDescription != null) { contentDescription } else { val editableText = config.getOrNull(SemanticsProperties.EditableText)?.text editableText ?: config.getOrNull(SemanticsProperties.Text) ?.joinToString("\n") { it.text } } }

Slide 45

Slide 45 text

override fun accessibilityHint(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityHint) { cachedConfig.getOrNull(SemanticsActions.OnClick)?.label }

Slide 46

Slide 46 text

override fun accessibilityHint(): String? = getOrElse(CachedAccessibilityPropertyKeys.accessibilityHint) { cachedConfig.getOrNull(SemanticsActions.OnClick)?.label }

Slide 47

Slide 47 text

override fun accessibilityTraits(): UIAccessibilityTraits = getOrElse(CachedAccessibilityPropertyKeys.accessibilityTraits) { config.getOrNull(SemanticsActions.OnClick)?.let { result = result or UIAccessibilityTraitButton } if (config.contains(SemanticsProperties.Heading)) { result = result or UIAccessibilityTraitHeader } }

Slide 48

Slide 48 text

override fun accessibilityTraits(): UIAccessibilityTraits = getOrElse(CachedAccessibilityPropertyKeys.accessibilityTraits) { config.getOrNull(SemanticsActions.OnClick)?.let { result = result or UIAccessibilityTraitButton } if (config.contains(SemanticsProperties.Heading)) { result = result or UIAccessibilityTraitHeader } }

Slide 49

Slide 49 text

override fun accessibilityTraits(): UIAccessibilityTraits = getOrElse(CachedAccessibilityPropertyKeys.accessibilityTraits) { config.getOrNull(SemanticsActions.OnClick)?.let { result = result or UIAccessibilityTraitButton } if (config.contains(SemanticsProperties.Heading)) { result = result or UIAccessibilityTraitHeader } }

Slide 50

Slide 50 text

Native iOS UI Accessibility Interoperability

Slide 51

Slide 51 text

UIKitView( factory = { MKMapView() }, modifier = Modifier.semantics { contentDescription = "Map of NYC” }, properties = UIKitInteropProperties( // false by default isNativeAccessibilityEnabled = false, ), )

Slide 52

Slide 52 text

UIKitView( factory = { MKMapView() }, properties = UIKitInteropProperties( isNativeAccessibilityEnabled = true, ), )

Slide 53

Slide 53 text

Testing Compose Multiplatform Accessibility

Slide 54

Slide 54 text

Android Accessibility Testing

Slide 55

Slide 55 text

App built by Google • Highlights accessibility issues via an overlay on top of the running app • Must install this app on your device or emulator • Performs pass/fail accessibility checks on the UI • Color contrast, touch target size, missing labels, etc. Accessibility Scanner

Slide 56

Slide 56 text

Manual screen reader testing • Android’s main screen reader technology • Requires interpreting the results • Can uncover deeper navigation and interaction issues • Should always be a part of your development work fl ow Talkback

Slide 57

Slide 57 text

• Espresso accessibility checks • Paparazzi’s AccessibilityRenderExtension • Can integrate into your CI setup Automated Testing

Slide 58

Slide 58 text

iOS Accessibility Testing

Slide 59

Slide 59 text

Manual screen reader testing • iOS’ main screen reader technology • Requires interpreting the results • Can uncover deeper navigation and interaction issues • Should always be a part of your development work fl ow VoiceOver

Slide 60

Slide 60 text

XCode developer tool • Examine accessibility properties of UI elements • Requires interpreting the results • Supports iOS simulators • Similar to screen readers (i.e. Talkback/VoiceOver) in a written format Accessibility Inspector

Slide 61

Slide 61 text

• performAccessibilityAudit on XCUIApplication in UI tests • Performs the same audit that is able to be manually run from the Accessibility Inspector tool • Performs pass/fail checks similar to Android’s Espresso checks Automated Testing

Slide 62

Slide 62 text

iOS Simulator Testing

Slide 63

Slide 63 text

iOSMain > MainViewController.kt fun MainViewController() = ComposeUIViewController() { App() } iOS Simulator Testing

Slide 64

Slide 64 text

iOSMain > MainViewController.kt fun MainViewController() = ComposeUIViewController( configure = { accessibilitySyncOptions = Always() } ) { App() } iOS Simulator Testing

Slide 65

Slide 65 text

Want to learn more? • github.com/JetBrains/compose-multiplatform: Examples and documentation • github.com/JetBrains/compose-multiplatform-core: All the internals • Compose Multiplatform on iOS: On the Road to Stable - Sebastian Aigner • Tomorrow at 3:25 PM

Slide 66

Slide 66 text

Thank you! Q&A?