Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Behind the Scenes with Auto Layout or How to Solve Constraints with the Cassowary Algorithm (iOSConfSG)

Behind the Scenes with Auto Layout or How to Solve Constraints with the Cassowary Algorithm (iOSConfSG)

Auto Layout is the most powerful tool on iOS today, to support developers in creating adaptive user interfaces on iOS. The power of this great technology lies in how simple its foundations are. The engineers behind Auto Layout at Apple started to reveal more of the internals of their technology recently, to help us understand its principles better, in order to get the best performance out of our layout code. Let’s look more closely at how the Auto Layout engine solves constraints through exploring the Cassowary algorithm, that’s running under the hood, turning constraints into linear equations, and solving them. By doing the math ourselves, we’ll have a better understanding of what layouts are easier, or more difficult to satisfy. What coding practices can help us achieve best Auto Layout performance.

6190e163993110536deb0767f9f63fdb?s=128

Agnes Vasarhelyi

January 18, 2019
Tweet

Transcript

  1. BEHIND THE SCENES WITH AUTO LAYOUT or how to solve

    constraints with the Cassowary algorithm Agnes Vasarhelyi @vasarhelyia
  2. WWDC '18 High Performance Auto Layout

  3. Constraints Displayed content ?

  4. Constraints Displayed content ? WHAT

  5. Constraints Displayed content ? WHAT RESULT

  6. Constraints Displayed content ? WHAT RESULT HOW

  7. From developer to user Developer Auto Layout User Constraints Render

    Loop Adaptivity
  8. Your input ! • Satisfiable, non-ambiguous layouts • Size and

    position of content • Constraints (either code, or IB)
  9. The Render Loop Constraints 1. Layout 2. Display 3. layoutSubviews()

    setNeedsLayout() layoutIfNeeded() updateConstraints() setNeedsUpdateConstraints() updateConstraintsIfNeeded() draw(_ :) setNeedsDisplay()
  10. The Render Loop Constraints 1. Layout 2.

  11. Auto Layout Engine Window

  12. Auto Layout Engine Window View

  13. Auto Layout Engine Window AL Engine View

  14. Auto Layout Engine Window AL Engine View Constraint

  15. Auto Layout Engine Window AL Engine View Constraint Add

  16. Auto Layout Engine Window AL Engine View Constraint = Equation

    Add
  17. Auto Layout Engine Window AL Engine View Constraint = Equation

    Add Variables
  18. Auto Layout Engine View Values AL Engine What are the

    values for this frame? Stores values
  19. Auto Layout Engine View Values Value changed AL Engine What

    are the values for this frame? Stores values
  20. Auto Layout Engine View Values Value changed AL Engine What

    are the values for this frame? Stores values setNeedsLayout() layoutSubviews()
  21. –Ken Ferry, WWDC '18 “The engine is a layout cache

    and dependency tracker.”
  22. WWDC 2018 • Layout performance scales linearly • Constraint inequalities

    are not expensive • Error minimization has a cost • Constraint solver algorithm: Simplex (Cassowary)
  23. PlanGrid Papers • "Paper" club • One per week •

    You pick, you lead
  24. Cassowary paper ‘97 • Based on Simplex (40's) linear programming

    algorithm • Constraint solving • UI applications • Incremental, optimized
  25. Incremental, optimized

  26. Incremental, optimized

  27. Incremental, optimized

  28. Incremental, optimized

  29. Incremental, optimized

  30. • Similar problems repeatedly • Add / remove / edit

    efficient Performance Incremental, optimized
  31. Linear Programming • Maximizing, or minimizing of model • Requirements

    = linear equations/inequalities • Objective function = goal
  32. Constraint solving LP problem • Objective function minimizing distance /

    error • Constraints linear equations / inequalities
  33. Let's solve constraints! x b y a leading leading width

  34. Let's solve constraints! x b y a 10 15 30

  35. Let's solve constraints! x b y a xa = 15

  36. Let's solve constraints! x b y a xa = 15

    widtha = 30
  37. Let's solve constraints! x b y a xa = 15

    widtha = 30 xb = xa + widtha + 10
  38. Let's solve constraints! x b y a xa = 15

    widtha = 30 xb = xa + widtha + 10 xb = 15 + 30 + 10 = 55
  39. Let's solve constraints! x b y a xa = 15

    widtha = 30 xb = xa + widtha + 10 xb = 15 + 30 + 10 = 55
  40. xleft ≤ xmiddle ≤ xright xleft ≥ 0 xright ≤

    100 ≤ ≤ x y xmiddle xright xleft
  41. xmiddle = (xleft + xright ) 2 xleft + 10

    ≤ xright ≤ ≤ x y xmiddle xright xleft
  42. 1. Augmented Simplex Form 2. Basic Feasible Solved Form 3.

    Basic Feasible Solution 4. Simplex Optimization Cassowary
  43. 1. Augmented Simplex Form • Convert inequalities to equalities •

    Separating out possibly negative variables
  44. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100
  45. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100
  46. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr
  47. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr
  48. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr
  49. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr xr + s2 = 100
  50. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr xr + s2 = 100
  51. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr 0 ≤ xl , s1 , s2 xr + s2 = 100
  52. Converting to equalities 2xm = xl + xr xl +

    10 ≤ xr xl ≥ 0 xr ≤ 100 2xm = xl + xr xl + 10 + s1 = xr 0 ≤ xl , s1 , s2 xr + s2 = 100 slack variables * s1 , s2 :
  53. minimize xm − xl

  54. x y xm − xl xmiddle xright xleft

  55. 2xm = xl + xr xl + 10 + s1

    = xr 0 ≤ xl , s1 , s2 xr + s2 = 100 minimize xm − xl subject to
  56. • Cu, unrestricted: possibly negative • Cs, Simplex: non-negative •

    Substituting in Cu to Cs Cu / Cs separation
  57. 2xm = xl + xr xl + 10 + s1

    = xr 0 ≤ xl , s1 , s2 xr + s2 = 100 Cu Cs Cu: unrestricted constraints, * CS: Simplex constraints
  58. 2xm = xl + xr xl + 10 + s1

    = xr 0 ≤ xl , s1 , s2 xr + s2 = 100 Cu Cs Cu: unrestricted constraints, * CS: Simplex constraints
  59. 2xm = xl + 100 − s2 xl + 10

    + s1 = 100 − s2 0 ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs Cu: unrestricted constraints, * CS: Simplex constraints
  60. 2xm = xl + 100 − s2 xl + 10

    + s1 = 100 − s2 0 ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs Cu: unrestricted constraints, * CS: Simplex constraints
  61. xl + 10 + s1 = 100 − s2 0

    ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs minimize subject to xm − xl 2xm = xl + 100 − s2 Cu: unrestricted constraints, * CS: Simplex constraints
  62. xl + 10 + s1 = 100 − s2 0

    ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs minimize subject to xm − xl Cu: unrestricted constraints, * CS: Simplex constraints
  63. xl + 10 + s1 = 100 − s2 0

    ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs minimize subject to xm = 50 + 1 2 xl − 1 2 s2 xm − xl Cu: unrestricted constraints, * CS: Simplex constraints
  64. xl + 10 + s1 = 100 − s2 0

    ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs minimize subject to xm = 50 + 1 2 xl − 1 2 s2 Cu: unrestricted constraints, * CS: Simplex constraints
  65. xl + 10 + s1 = 100 − s2 0

    ≤ xl , s1 , s2 xr = 100 − s2 Cu Cs minimize subject to xm = 50 + 1 2 xl − 1 2 s2 50 − 1 2 xl − 1 2 s2 Cu: unrestricted constraints, * CS: Simplex constraints
  66. 2. Basic feasible solved form x0 = c + a1

    x1 + . . . + an xn x0 is basic, x1 . . . xn are parameters (Cs) c ≥ 0
  67. s1 = xr = minimize subject to xm = 50

    − 1 2 xl − 1 2 s2 100 50 90 −s2 + 1 2 xl − 1 2 s2 −xl −s2 2. Basic feasible solved form xm , xr , s1 are basic, * xl , s2 are parametric
  68. s1 = xr = minimize subject to xm = 50

    − 1 2 xl − 1 2 s2 100 50 90 2. Basic feasible solved form xm , xr , s1 are basic, * xl , s2 are parametric
  69. 3. Basic feasible solution xr 100, xm 50, s1 90,

    xl 0, s2 0
  70. 3. Basic feasible solution xr 100, xm 50, s1 90,

    xl 0, s2 0 50 − 1 2 xl − 1 2 s2 Objective function
  71. 3. Basic feasible solution xr 100, xm 50, s1 90,

    xl 0, s2 0 50 − 1 2 xl − 1 2 s2 Objective function 50
  72. x y 50 100 0 xr 100 xm 50, xl

    0, 50 A basic feasible solution xmiddle xright xleft
  73. x y 50 100 0 xr 100 xm 50, xl

    0, 50 xm = (xl + xr ) 2 xl + 10 ≤ xr xl ≥ 0 xr ≤ 100 Original constraints A basic feasible solution xmiddle xright xleft
  74. 4. Simplex Optimization 1. Find (adjacent) basic feasible solved form

    2. Pivot: swap out basic and parametric vars fobj
  75. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 50 − 1 2 xl − 1 2 s2 100 90 −s2 50 + 1 2 xl − 1 2 s2 − −s2 xl =
  76. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 50 − 1 2 xl − 1 2 s2 100 90 −s2 50 + 1 2 xl − 1 2 s2 − −s2 xl =
  77. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 50 − 1 2 xl − 1 2 s2 100 90 −s2 50 + 1 2 xl − 1 2 s2 − −s2 xl =
  78. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 50 − 1 2 xl − 1 2 s2 100 90 −s2 − −s2 xl =
  79. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 50 − 1 2 xl − 1 2 s2 100 90 −s2 − −s2 xl = 95 − 1 2 s1 −s2
  80. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 100 90 −s2 − −s2 xl = 95 − 1 2 s1 −s2
  81. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 100 90 −s2 − −s2 xl = 5 + 1 2 s1 95 − 1 2 s1 −s2
  82. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 100 90 −s2 − −s2 xl = 5 + 1 2 s1 95 − 1 2 s1 −s2 xm , xr , xl are basic, * s1 , s2 are parametric
  83. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 100 90 −s2 − −s2 xl = 5 + 1 2 s1 95 − 1 2 s1 −s2 xm , xr , xl are basic, * s1 , s2 are parametric
  84. 4. Simplex Optimization s1 xr = minimize subject to xm

    = 100 90 −s2 − −s2 xl = 5 + 1 2 s1 95 − 1 2 s1 −s2 5 xm , xr , xl are basic, * s1 , s2 are parametric
  85. x y 90 100 0 90 95 xr 100 xm

    95, xl 90, 5 Optimal solution xm xr xl
  86. x y 90 100 0 90 95 xr 100 xm

    95, xl 90, 5 Optimal solution xm = (xl + xr ) 2 xl + 10 ≤ xr xl ≥ 0 xr ≤ 100 Original constraints xm xr xl
  87. None
  88. Adding constraints • Convert to augmented simplex form • Substitute

    new constraint in using current table • Reach basic feasible solved form
  89. Removing constraints • Marker variables to keep track of original

    constraints • Pivot until marker is basic • Remove row
  90. Constraint priorities • Weights added to constraints • Makes objective

    function non-linear • Quasi-linear optimization
  91. WWDC '18 • Performance scales linearly (in independent views) •

    Inequalities are not expensive (one extra variable) • Error minimization has a cost (constraint priorities)
  92. WWDC '18 • Performance scales linearly (in independent views) •

    Inequalities are not expensive (one extra variable) • Error minimization has a cost (constraint priorities)
  93. WWDC '18 • Performance scales linearly (in independent views) •

    Inequalities are not expensive (one extra variable) • Error minimization has a cost (constraint priorities)
  94. WWDC '18 • Performance scales linearly (in independent views) •

    Inequalities are not expensive (one extra variable) • Error minimization has a cost (constraint priorities)
  95. “You don’t pay for what you don’t use” –Ken Ferry,

    WWDC '18
  96. Agnes Vasarhelyi @vasarhelyia Thank you!

  97. Resources • Auto Layout Documentation • High Performance Auto Layout

    WWDC '18 • Cassowary paper '97 • The Simplex Method: An Example (UT Dallas)