Xuanyi
September 29, 2022
98

# On Notation

My slides for my DataEngConfAU 2022 talk, complete with additional material (FAQs and other supplementary material)

## Xuanyi

September 29, 2022

## Transcript

1. ### Follow @chewxy on Twitter बह#\$पते परथमं वाचो अ1ं यत परैरत

नामधेय6दधानाः यदेषां शरे;ठं यद=र>मासीA>ेणा तदेषां CनDहतं गुहाGवः On Notation (or YAML MUST DIE)

3. ### Follow @chewxy on Twitter How Do We Solve Problems? 1.

Simplify the problem 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
4. ### Follow @chewxy on Twitter How Do We Solve Problems? 1.

Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
5. ### Follow @chewxy on Twitter How Do We Solve Problems? 1.

Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem
6. ### Follow @chewxy on Twitter How Do We Solve Problems? 1.

Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem a. Combine solutions b. Increase resolution for the solution

8. ### Follow @chewxy on Twitter 「白马非马」，可乎？ 曰：可。 曰：何哉？ 曰：马者，所以命形也；白者 所以命色也。命色者非命形也 。故曰：「

白马非马」。 Language: A Unifying Theme
9. ### Follow @chewxy on Twitter What I’ve Been Thinking About Deep

Learning in Go https://gorgonia.org
10. ### Follow @chewxy on Twitter Neural Network Notations y = σ(Wx

+ b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩
11. ### Follow @chewxy on Twitter Neural Network Notations y = σ(Wx

+ b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩
12. ### Follow @chewxy on Twitter Abstraction, Abstraction, Abstraction* λ As an

ML (of the ISWIM kind) - 2017 * just don’t look under the lambda λ>> let x = Matrix(2,2) of float32 WithInit(GlorotN) λ>> let y = Matrix(2,2) of float32 WithInit(Uniform) λ>> let z = mul x y λ>> :∂ z (* manually trigger differentiation of z wrt its inputs *) λ>> (:deriv x) = y (* check that the partial deriv of x is equal to y *) True λ>> let id = fun x -> x in id x ...
13. ### Follow @chewxy on Twitter As An Algol Descendent (Go-like) -

2018 Commands, Commands Commands λ>> x = Matrix((2,2), Float32, WithInit(GlorotN)) λ>> y = Matrix((2,2), Float32, WithInit(Uniform)) λ>> z = MatMul(x, y) λ>> x.∂ == y // check that the partial deriv of x is equal to y true λ>> func id(x [a]) [a] { return x } id(x): ∀[a]: a → a ...
14. ### Follow @chewxy on Twitter As An APL - v1 2018

- 2022, v2 2022 life ← {⊃1 ⍵ ∨.∧ 3 4 = +/ +⌿ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵}
15. ### Follow @chewxy on Twitter As An APL - v1 2018

- 2022, v2 2022 x ← ⎕?GN 2 2 y ← ⎕?U 2 2 z ← x +.× y ∂ x = y ⍝ check that partial deriv of x = y 1

Go
18. ### Follow @chewxy on Twitter INiGo INiGo - Iverson Notation in

Go MoNTOYA - Materializer of Names That Otherwise Yield Aught
19. ### Follow @chewxy on Twitter INiGo - History • v1 -

MNG (2018 - 2022) • v2 - INiGo (current)
20. ### Follow @chewxy on Twitter INiGo • Slightly different notation from

traditional APLs (GNU APL or Dyalog APL) • Tree Calculus-like semantics and a Jay-style shapely type system • More symbols: ◦ ⚠ - Mark function as unsafe ◦ ∇ - Auto differentiation (𝖄 is used for recursion instead) ◦ ∂ - Partial derivative of a value ◦ 🀱 - Horizontal stack ◦ 🁣 - Vertical stack ◦ ⌘ - Command/Escape hatch ◦ ⍼ - Plot ◦ ⎄ - Unpivot

1 2
30. ### Follow @chewxy on Twitter On + and × 1 +

1 2 1 2 3 4 × 1 10 100 1000 1 20 300 4000

3 4 10

f

f

f a×d

f a×d b×e
37. ### Follow @chewxy on Twitter +.× a b c d e

f a×d b×e c×f
38. ### Follow @chewxy on Twitter +.× a b c d e

f a×d b×e c×f
39. ### Follow @chewxy on Twitter +.× a b c d e

f a×d + b×e + c×f

6 7 8 70
47. ### Follow @chewxy on Twitter 1 2 3 4 +.× 5

6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32
48. ### Follow @chewxy on Twitter 1 2 3 4 +.× 5

6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32 1 2 3 4 ,.{⍺'×'⍵','} 5 6 7 8 1 × 5 , 2 × 6 , 3 × 7 , 4 × 8 ,

51. ### Follow @chewxy on Twitter Things I Discovered By Virtue of

New Notation • Rediscovery of differentiation by means of skolemization of dual numbers. • S(·) → + → × → ✳ • Various isomorphisms that may be useful for developing neural networks.

in the small
55. ### Follow @chewxy on Twitter “Software engineering is programming, integrated over

time” - Titus Winters

59. ### Follow @chewxy on Twitter The Properties of a Good Notation

• Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
60. ### Follow @chewxy on Twitter The Properties of a Good Notation

• Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
61. ### Follow @chewxy on Twitter The Properties of a Good Notation

• Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
62. ### Follow @chewxy on Twitter The Properties of a Good Notation

• Ease of Expressing Constructs in Problem Solving • Suggestivity • Subordination of Detail • Economy • Amenable to Formal Proofs
63. ### Follow @chewxy on Twitter Multiple Notation In Use • Specification

and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers
64. ### Follow @chewxy on Twitter Multiple Notation In Use • Specification

and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL
65. ### Follow @chewxy on Twitter Multiple Notation In Use • Specification

and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL • Hierarchical use ◦ Functional core, imperative shell ◦ Imperative core, functional BAU (e.g. NaughtyDog’s Game Oriented Assembly Lisp )
66. ### Follow @chewxy on Twitter Multiple Notation In Use • Specification

and implementation ◦ Type language vs term language ◦ Googlers Protobuf transformers • Mixed computational models ◦ Regular expressions ◦ SQL • Hierarchical use ◦ Functional core, imperative shell ◦ Imperative core, functional BAU (e.g. NaughtyDog’s GOAL - OpenGOAL.dev) • DSLs and Mixins • Intensional and extensional views w/ periodic unification • Multi-surface languages (e.g. Racket, JetBrains MPS, etc)
67. ### Follow @chewxy on Twitter Poly(glot) Pleasure “A change in perspective

is worth 80 IQ points” - Alan Kay
68. ### Follow @chewxy on Twitter Properties of Good Notations • Good

interoperation/integration • Fit for purpose • Distinctive

1
70. ### Follow @chewxy on Twitter Good Interop Gold standard: 1 +

1 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx
71. ### Follow @chewxy on Twitter Good Interop Gold standard: 1 +

1 Whelmed standard: import arith .. arith.add(arith.one, arith.one) 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx
72. ### Follow @chewxy on Twitter Properties of Good Notations • Good

interoperation/integration • Fit for purpose • Distinctive
73. ### Follow @chewxy on Twitter “It can scarcely be denied that

the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” - Albert Einstein
74. ### Follow @chewxy on Twitter Properties of Good Notations • Good

interoperation/integration • Fit for purpose • Distinctive

76. ### Follow @chewxy on Twitter Recommendations • Invent specific languages to

solve your specific problem. • Invent multiple languages to solve one problem. • Take a hundred metre view: ◦ Survey similarities ◦ Examine exceptions ◦ Understand unifiers
77. ### Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf

instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds
78. ### Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf

instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds Deterministic Name Algebra }

80. ### Follow @chewxy on Twitter Takeaways • Language := rules +

elements that the rules work on. • Solving problems := establishing rules/constraints on a bunch of things. • Notation := how language is written. • Changing the way we write changes the way we solve problems. • Use more different notations to help us think through and solve problems. • Don’t add more cruft to the already fragile ecosystem.
81. ### Follow @chewxy on Twitter @chewxy on Twitter Thank You Mei

Lee Gary Miller Sarah Catanzaro Dane Laban Gajan Suthokumar Adam Burgess Daniel Owji Darrell Chua Gareth Seneque

83. ### Follow @chewxy on Twitter FAQs - On INiGo’s Creation •

Why not lisp/Have you heard of Racket? ◦ (((o(°▽°)o))) • Why not Rust? ◦ People keep comparing Rust to Go. They should really compare Rust with chess. Both are tactical games. Go is more of a strategy game. • Why not Haskell/Ocaml? ◦ Both are pure functional programming languages. Ocaml is pure, and Haskell is extra virgin. I can’t afford the purity because I have a day job. • Why don’t you use Dyalog APL? ◦ I don’t understand their business model, which appears to be stuck 20 years in the past. Plus, I control my own language.
84. ### Follow @chewxy on Twitter FAQs - On Use of INiGo

and APLs • How do you type INiGo or APL code? ◦ With a keyboard. Specifically, all the keyboards I use either have QMK or ZMK as a firmware. The QMK keyboards have a special APL layer, which allows direct input. • Are the symbols necessary? ◦ Yes and no. They may be replaced by words or any other combination of symbols on a keyboard (like J-lang does). But I feel the symbols are especially well thought out and are suggestive of what they do. ◦ e.g ⍉ is transpose and ⌽ is reverse/rotate along the Y axis. These give visual cues as to what you’re doing.
85. ### Follow @chewxy on Twitter FAQs • What is the Sanskrit

text background in the first slide? ◦ Book 10 of the Rig Veda, on knowledge ◦ The verse reads: “Bṛhaspati, when the wise established name-giving - the first principles of Vāc (Language) - all that was excellent and pure was brought forth to light” • What is the Chinese text in the background of the Language slide? ◦ The White Horse Dialogue by Gongsun Long (320-250 B.C) ◦ The words observes a linguistic paradox - that a white horse is not a horse ◦ The crux of the paradox is summed up by Bill Clinton: “that depends on what your definition of ‘is’ is”.
86. ### Follow @chewxy on Twitter FAQs - on Representing AI-ish Things

• Have you heard of Dyna? Yes. I have also used it. It’s a declarative language for maths that I feel is better than traditional mathematical language. • The mathematical notations on neural networks aren’t all equally showing the same information! First, that’s not a question. And yes, I am aware of that. Surely I get to bias things a little to make my point.
87. ### Follow @chewxy on Twitter FAQs • What is with the

scary head picture on the ALGOL slide? ◦ Algol is known as the demon star in the constellation of Perseus. ◦ Perseus killed Medusa, a Gorgon. The demon star refers to Medusa. ◦ Gorgonia was named after the Gorgons.
88. ### Follow @chewxy on Twitter FAQs on Multiple Notation • Didn’t

we already go through this with the mess of UML and the like in the 90s? You want us to go back to writing specs like that? Yes. We lacked wisdom then. We are wiser now. Knowing when to use things like UML for problem solving and when to use it for specification is wisdom.

90. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change ✳ * *
91. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Traditionally: • Repeated S ⇒ + • Repeated + ⇒ × • Repeated × ⇒ *
92. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Traditionally: • Repeated S ⇒ + • Repeated + ⇒ × • Repeated × ⇒ * But: ab ≠ ba Compiler writers love the commutativity trick!
93. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity?
94. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b
95. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b ✓ (a ✳ b) ✳ c = a ✳ (b ✳ c) ✓ a ✳ b = b ✳ a ✓ a ✳ (b × c) = a ✳ b × a ✳ c Identity: e Inverse Identity: e1/ln(a)
96. ### Follow @chewxy on Twitter The Hierarchy of Operations in the

Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b ✓ (a ✳ b) ✳ c = a ✳ (b ✳ c) ✓ a ✳ b = b ✳ a ✓ a ✳ (b × c) = a ✳ b × a ✳ c Identity: e Inverse Identity: e1/ln(a) a, b, c > 0