Xuanyi
September 29, 2022
82

# On Notation

My slides for my DataEngConfAU 2022 talk, complete with additional material (FAQs and other supplementary material)

## Xuanyi

September 29, 2022

## Transcript

बह#\$पते परथमं वाचो अ1ं यत
परैरत नामधेय6दधानाः
यदेषां शरे;ठं यद=र>मासीA>ेणा
तदेषां CनDहतं गुहाGवः
On Notation
(or YAML MUST DIE)

On Problem Solving

How Do We Solve Problems?
1. Simplify the problem
2. Solve the simplified problem(s) recursively
3. Solve the initial problem

How Do We Solve Problems?
1. Simplify the problem
a. Breaking down the problem into component problems
b. Model the problem with less resolution
2. Solve the simplified problem(s) recursively
3. Solve the initial problem

How Do We Solve Problems?
1. Simplify the problem
a. Breaking down the problem into component problems
b. Model the problem with less resolution
2. Solve the simplified problem(s) recursively
3. Solve the initial problem

How Do We Solve Problems?
1. Simplify the problem
a. Breaking down the problem into component problems
b. Model the problem with less resolution
2. Solve the simplified problem(s) recursively
3. Solve the initial problem
a. Combine solutions
b. Increase resolution for the solution

How Do We Represent Problems?
github.com/owulveryck/wardleytogo

「白马非马」，可乎？
曰：可。
曰：何哉？
曰：马者，所以命形也；白者
所以命色也。命色者非命形也
。故曰：「 白马非马」。
Language:
A Unifying Theme

Deep Learning in Go
https://gorgonia.org

Neural Network Notations
y = σ(Wx + b)
yi
l = f(Σj
wi,j
l xi
l-1 + bi
l)
y = f(wi
xi + b)
y = f ⟨w|x⟩ + |b⟩

Neural Network Notations
y = σ(Wx + b)
yi
l = f(Σj
wi,j
l xi
l-1 + bi
l)
y = f(wi
xi + b)
y = f ⟨w|x⟩ + |b⟩

Abstraction, Abstraction, Abstraction*
λ
As an ML (of the ISWIM kind) - 2017
* just don’t look under the lambda
λ>> let x = Matrix(2,2) of float32 WithInit(GlorotN)
λ>> let y = Matrix(2,2) of float32 WithInit(Uniform)
λ>> let z = mul x y
λ>> :∂ z (* manually trigger differentiation of z wrt its inputs *)
λ>> (:deriv x) = y (* check that the partial deriv of x is equal to y
*)
True
λ>> let id = fun x -> x in id x
...

As An Algol Descendent (Go-like) - 2018
Commands, Commands Commands
λ>> x = Matrix((2,2), Float32, WithInit(GlorotN))
λ>> y = Matrix((2,2), Float32, WithInit(Uniform))
λ>> z = MatMul(x, y)
λ>> x.∂ == y // check that the partial deriv of x is equal to y
true
λ>> func id(x [a]) [a] { return x }
id(x): ∀[a]: a → a
...

As An APL - v1 2018 - 2022, v2 2022
life ← {⊃1 ⍵ ∨.∧ 3 4 = +/ +⌿ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵}

As An APL - v1 2018 - 2022, v2 2022
x ← ⎕?GN 2 2
y ← ⎕?U 2 2
z ← x +.× y
∂ x = y ⍝ check that partial deriv of x = y
1

INiGo

INiGo
INiGo - Iverson Notation in Go

INiGo
INiGo - Iverson Notation in Go
MoNTOYA - Materializer of Names That Otherwise Yield Aught

INiGo - History
● v1 - MNG (2018 - 2022)
● v2 - INiGo (current)

INiGo
● Slightly different notation from traditional APLs (GNU APL or Dyalog APL)
● Tree Calculus-like semantics and a Jay-style shapely type system
● More symbols:
○ ⚠ - Mark function as unsafe
○ ∇ - Auto differentiation (𝖄 is used for recursion instead)
○ ∂ - Partial derivative of a value
○ 🀱 - Horizontal stack
○ 🁣 - Vertical stack
○ ⌘ - Command/Escape hatch
○ ⍼ - Plot
○ ⎄ - Unpivot

Change The Way You Think

Think different.

+.×

+.×
Inner Product

+.×

+.×

+.×

+.×

On + and ×
1 + 1
2

On + and ×
1 + 1
2
1 2 3 4 × 1 10 100 1000
1 20 300 4000

Side Note: Reduction
+/ 1 2 3 4
10

+.×

+.×
a b c
d
e
f

+.×
a b c
d
e
f

+.×
a b c
d
e
f
a×d

+.×
a b c
d
e
f
a×d b×e

+.×
a b c
d
e
f
a×d b×e c×f

+.×
a b c
d
e
f
a×d b×e c×f

+.×
a b c
d
e
f
a×d + b×e + c×f

+.×

+.×

+.×

+.×

+.×

⋀.⋀

1 2 3 4 +.× 5 6 7 8
70

1 2 3 4 +.× 5 6 7 8
70
1 2 3 4 ,.× 5 6 7 8
5 12 21 32

1 2 3 4 +.× 5 6 7 8
70
1 2 3 4 ,.× 5 6 7 8
5 12 21 32
1 2 3 4 ,.{⍺'×'⍵','} 5 6 7 8
1 × 5 , 2 × 6 , 3 × 7 , 4 × 8 ,

Beautiful APL
c (×÷+.×) p

cp ← {({(⍳∘(⌈/⍵))⍵}|+\(⍵-(+/÷≢)⍵))⌷⍺}
Real Life APL

Things I Discovered By Virtue of New Notation
● Rediscovery of differentiation by means of skolemization of dual numbers.
● S(·) → + → × → ✳
● Various isomorphisms that may be useful for developing neural networks.

On Problems (Redux)

github.com/chewxy/annotation-mode.el

Programming in the large
vs
Programming in the small

“Software engineering is programming, integrated over
time”
- Titus Winters

On Good Notation(s)

The Properties of a Good Notation
● Ease of Expressing Constructs in Problem Solving
● Suggestivity
● Subordination of Detail
● Economy
● Amenable to Formal Proofs

The Properties of a Good Notation
● Ease of Expressing Constructs in Problem Solving
● Suggestivity
● Subordination of Detail
● Economy
● Amenable to Formal Proofs

The Properties of a Good Notation
● Ease of Expressing Constructs in Problem Solving
● Suggestivity
● Subordination of Detail
● Economy
● Amenable to Formal Proofs

The Properties of a Good Notation
● Ease of Expressing Constructs in Problem Solving
● Suggestivity
● Subordination of Detail
● Economy
● Amenable to Formal Proofs

Multiple Notation In Use
● Specification and implementation
○ Type language vs term language

Multiple Notation In Use
● Specification and implementation
○ Type language vs term language
● Mixed computational models
○ Regular expressions
○ SQL

Multiple Notation In Use
● Specification and implementation
○ Type language vs term language
● Mixed computational models
○ Regular expressions
○ SQL
● Hierarchical use
○ Functional core, imperative shell
○ Imperative core, functional BAU (e.g. NaughtyDog’s Game Oriented Assembly Lisp )

Multiple Notation In Use
● Specification and implementation
○ Type language vs term language
● Mixed computational models
○ Regular expressions
○ SQL
● Hierarchical use
○ Functional core, imperative shell
○ Imperative core, functional BAU (e.g. NaughtyDog’s GOAL - OpenGOAL.dev)
● DSLs and Mixins
● Intensional and extensional views w/ periodic unification
● Multi-surface languages (e.g. Racket, JetBrains MPS, etc)

Poly(glot) Pleasure
“A change in perspective is worth 80 IQ points”
- Alan Kay

Properties of Good Notations
● Good interoperation/integration
● Fit for purpose
● Distinctive

Good Interop
Gold standard:
1 + 1

Good Interop
Gold standard:
1 + 1
💩 standard:
λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx

Good Interop
Gold standard:
1 + 1
Whelmed standard:
import arith ..
💩 standard:
λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx

Properties of Good Notations
● Good interoperation/integration
● Fit for purpose
● Distinctive

“It can scarcely be denied that the supreme goal of all theory is to make the irreducible
basic elements as simple and as few as possible without having to surrender the
adequate representation of a single datum of experience.”
- Albert Einstein

Properties of Good Notations
● Good interoperation/integration
● Fit for purpose
● Distinctive

Reality Knocks On My Door…

Recommendations
● Invent specific languages to solve your specific problem.
● Invent multiple languages to solve one problem.
● Take a hundred metre view:
○ Survey similarities
○ Examine exceptions
○ Understand unifiers

Alternatives
dbt for “dependent” type checking of
databases
Malloy for “modular” SQL
Nix for reproducible builds

Alternatives
dbt for “dependent” type checking of
databases
Malloy for “modular” SQL
Nix for reproducible builds
Deterministic Name Algebra
}

Takeaways

Takeaways
● Language := rules + elements that the rules work on.
● Solving problems := establishing rules/constraints on a bunch of things.
● Notation := how language is written.
● Changing the way we write changes the way we solve problems.
● Use more different notations to help us think through and solve problems.

Thank You
Mei Lee
Gary Miller
Sarah Catanzaro
Dane Laban
Gajan Suthokumar
Daniel Owji
Darrell Chua
Gareth Seneque

Appendix

FAQs - On INiGo’s Creation
● Why not lisp/Have you heard of Racket?
○ (((o(°▽°)o)))
● Why not Rust?
○ People keep comparing Rust to Go. They should really compare Rust with chess. Both are tactical
games. Go is more of a strategy game.
○ Both are pure functional programming languages. Ocaml is pure, and Haskell is extra virgin. I
can’t afford the purity because I have a day job.
● Why don’t you use Dyalog APL?
○ I don’t understand their business model, which appears to be stuck 20 years in the past. Plus, I
control my own language.

FAQs - On Use of INiGo and APLs
● How do you type INiGo or APL code?
○ With a keyboard. Specifically, all the keyboards I use either have QMK or ZMK as a firmware. The
QMK keyboards have a special APL layer, which allows direct input.
● Are the symbols necessary?
○ Yes and no. They may be replaced by words or any other combination of symbols on a keyboard
(like J-lang does). But I feel the symbols are especially well thought out and are suggestive of
what they do.
○ e.g ⍉ is transpose and ⌽ is reverse/rotate along the Y axis. These give visual cues as to what
you’re doing.

FAQs
● What is the Sanskrit text background in the first slide?
○ Book 10 of the Rig Veda, on knowledge
“Bṛhaspati, when the wise established name-giving - the first principles of Vāc (Language) - all that
was excellent and pure was brought forth to light”
● What is the Chinese text in the background of the Language slide?
○ The White Horse Dialogue by Gongsun Long (320-250 B.C)
○ The words observes a linguistic paradox - that a white horse is not a horse
○ The crux of the paradox is summed up by Bill Clinton:
“that depends on what your definition of ‘is’ is”.

FAQs - on Representing AI-ish Things
● Have you heard of Dyna?
Yes. I have also used it. It’s a declarative language for maths that I feel is better
● The mathematical notations on neural networks aren’t all equally showing the
same information!
First, that’s not a question. And yes, I am aware of that. Surely I get to bias
things a little to make my point.

FAQs
● What is with the scary head picture on the ALGOL slide?
○ Algol is known as the demon star in the constellation of Perseus.
○ Perseus killed Medusa, a Gorgon. The demon star refers to Medusa.
○ Gorgonia was named after the Gorgons.

FAQs on Multiple Notation
● Didn’t we already go through this with the mess of UML and the like in the 90s?
You want us to go back to writing specs like that?
Yes. We lacked wisdom then. We are wiser now. Knowing when to use things
like UML for problem solving and when to use it for specification is wisdom.

Supplementary Material

The Hierarchy of Operations in the Arithmetic Universe is about to Change

*
*

The Hierarchy of Operations in the Arithmetic Universe is about to Change
● Repeated S ⇒ +
● Repeated + ⇒ ×
● Repeated × ⇒ *

The Hierarchy of Operations in the Arithmetic Universe is about to Change
● Repeated S ⇒ +
● Repeated + ⇒ ×
● Repeated × ⇒ *
But:
ab ≠ ba
Compiler writers love the commutativity trick!

The Hierarchy of Operations in the Arithmetic Universe is about to Change
Is there an operation ✳ that is a “next level”, and still maintains commutativity,
associativity and distributivity?

The Hierarchy of Operations in the Arithmetic Universe is about to Change
Is there an operation ✳ that is a “next level”, and still maintains commutativity,
associativity and distributivity?
a ✳ b = aln b

The Hierarchy of Operations in the Arithmetic Universe is about to Change
Is there an operation ✳ that is a “next level”, and still maintains commutativity,
associativity and distributivity?
a ✳ b = aln b
✓ (a ✳ b) ✳ c = a ✳ (b ✳ c)
✓ a ✳ b = b ✳ a
✓ a ✳ (b × c) = a ✳ b × a ✳ c
Identity: e
Inverse Identity: e1/ln(a)