On Notation - Speaker Deck

Slide 1

Slide 1 text

Follow @chewxy on Twitter बह#$पते परथमं वाचो अ1ं यत परैरत नामधेय6दधानाः यदेषां शरे;ठं यद=र>मासीA>ेणा तदेषां CनDहतं गुहाGवः On Notation (or YAML MUST DIE)

Slide 2

Slide 2 text

Follow @chewxy on Twitter On Problem Solving

Slide 3

Slide 3 text

Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem 2. Solve the simplified problem(s) recursively 3. Solve the initial problem

Slide 4

Slide 4 text

Follow @chewxy on Twitter How Do We Solve Problems? 1. Simplify the problem a. Breaking down the problem into component problems b. Model the problem with less resolution 2. Solve the simplified problem(s) recursively 3. Solve the initial problem

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Follow @chewxy on Twitter How Do We Represent Problems? github.com/owulveryck/wardleytogo

Slide 8

Slide 8 text

Follow @chewxy on Twitter 「白马非马」，可乎？曰：可。曰：何哉？曰：马者，所以命形也；白者所以命色也。命色者非命形也。故曰：「白马非马」。 Language: A Unifying Theme

Slide 9

Slide 9 text

Follow @chewxy on Twitter What I’ve Been Thinking About Deep Learning in Go https://gorgonia.org

Slide 10

Slide 10 text

Follow @chewxy on Twitter Neural Network Notations y = σ(Wx + b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩

Slide 11

Slide 11 text

Follow @chewxy on Twitter Neural Network Notations y = σ(Wx + b) yi l = f(Σj wi,j l xi l-1 + bi l) y = f(wi xi + b) y = f ⟨w|x⟩ + |b⟩

Slide 12

Slide 12 text

Follow @chewxy on Twitter Abstraction, Abstraction, Abstraction* λ As an ML (of the ISWIM kind) - 2017 * just don’t look under the lambda λ>> let x = Matrix(2,2) of float32 WithInit(GlorotN) λ>> let y = Matrix(2,2) of float32 WithInit(Uniform) λ>> let z = mul x y λ>> :∂ z (* manually trigger differentiation of z wrt its inputs *) λ>> (:deriv x) = y (* check that the partial deriv of x is equal to y *) True λ>> let id = fun x -> x in id x ...

Slide 13

Slide 13 text

Follow @chewxy on Twitter As An Algol Descendent (Go-like) - 2018 Commands, Commands Commands λ>> x = Matrix((2,2), Float32, WithInit(GlorotN)) λ>> y = Matrix((2,2), Float32, WithInit(Uniform)) λ>> z = MatMul(x, y) λ>> x.∂ == y // check that the partial deriv of x is equal to y true λ>> func id(x [a]) [a] { return x } id(x): ∀[a]: a → a ...

Slide 14

Slide 14 text

Follow @chewxy on Twitter As An APL - v1 2018 - 2022, v2 2022 life ← {⊃1 ⍵ ∨.∧ 3 4 = +/ +⌿ ¯1 0 1 ∘.⊖ ¯1 0 1 ⌽¨ ⊂⍵}

Slide 15

Slide 15 text

Follow @chewxy on Twitter As An APL - v1 2018 - 2022, v2 2022 x ← ⎕?GN 2 2 y ← ⎕?U 2 2 z ← x +.× y ∂ x = y ⍝ check that partial deriv of x = y 1

Slide 16

Slide 16 text

Follow @chewxy on Twitter INiGo

Slide 17

Slide 17 text

Follow @chewxy on Twitter INiGo INiGo - Iverson Notation in Go

Slide 18

Slide 18 text

Follow @chewxy on Twitter INiGo INiGo - Iverson Notation in Go MoNTOYA - Materializer of Names That Otherwise Yield Aught

Slide 19

Slide 19 text

Follow @chewxy on Twitter INiGo - History ● v1 - MNG (2018 - 2022) ● v2 - INiGo (current)

Slide 20

Slide 20 text

Follow @chewxy on Twitter INiGo ● Slightly different notation from traditional APLs (GNU APL or Dyalog APL) ● Tree Calculus-like semantics and a Jay-style shapely type system ● More symbols: ○ ⚠ - Mark function as unsafe ○ ∇ - Auto differentiation (𝖄 is used for recursion instead) ○ ∂ - Partial derivative of a value ○ 🀱 - Horizontal stack ○ 🁣 - Vertical stack ○ ⌘ - Command/Escape hatch ○ ⍼ - Plot ○ ⎄ - Unpivot

Slide 21

Slide 21 text

Follow @chewxy on Twitter Change The Way You Think

Slide 22

Slide 22 text

Follow @chewxy on Twitter Think different.

Slide 23

Slide 23 text

Follow @chewxy on Twitter +.×

Slide 24

Slide 24 text

Follow @chewxy on Twitter +.× Inner Product

Slide 25

Slide 25 text

Follow @chewxy on Twitter +.×

Slide 26

Slide 26 text

Follow @chewxy on Twitter +.×

Slide 27

Slide 27 text

Follow @chewxy on Twitter +.×

Slide 28

Slide 28 text

Follow @chewxy on Twitter +.×

Slide 29

Slide 29 text

Follow @chewxy on Twitter On + and × 1 + 1 2

Slide 30

Slide 30 text

Follow @chewxy on Twitter On + and × 1 + 1 2 1 2 3 4 × 1 10 100 1000 1 20 300 4000

Slide 31

Slide 31 text

Follow @chewxy on Twitter Side Note: Reduction +/ 1 2 3 4 10

Slide 32

Slide 32 text

Follow @chewxy on Twitter +.×

Slide 33

Slide 33 text

Follow @chewxy on Twitter +.× a b c d e f

Slide 34

Slide 34 text

Follow @chewxy on Twitter +.× a b c d e f

Slide 35

Slide 35 text

Follow @chewxy on Twitter +.× a b c d e f a×d

Slide 36

Slide 36 text

Follow @chewxy on Twitter +.× a b c d e f a×d b×e

Slide 37

Slide 37 text

Follow @chewxy on Twitter +.× a b c d e f a×d b×e c×f

Slide 38

Slide 38 text

Follow @chewxy on Twitter +.× a b c d e f a×d b×e c×f

Slide 39

Slide 39 text

Follow @chewxy on Twitter +.× a b c d e f a×d + b×e + c×f

Slide 40

Slide 40 text

Follow @chewxy on Twitter +.×

Slide 41

Slide 41 text

Follow @chewxy on Twitter +.×

Slide 42

Slide 42 text

Follow @chewxy on Twitter +.×

Slide 43

Slide 43 text

Follow @chewxy on Twitter +.×

Slide 44

Slide 44 text

Follow @chewxy on Twitter +.×

Slide 45

Slide 45 text

Follow @chewxy on Twitter ⋀.⋀

Slide 46

Slide 46 text

Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70

Slide 47

Slide 47 text

Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32

Slide 48

Slide 48 text

Follow @chewxy on Twitter 1 2 3 4 +.× 5 6 7 8 70 1 2 3 4 ,.× 5 6 7 8 5 12 21 32 1 2 3 4 ,.{⍺'×'⍵','} 5 6 7 8 1 × 5 , 2 × 6 , 3 × 7 , 4 × 8 ,

Slide 49

Slide 49 text

Follow @chewxy on Twitter Beautiful APL c (×÷+.×) p

Slide 50

Slide 50 text

Follow @chewxy on Twitter cp ← {({(⍳∘(⌈/⍵))⍵}|+\(⍵-(+/÷≢)⍵))⌷⍺} Real Life APL

Slide 51

Slide 51 text

Follow @chewxy on Twitter Things I Discovered By Virtue of New Notation ● Rediscovery of differentiation by means of skolemization of dual numbers. ● S(·) → + → × → ✳ ● Various isomorphisms that may be useful for developing neural networks.

Slide 52

Slide 52 text

Follow @chewxy on Twitter On Problems (Redux)

Slide 53

Slide 53 text

Follow @chewxy on Twitter github.com/chewxy/annotation-mode.el

Slide 54

Slide 54 text

Follow @chewxy on Twitter Programming in the large vs Programming in the small

Slide 55

Slide 55 text

Follow @chewxy on Twitter “Software engineering is programming, integrated over time” - Titus Winters

Slide 56

Slide 56 text

Follow @chewxy on Twitter

Slide 57

Slide 57 text

Follow @chewxy on Twitter On Good Notation(s)

Slide 58

Slide 58 text

Follow @chewxy on Twitter

Slide 59

Slide 59 text

Follow @chewxy on Twitter The Properties of a Good Notation ● Ease of Expressing Constructs in Problem Solving ● Suggestivity ● Subordination of Detail ● Economy ● Amenable to Formal Proofs

Slide 60

Slide 60 text

Follow @chewxy on Twitter The Properties of a Good Notation ● Ease of Expressing Constructs in Problem Solving ● Suggestivity ● Subordination of Detail ● Economy ● Amenable to Formal Proofs

Slide 61

Slide 61 text

Follow @chewxy on Twitter The Properties of a Good Notation ● Ease of Expressing Constructs in Problem Solving ● Suggestivity ● Subordination of Detail ● Economy ● Amenable to Formal Proofs

Slide 62

Slide 62 text

Follow @chewxy on Twitter The Properties of a Good Notation ● Ease of Expressing Constructs in Problem Solving ● Suggestivity ● Subordination of Detail ● Economy ● Amenable to Formal Proofs

Slide 63

Slide 63 text

Follow @chewxy on Twitter Multiple Notation In Use ● Specification and implementation ○ Type language vs term language ○ Googlers Protobuf transformers

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Follow @chewxy on Twitter Multiple Notation In Use ● Specification and implementation ○ Type language vs term language ○ Googlers Protobuf transformers ● Mixed computational models ○ Regular expressions ○ SQL ● Hierarchical use ○ Functional core, imperative shell ○ Imperative core, functional BAU (e.g. NaughtyDog’s Game Oriented Assembly Lisp )

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Follow @chewxy on Twitter Poly(glot) Pleasure “A change in perspective is worth 80 IQ points” - Alan Kay

Slide 68

Slide 68 text

Follow @chewxy on Twitter Properties of Good Notations ● Good interoperation/integration ● Fit for purpose ● Distinctive

Slide 69

Slide 69 text

Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1

Slide 70

Slide 70 text

Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx

Slide 71

Slide 71 text

Follow @chewxy on Twitter Good Interop Gold standard: 1 + 1 Whelmed standard: import arith .. arith.add(arith.one, arith.one) 💩 standard: λm.λn.λf.λx.mf(nfx) λf.λx.fx λf.λx.fx

Slide 72

Slide 72 text

Follow @chewxy on Twitter Properties of Good Notations ● Good interoperation/integration ● Fit for purpose ● Distinctive

Slide 73

Slide 73 text

Follow @chewxy on Twitter “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” - Albert Einstein

Slide 74

Slide 74 text

Follow @chewxy on Twitter Properties of Good Notations ● Good interoperation/integration ● Fit for purpose ● Distinctive

Slide 75

Slide 75 text

Follow @chewxy on Twitter Reality Knocks On My Door…

Slide 76

Slide 76 text

Follow @chewxy on Twitter Recommendations ● Invent specific languages to solve your specific problem. ● Invent multiple languages to solve one problem. ● Take a hundred metre view: ○ Survey similarities ○ Examine exceptions ○ Understand unifiers

Slide 77

Slide 77 text

Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds

Slide 78

Slide 78 text

Follow @chewxy on Twitter Alternatives Cue/Dhall instead of YAML Protobuf instead of json dbt for “dependent” type checking of databases Malloy for “modular” SQL Nix for reproducible builds Deterministic Name Algebra }

Slide 79

Slide 79 text

Follow @chewxy on Twitter Takeaways

Slide 80

Slide 80 text

Follow @chewxy on Twitter Takeaways ● Language := rules + elements that the rules work on. ● Solving problems := establishing rules/constraints on a bunch of things. ● Notation := how language is written. ● Changing the way we write changes the way we solve problems. ● Use more different notations to help us think through and solve problems. ● Don’t add more cruft to the already fragile ecosystem.

Slide 81

Slide 81 text

Follow @chewxy on Twitter @chewxy on Twitter Thank You Mei Lee Gary Miller Sarah Catanzaro Dane Laban Gajan Suthokumar Adam Burgess Daniel Owji Darrell Chua Gareth Seneque

Slide 82

Slide 82 text

Follow @chewxy on Twitter Appendix

Slide 83

Slide 83 text

Follow @chewxy on Twitter FAQs - On INiGo’s Creation ● Why not lisp/Have you heard of Racket? ○ (((o(°▽°)o))) ● Why not Rust? ○ People keep comparing Rust to Go. They should really compare Rust with chess. Both are tactical games. Go is more of a strategy game. ● Why not Haskell/Ocaml? ○ Both are pure functional programming languages. Ocaml is pure, and Haskell is extra virgin. I can’t afford the purity because I have a day job. ● Why don’t you use Dyalog APL? ○ I don’t understand their business model, which appears to be stuck 20 years in the past. Plus, I control my own language.

Slide 84

Slide 84 text

Follow @chewxy on Twitter FAQs - On Use of INiGo and APLs ● How do you type INiGo or APL code? ○ With a keyboard. Specifically, all the keyboards I use either have QMK or ZMK as a firmware. The QMK keyboards have a special APL layer, which allows direct input. ● Are the symbols necessary? ○ Yes and no. They may be replaced by words or any other combination of symbols on a keyboard (like J-lang does). But I feel the symbols are especially well thought out and are suggestive of what they do. ○ e.g ⍉ is transpose and ⌽ is reverse/rotate along the Y axis. These give visual cues as to what you’re doing.

Slide 85

Slide 85 text

Follow @chewxy on Twitter FAQs ● What is the Sanskrit text background in the first slide? ○ Book 10 of the Rig Veda, on knowledge ○ The verse reads: “Bṛhaspati, when the wise established name-giving - the first principles of Vāc (Language) - all that was excellent and pure was brought forth to light” ● What is the Chinese text in the background of the Language slide? ○ The White Horse Dialogue by Gongsun Long (320-250 B.C) ○ The words observes a linguistic paradox - that a white horse is not a horse ○ The crux of the paradox is summed up by Bill Clinton: “that depends on what your definition of ‘is’ is”.

Slide 86

Slide 86 text

Follow @chewxy on Twitter FAQs - on Representing AI-ish Things ● Have you heard of Dyna? Yes. I have also used it. It’s a declarative language for maths that I feel is better than traditional mathematical language. ● The mathematical notations on neural networks aren’t all equally showing the same information! First, that’s not a question. And yes, I am aware of that. Surely I get to bias things a little to make my point.

Slide 87

Slide 87 text

Follow @chewxy on Twitter FAQs ● What is with the scary head picture on the ALGOL slide? ○ Algol is known as the demon star in the constellation of Perseus. ○ Perseus killed Medusa, a Gorgon. The demon star refers to Medusa. ○ Gorgonia was named after the Gorgons.

Slide 88

Slide 88 text

Follow @chewxy on Twitter FAQs on Multiple Notation ● Didn’t we already go through this with the mess of UML and the like in the 90s? You want us to go back to writing specs like that? Yes. We lacked wisdom then. We are wiser now. Knowing when to use things like UML for problem solving and when to use it for specification is wisdom.

Slide 89

Slide 89 text

Follow @chewxy on Twitter Supplementary Material

Slide 90

Slide 90 text

Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change ✳ * *

Slide 91

Slide 91 text

Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Traditionally: ● Repeated S ⇒ + ● Repeated + ⇒ × ● Repeated × ⇒ *

Slide 92

Slide 92 text

Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Traditionally: ● Repeated S ⇒ + ● Repeated + ⇒ × ● Repeated × ⇒ * But: ab ≠ ba Compiler writers love the commutativity trick!

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Follow @chewxy on Twitter The Hierarchy of Operations in the Arithmetic Universe is about to Change Is there an operation ✳ that is a “next level”, and still maintains commutativity, associativity and distributivity? a ✳ b = aln b ✓ (a ✳ b) ✳ c = a ✳ (b ✳ c) ✓ a ✳ b = b ✳ a ✓ a ✳ (b × c) = a ✳ b × a ✳ c Identity: e Inverse Identity: e1/ln(a)