Going Structural with Named Tuples

Slide 1

Slide 1 text

Going Structural with Named Tuples Jamie Thompson @bishabosha (GitHub, Twitter/X) 1

Slide 2

Slide 2 text

● Engineer @ Mibex Software ● ex. Scala Center 2019-2024 Recent Scala OSS Contributions ● Mill Scala 3 upgrade ● Scala 3 build pipelining ● -Ytasty-reader Scala 2 ﬂag ● Mirror Framework bishabosha.github.io About Me 📖 2

Slide 3

Slide 3 text

Learning Goals 🎯 ● Why are Named Tuples being introduced? ● What is structural typing, and why is it useful? ● How to use Named Tuples? ● What are their limitations? (🚨 Nerd Snipe alert 🚨) 3

Slide 4

Slide 4 text

Problem space why Named Tuples? 4

Slide 5

Slide 5 text

Pattern Matching woes 5 def foo(m: MyUglyCaseClass) = m match case MyUglyCaseClass( _, _, _, _, ..., bar, _, _ ) => bar pattern_match.scala 😭

Slide 6

Slide 6 text

Pattern Matching woes 6 def foo(m: MyUglyCaseClass) = m match case MyUglyCaseClass( bar = bar, ) => bar pattern_match.scala There is a scala feature request for this since 2012! 🤔

Slide 7

Slide 7 text

class Seq[T]: def partition(f: T => Boolean): (Seq[T], Seq[T]) = ??? Multiple return values multiple_returns.scala 7 which side is which? 😭

Slide 8

Slide 8 text

class Seq[T]: def partition(f: T => Boolean): (matches: Seq[T], rest: Seq[T]) = ??? Multiple return values multiple_returns.scala 8 🤔

Slide 9

Slide 9 text

Address shortcomings of structural typing 9 type Person = Record { val name: String; val age: Int } def test(person: Person) = assert(person.name == "Jamie") structural.scala 🤔 “width” subtyping Unordered field access Since Scala 2.6.0

Slide 10

Slide 10 text

Address shortcomings of structural typing 10 class Record(data: Map[String, Any]) extends Selectable: def selectDynamic(name: String): Any = data(name) Record.scala val jamie = Record(Map( "name" -> "Jamie", "age" -> 28 )).asInstanceOf[Person] // casting!!! jamie.name // jamie.selectDynamic(“name”).asInstanceOf[...] 😱 example.scala Hard to “safely” inspect/construct without macros

Slide 11

Slide 11 text

Address shortcomings of structural typing 11 type FrontMatter = model.FrontMatter { val title: String val description: String val isIndex: Boolean val redirectFrom: List[String] val publishDate: String } Blog.scala Schema for YAML in markdown

Slide 12

Slide 12 text

Address shortcomings of structural typing 12 class FrontMatter(data: Map[String, List[String]]) extends Selectable: def selectDynamic(name: String): Any = name match case s"is$_" => data .get(name) .flatMap(ls => if ls.isEmpty then Some(true) else ls.head.toBooleanOption ) .getOrElse(false) case s"${_}s" => data.get(name).getOrElse(Nil) case _ => data.get(name).flatMap(_.headOption).getOrElse("") Blog.scala Either stick to convention, or pass type-derived schema in constructor. Again - you can’t inspect structural reﬁnement types without macros 🫣

Slide 13

Slide 13 text

Structural typing Why is it useful? 13

Slide 14

Slide 14 text

Why Structural Types? 14 ● Avoid naming fatigue, if only the shape matters ● Temporary, ad-hoc types ● narrow “view” over data (e.g. forget unused ﬁelds) ● compose arbitrary values while preserving types ● Derive new types from values

Slide 15

Slide 15 text

val people = spark.read.parquet("...") val department = spark.read.parquet("...") people.filter("age > 30") .join(department, people("deptId") === department("id")) .groupBy(department("name"), people("gender")) .agg(avg(people("salary")), max(people("age"))) DSLs spark.scala 15 Named Tuples could give better type safety without macros Example from https://spark.apache.org/docs

Slide 16

Slide 16 text

type Id[T] = T // City[Id] ==> return rows from DB class Expr[T] // City[Expr] ==> select columns in a query case class City[T[_]]( id: T[Int], name: T[String], countryCode: T[String], district: T[String], population: T[Long] ) ORM / SQL Query wrapper scalasql.scala 16 Can named tuples provide a different way? val fewLargestCities = db.run( City.select .sortBy(_.population).desc .drop(5).take(3) .map(c => (c.name, c.population)) ) query.scala Example from github.com/com-lihaoyi/scalasql “Quoted” DSL Diff backends

Slide 17

Slide 17 text

Named Tuple Overview 17 STABLE in Scala 3.7.0-RC1 SIP 58

Slide 18

Slide 18 text

What are Named Tuples? type Person = (name: String, age: Int) val Jamie = (name = "Jamie", age = 28) assert(Jamie.name == "Jamie") assert(Jamie.age == 28) Person.scala 18

Slide 19

Slide 19 text

What are Named Tuples? type Person = (name: String, age: Int) val Jamie = (name = "Jamie", age = 28) assert(Jamie.name == "Jamie") assert(Jamie.age == 28) Person.scala Type syntax 19

Slide 20

Slide 20 text

What are Named Tuples? type Person = (name: String, age: Int) val Jamie = (name = "Jamie", age = 28) assert(Jamie.name == "Jamie") assert(Jamie.age == 28) Person.scala Value syntax 20

Slide 21

Slide 21 text

What are Named Tuples? type Person = (name: String, age: Int) val Jamie = (name = "Jamie", age = 28) assert(Jamie.name == "Jamie") assert(Jamie.age == 28) Person.scala field selection 21

Slide 22

Slide 22 text

Type Inference def makeAccumulator() = var acc = 0 ( add = (x: Int) => acc += x, reset = () => acc = 0, get = () => acc ) val acc = makeAccumulator() // acc: ( // add : Int => Unit, // reset : () => Unit, // get : () => Int // ) accumulator.scala 22

Slide 23

Slide 23 text

def printPersonA(p: Person) = p match case (age = a, name = n) => println(s"$n is $a years old") def printPersonB(p: Person) = p match case (name = n) => println(s"$n is ${p.age} years old") Named Tuple patterns 🧩 patterns.scala 23 swap order forget fields

Slide 24

Slide 24 text

case class Point(x: Int, y: Int) def printX(p: Point) = p match case Point(x = x) => println(x) Named Tuple patterns 🧩 patterns.scala (case class) Coming to case classes too! 24

Slide 25

Slide 25 text

How do they work? 25

Slide 26

Slide 26 text

Comparisons To Product/Structural type Person = (name: String, age: Int) Person.scala case class Person(name: String, age: Int) type Person = Record {val name: String; val age: Int} 🤔 Q: What am I ? Product type? Structural type? 26

Slide 27

Slide 27 text

✅ type Person = Record {val name: String; val age: Int} ❌ case class Person(name: String, age: Int) A: Product type Comparisons To Product/Structural type Person = (name: String, age: Int) Person.scala 27 But not “scala.Product”

Slide 28

Slide 28 text

type HasName = (name: String) type HasAge = (age: Int) def person: HasName & HasAge = ??? person.name person.age Not A Structural type Person.scala ❌ Can’t create intersection of fields Same underlying field Overlapping field 28 uninhabited

Slide 29

Slide 29 text

Type Syntax Sugar type Person = (name: String, age: Int) type Person = NamedTuple[("name", "age"), (String, Int)] ⬇ 29

Slide 30

Slide 30 text

type Person = NamedTuple[("name", "age"), (String, Int)] type Person = (name: String, age: Int) ⬇ 30 Only at compile-time Type Syntax Sugar

Slide 31

Slide 31 text

type Person = NamedTuple[("name", "age"), (String, Int)] ⬇ 31 Underlying type type Person = (name: String, age: Int) Type Syntax Sugar

Slide 32

Slide 32 text

NamedTuple[("name" *: "age" *: EmptyTuple), (String *: Int *: EmptyTuple)] ⬇ 32 Field labels are first class types No macro required! type Person = (name: String, age: Int) type Person = NamedTuple[("name", "age"), (String, Int)] ⬇ Type Syntax Sugar

Slide 33

Slide 33 text

opaque type AnyNamedTuple opaque type NamedTuple[N <: Tuple, +T <: Tuple] <: AnyNamedTuple = T NamedTuple.scala 33 Type Syntax Sugar

Slide 34

Slide 34 text

Conversion/Operations 34

Slide 35

Slide 35 text

Named Tuples are Product types type Person = (name: String, age: Int) val p: Person = ("Alice", 42).withNames[("name", "age")] assert(p(1) == p.age) summon[Mirror.Of[Person]].fromProduct(p.toTuple) Person.scala ✚ O(1) Random-access fields ✚ Type class derivation support ✚ Zero-cost conversion to/from tuple 35

Slide 36

Slide 36 text

val nameT = (name = "Alice") val ageT = (age = 42) val person: Person = nameT ++ ageT person(0) == person.name person(1) == person.age Like a Generic Tuple Person.scala Generic Operations! Tuple concatenation 36

Slide 37

Slide 37 text

val nameT = (name = "Alice") val ageT = (age = 42) val person: Person = nameT ++ ageT val err = ageT ++ ageT Like a Generic Tuple Person.scala Generic Operations! Error: can’t prove disjoint ❌ 37

Slide 38

Slide 38 text

Like a Generic Tuple type Person = (name: String, age: Int) val optPerson: NamedTuple.Map[Person, Option] = (name = Some("Alice"), age = None) PersonMapped.scala 38 Generic Operations! Each field is wrapped with Option[T]

Slide 39

Slide 39 text

case class City(name: String, population: Int) val Warsaw: NamedTuple.From[City] = (name = "Warsaw", population = 1_800_000) Like a Generic Tuple City.scala Generic Operations! Provide schema from a case class 39

Slide 40

Slide 40 text

Use Cases Out of the box 40

Slide 41

Slide 41 text

class Seq[T]: def partition(f: T => Boolean): (matches: Seq[T], rest: Seq[T]) = ??? Multiple return values multiple_returns.scala 41 self-documenting code

Slide 42

Slide 42 text

val directions = List( (dx = 1, dy = 0), (dx = 0, dy = 1), (dx = -1, dy = 0), (dx = 0, dy = -1) ) Test input data Data 42 Ad-hoc data

Slide 43

Slide 43 text

Batch jobs val config = ( currency = (code = "USD", symbol = "$"), invoice = ( id = 5, period = (start = (2025, 1, 27), days = 30) ), listings = ( items = List( (desc = "Premium Insurance (1 year)", qty = 2, price = 250_00), (desc = "Samsung Galaxy S13", qty = 1, price = 999_00) ), taxRate = 12 ), business = ( name = "Acme Corp.", ... make_invoice.sc 43 Ad-hoc data

Slide 44

Slide 44 text

Custom types with Structural selection class Foo extends scala.Selectable: type Fields <: AnyNamedTuple // concrete named tuple type def selectDynamic(name: String): Any = ??? QueryDSL.scala 44 val f: Foo { type Fields = (x: Int, y: Int) } = ??? f.x // f.selectDynamic("x").asInstanceOf[Int] Example.scala Structural field

Slide 45

Slide 45 text

Use Cases Library enhanced 45 github.com/bishabosha/scalar-2025 Demos available!

Slide 46

Slide 46 text

val r = sttp.client4.quick.quickRequest .post(uri"http://localhost:11434/api/chat") .body( upickle.default.write( ( model = "gemma3:4b", messages = Seq( ( role = "user", content = "write me a haiku about Scala" ) ), stream = false, ) ) ) .send() val msg = upickle.default.read[(message: (content: String))](r.body) println(msg.message.content) Read/Write JSON Data Generate ad-hoc JSON codecs 46 Using softwaremill/sttp and com-lihaoyi/upickle github.com/bishabosha/scalar-2025 Ollama chat api

Slide 47

Slide 47 text

Type Conversions case class UserV1(name: String) case class UserV2(name: String, age: Option[Int]) def convert(u1: UserV1): UserV2 = u1.asNamedTuple .withField((age = None)) .as[UserV2] fieldMapper.scala 47 github.com/bishabosha/scalar-2025 Convert via scala.deriving.Mirror Concat with ++ Could be made lazy?

Slide 48

Slide 48 text

DataFrame val text = "The quick brown fox jumps over the lazy dog" val toLower = (_: String).toLowerCase val stats = DataFrame .column((words = text.split("\\s+"))) .withComputed( (lowerCase = fun(toLower)(col.words)) ) .groupBy(col.lowerCase) .agg( group.key ++ (freq = group.size) ) .sort(col.freq, descending = true) println(stats.show(Int.MaxValue)) dataframe.scala shape: (8, 2) ┌───────────┬──────┐ │ lowerCase ┆ freq │ ╞═══════════╪══════╡ │ the ┆ 2 │ │ over ┆ 1 │ │ quick ┆ 1 │ │ lazy ┆ 1 │ │ jumps ┆ 1 │ │ brown ┆ 1 │ │ dog ┆ 1 │ │ fox ┆ 1 │ └───────────┴──────┘ 48 github.com/bishabosha/scalar-2025 Structural field

Slide 49

Slide 49 text

case class City( id: Int, name: String, countryCode: String, district: String, population: Long ) object City extends Table[City] Data query Domain.scala val allCities: Seq[City] = db.run(City.select) // Adding up population of all cities in Poland val citiesPop: Long = db.run: City.select .filter(c => c.countryCode === "POL") .map(c => c.population) .sum Query.scala Example adapted for Named Tuples from github.com/com-lihaoyi/scalasql 49 dropped ScalaSql’s required type param github.com/bishabosha/scalar-2025 Structural field

Slide 50

Slide 50 text

Custom types with Structural selection QueryDSL.scala case class City(name: String, pop: Int) class Expr[Schema] extends scala.Selectable { type Fields = ... } val c: Expr[City] = ??? // City.select.filter(c => c.name === ...) // ^ c.Fields =:= (name: Expr[String], pop: Expr[Int]) val name: Expr[String] = c.name // c.selectDynamic(“name”) val pop: Expr[Int] = c.pop // c.selectDynamic(“pop”) 50

Slide 51

Slide 51 text

Use Cases Full-stack app 51

Slide 52

Slide 52 text

Endpoint Derivation val schema = HttpService.endpoints[NoteService] // schema.Fields =:= ( // createNote: Endpoint[...], // getAllNotes: Endpoint[...] // deleteNote: Endpoint[...], // ) client.scala object model: type Note = (id: String, title: String, content: String) type CreateNote = (title: String, content: String) trait NoteService derives HttpService: @post("/api/notes") def createNote(@body body: CreateNote): Note @get("/api/notes") def getAllNotes(): Seq[Note] @delete("/api/notes/{id}") def deleteNote(@path id: String): Unit NoteService.scala 52 github.com/bishabosha/scalar-2025 Using bishabosha/ops-mirror

Slide 53

Slide 53 text

HTTP Client: Frontend val schema = HttpService.endpoints[NoteService] val client = Client.ofEndpoints( schema, baseURI = "http://localhost:8080/" ) // client.Fields =:= ( // createNote: PartialRequest[...], // getAllNotes: PartialRequest[...], // deleteNote: PartialRequest[...], // ) client.scala object model: type Note = (id: String, title: String, content: String) type CreateNote = (title: String, content: String) trait NoteService derives HttpService: @post("/api/notes") def createNote(@body body: CreateNote): Note @get("/api/notes") def getAllNotes(): Seq[Note] @delete("/api/notes/{id}") def deleteNote(@path id: String): Unit NoteService.scala 53 github.com/bishabosha/scalar-2025

Slide 54

Slide 54 text

HTTP Client: Frontend val schema = HttpService.endpoints[NoteService] val client = Client.ofEndpoints( schema, baseURI = "http://localhost:8080/" ) client.createNote.send( (body = (title = ..., content = ...)) ) // : Future[Note] client.getAllNotes.send(Empty) // ... client.deleteNote.send((id = ...)) // client.scala object model: type Note = (id: String, title: String, content: String) type CreateNote = (title: String, content: String) trait NoteService derives HttpService: @post("/api/notes") def createNote(@body body: CreateNote): Note @get("/api/notes") def getAllNotes(): Seq[Note] @delete("/api/notes/{id}") def deleteNote(@path id: String): Unit NoteService.scala 54 github.com/bishabosha/scalar-2025

Slide 55

Slide 55 text

HTTP Server object model: type Note = (id: String, title: String, content: String) type CreateNote = (title: String, content: String) trait NoteService derives HttpService: @post("/api/notes") def createNote(@body body: CreateNote): Note @get("/api/notes") def getAllNotes(): Seq[Note] @delete("/api/notes/{id}") def deleteNote(@path id: String): Unit NoteService.scala val schema = HttpService.endpoints[NoteService] val app = router(schema) def routes(db: DB): app.Routes = ( createNote = p => db.run(...), getAllNotes = _ => db.run(...), deleteNote = p => db.run(...) ) val server = app .handle(routes(LogBasedStore())) .listen(port = 8080) server.scala 55 github.com/bishabosha/scalar-2025

Slide 56

Slide 56 text

HTTP Server object model: type Note = (id: String, title: String, content: String) type CreateNote = (title: String, content: String) trait NoteService derives HttpService: @post("/api/notes") def createNote(@body body: CreateNote): Note @get("/api/notes") def getAllNotes(): Seq[Note] @delete("/api/notes/{id}") def deleteNote(@path id: String): Unit NoteService.scala def routes(db: DB): app.Routes = ( createNote = p => db.run( // p: (body: CreateNote) Note.insert.values(p.body) ), getAllNotes = _ => db.run( // _: NamedTuple.Empty Note.select ), deleteNote = p => db.run( // p: (id: String) Note.delete.filter(_.id == p.id) ) ) server.scala 56 github.com/bishabosha/scalar-2025 Backend agnostic

Slide 57

Slide 57 text

Demo 57 github.com/bishabosha/scalar-2025

Slide 58

Slide 58 text

Type computation case class City(name: String, pop: Int) val c: Expr[City] = ??? // ^ c.Fields =:= (name: Expr[String], pop: Expr[Int]) ● Match types ● Mirror Framework ● scala.compiletime.ops package ● Inline match ● Macros 58 github.com/bishabosha/scalar-2025 Inspect the demos

Slide 59

Slide 59 text

Limitations 59

Slide 60

Slide 60 text

No Recursion recursion.scala val rec = ( x = 23 y = x ) 60 Outer scope recursion.scala val rec = new Selectable { val x = 23 val y = x } Inferred structural type ✅ ❌

Slide 61

Slide 61 text

Solution? recursion.scala val rec = rec(self => ( x = 23 y = self.x ) ) 61

Slide 62

Slide 62 text

No Explicit types types.scala val t = ( x: Int = 23 y: String = "y" ) 62

Slide 63

Slide 63 text

Limited support for match types 63 Still ok if you guarantee it will be a NT type

Slide 64

Slide 64 text

Computing Named Tuple Types 64

Slide 65

Slide 65 text

Match Types builtins.scala Pattern match on types! NamedTuple.Map[T, [A] =>> Option[A]] // apply type lambda to each elem NamedTuple.Concat[T, U] // join two Named Tuple types NamedTuple.From[C] // convert case class to Named Tuple NamedTuple.Names[T] // extract the names tuple NamedTuple.DropNames[T] // extract the values tuple // ...more in NamedTuple object 65

Slide 66

Slide 66 text

Match Types custom.scala Pattern match on types! type Select[N <: String, T <: AnyNamedTuple] <: Option[Any] = T match case NamedTuple[ns, vs] => (ns, vs) match case (N *: _, v *: _) => Some[v] case (_ *: ns, _ *: vs) => Select[N, NamedTuple[ns, vs]] case (EmptyTuple, EmptyTuple) => None.type summon[Select["name", (name: String, age: Int)] =:= Some[String]] summon[Select["age", (name: String, age: Int)] =:= Some[Int]] summon[Select["???", (name: String, age: Int)] =:= None.type] 66