Nix for Scala folks - Speaker Deck

Slide 1

Slide 1 text

Nix for Scala folks Jakub Kozłowski, Disney Streaming | 11.05.2022 | Warsaw

Slide 2

Slide 2 text

Agenda Two words about Nix Nix in the Scala world Nix+Scala in the future

Slide 3

Slide 3 text

Agenda Two words about Nix Nix in the Scala world Nix+Scala in the future

Slide 4

Slide 4 text

What is Nix?

Slide 5

Slide 5 text

What is Nix? • Package manager

Slide 6

Slide 6 text

What is Nix? • Package manager • Build system

Slide 7

Slide 7 text

What is Nix? • Package manager • Build system • Language

Slide 8

Slide 8 text

What is Nix? • Package manager • Build system • Language • Linux distro (NixOS)

Slide 9

Slide 9 text

Nix is a scalable tool for building and managing packages in a reproducible fashion. tl;dr

Slide 10

Slide 10 text

The main repository with Nix package de f initions github.com/nixos/nixpkgs Nixpkgs

Slide 11

Slide 11 text

Agenda Two words about Nix Nix in the Scala world Nix+Scala in the future

Slide 12

Slide 12 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt)

Slide 13

Slide 13 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) What needs to be compiled

Slide 14

Slide 14 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) git, node, awscli, docker-client, ...

Slide 15

Slide 15 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) JAVA_OPTS, SBT_OPTS, ...

Slide 16

Slide 16 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) build.sbt, project/*

Slide 17

Slide 17 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) *.graphql, *.proto, *.smithy

Slide 18

Slide 18 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) runs sbt, compiler, tests...

Slide 19

Slide 19 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) scala-compiler-2.13.6.jar

Slide 20

Slide 20 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) cats-e ff ect-3.3.0.jar fs2-core-3.2.0.jar

Slide 21

Slide 21 text

But f irst... typical sbt build inputs (let's be honest, everyone uses sbt) bin/sbt, sbt-1.6.2.jar

Slide 22

Slide 22 text

Environment setup

Slide 23

Slide 23 text

Environment setup with Nix let pkgs = import (builtins.fetchTarball { url = "https: / / github.com/NixOS/nixpkgs/archive/20eabe33864104ad9928450e5927ab7835f8f9e8.tar.gz"; sha256 = "0l00xjjd436ddq1llfvpcipz06b6s0zxqrmfm9mgj7c0ig4y4c0r"; }) { }; in pkgs.mkShell { packages = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }

Slide 24

Slide 24 text

Environment setup with Nix pkgs.mkShell { packages = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }

Slide 25

Slide 25 text

Environment setup with Nix

Slide 26

Slide 26 text

Why bother? • Doesn't pollute your environment 🌳

Slide 27

Slide 27 text

Why bother? • Doesn't pollute your environment 🌳 • Doesn't con f lict with globally installed programs

Slide 28

Slide 28 text

Why bother? • Doesn't pollute your environment 🌳 • Doesn't con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/...

Slide 29

Slide 29 text

Why bother? • Doesn't pollute your environment 🌳 • Doesn't con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/... • Share tools with the team (Git)

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Environment variables in Nix shell pkgs.mkShell { packages = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; JAVA_OPTS = "-Xmx4G"; }

Slide 32

Slide 32 text

Temporary shells

Slide 33

Slide 33 text

Bonus: nix-direnv

Slide 34

Slide 34 text

Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: }

Slide 35

Slide 35 text

Bonus: f lakes flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in );

Slide 36

Slide 36 text

Bonus: f lakes { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; }

Slide 37

Slide 37 text

Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }

Slide 38

Slide 38 text

Bonus: f lakes • Uses lock f ile to specify hash of Nixpkgs and other repositories { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }

Slide 39

Slide 39 text

Bonus: f lakes • Uses lock f ile to specify hash of Nixpkgs and other repositories • Less boilerplate for pinning { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }

Slide 40

Slide 40 text

Bonus: f lakes • Uses lock f ile to specify hash of Nixpkgs and other repositories • Less boilerplate for pinning • Still experimental, e ff orts to further simplify { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }

Slide 41

Slide 41 text

See also: devshell • TOML-based • Nix language not needed for 80% use cases • Experimental https://github.com/numtide/devshell [devshell] packages = [ "go" ]

Slide 42

Slide 42 text

Installing packages

Slide 43

Slide 43 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER

Slide 44

Slide 44 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER • Atomic installation, rollbacks

Slide 45

Slide 45 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER • Atomic installation, rollbacks • No transitive con f licts

Slide 46

Slide 46 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER • Atomic installation, rollbacks • No transitive con f licts • User-scoped

Slide 47

Slide 47 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER • Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager

Slide 48

Slide 48 text

nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER • Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager • Also experimental, see nix-env

Slide 49

Slide 49 text

Recommendations

Slide 50

Slide 50 text

Recommendations Before things get out of hand

Slide 51

Slide 51 text

Recommendations • Use nix shells in your projects!

Slide 52

Slide 52 text

Recommendations • Use nix shells in your projects! • Use one-time shells to try stu ff out

Slide 53

Slide 53 text

Recommendations • Use nix shells in your projects! • Use one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰

Slide 54

Slide 54 text

Recommendations • Use nix shells in your projects! • Use one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager

Slide 55

Slide 55 text

Recommendations • Use nix shells in your projects! • Use one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager • Use shells in CI

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

More use cases for Nix • System/user program management • Declarative system con f iguration • CI pipelines • Deployment • Building small docker images • ??? • ???

Slide 58

Slide 58 text

Build system

Slide 59

Slide 59 text

Nix as a build system def makePackage( dependencies: List[Package], src: Path, buildScript: String, ): Package = ??? (simpli f ied)

Slide 60

Slide 60 text

Nix as a build system def makePackage( dependencies: List[Package], src: Path, buildScript: String, ): Package = ??? (simpli f ied) • No access to internet*, other f iles on disk • Build script has to output f iles to a speci f ic location

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Typical sbt build inputs (again)

Slide 63

Slide 63 text

State of the art

Slide 64

Slide 64 text

State of the art • Single dependency hash

Slide 65

Slide 65 text

State of the art • Single dependency hash • Waste of space (duplicated deps)

Slide 66

Slide 66 text

State of the art • Single dependency hash • Waste of space (duplicated deps) • Slow, non-incremental

Slide 67

Slide 67 text

Agenda Two words about Nix Nix in the Scala world Nix+Scala in the future

Slide 68

Slide 68 text

Inspiration

Slide 69

Slide 69 text

Inspiration: Rust Nix-friendly lock f ile based on Cargo.lock

Slide 70

Slide 70 text

Inspiration: Rust Dependencies are built in isolated, parallel Nix builds

Slide 71

Slide 71 text

Hope

Slide 72

Slide 72 text

Nix+Scala in a perfect world

Slide 73

Slide 73 text

Nix+Scala in a perfect world • Lock f ile generated from sbt with a single line

Slide 74

Slide 74 text

Nix+Scala in a perfect world • Lock f ile generated from sbt with a single line • Fast setup, using existing caches

Slide 75

Slide 75 text

Nix+Scala in a perfect world • Lock f ile generated from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse

Slide 76

Slide 76 text

Nix+Scala in a perfect world • Lock f ile generated from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules?

Slide 77

Slide 77 text

Nix+Scala in a perfect world • Lock f ile generated from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop?

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

Nix+Scala work today

Slide 81

Slide 81 text

Nix+Scala work today • sbt-derivation (not actively developed)

Slide 82

Slide 82 text

Nix+Scala work today • sbt-derivation (not actively developed) • Open ticket in sbt-derivation to support lock f iles

Slide 83

Slide 83 text

Nix+Scala work today • sbt-derivation (not actively developed) • Open ticket in sbt-derivation to support lock f iles • Possibly needed: new feature in sbt

Slide 84

Slide 84 text

How to get started? sh <(curl -L https: // nixos.org/nix/install) -- daemon • https://nixos.org/guides/nix-pills + https://nixos.org/manual/nix/stable/ • https://nix.dev • I'm writing a book! Follow https://leanpub.com/nix-book The only time you'll ever have to sh from curl :)

Slide 85

Slide 85 text

Thank you! 📰 blog.kubukoz.com 🐦 @kubukoz Slides: speakerdeck.com/kubukoz YouTube: yt.kubukoz.com