$30 off During Our Annual Pro Sale. View Details »

Nix for Scala folks

Nix for Scala folks

Quick intro to Nix and optimal usage with the Scala ecosystem in 2022.

Jakub Kozłowski

May 11, 2022
Tweet

More Decks by Jakub Kozłowski

Other Decks in Programming

Transcript

  1. Nix for Scala folks Jakub Kozłowski, Disney Streaming | 11.05.2022

    | Warsaw
  2. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  3. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  4. What is Nix?

  5. What is Nix? • Package manager

  6. What is Nix? • Package manager • Build system

  7. What is Nix? • Package manager • Build system •

    Language
  8. What is Nix? • Package manager • Build system •

    Language • Linux distro (NixOS)
  9. Nix is a scalable tool for building and managing packages

    in a reproducible fashion. tl;dr
  10. The main repository with Nix package de f initions github.com/nixos/nixpkgs

    Nixpkgs
  11. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  12. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt)
  13. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) What needs to be compiled
  14. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) git, node, awscli, docker-client, ...
  15. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) JAVA_OPTS, SBT_OPTS, ...
  16. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) build.sbt, project/*
  17. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) *.graphql, *.proto, *.smithy
  18. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) runs sbt, compiler, tests...
  19. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) scala-compiler-2.13.6.jar
  20. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) cats-e ff ect-3.3.0.jar fs2-core-3.2.0.jar
  21. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) bin/sbt, sbt-1.6.2.jar
  22. Environment setup

  23. Environment setup with Nix let pkgs = import (builtins.fetchTarball {

    url = "https: / / github.com/NixOS/nixpkgs/archive/20eabe33864104ad9928450e5927ab7835f8f9e8.tar.gz"; sha256 = "0l00xjjd436ddq1llfvpcipz06b6s0zxqrmfm9mgj7c0ig4y4c0r"; }) { }; in pkgs.mkShell { packages = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }
  24. Environment setup with Nix pkgs.mkShell { packages = [ pkgs.sbt

    pkgs.coursier pkgs.nodejs ]; }
  25. Environment setup with Nix

  26. Why bother? • Doesn't pollute your environment 🌳

  27. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs
  28. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/...
  29. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/... • Share tools with the team (Git)
  30. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/... • Share tools with the team (Git) • Same setup locally and in Continuous Integration
  31. Environment variables in Nix shell pkgs.mkShell { packages = [

    pkgs.sbt pkgs.coursier pkgs.nodejs ]; JAVA_OPTS = "-Xmx4G"; }
  32. Temporary shells

  33. Bonus: nix-direnv

  34. Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils";

    outputs = { self, nixpkgs, flake-utils, . .. }: }
  35. Bonus: f lakes flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs

    { inherit system; }; in );
  36. Bonus: f lakes { devShells.default = pkgs.mkShell { buildInputs =

    [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; }
  37. Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils";

    outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  38. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  39. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories • Less boilerplate for pinning { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  40. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories • Less boilerplate for pinning • Still experimental, e ff orts to further simplify { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  41. See also: devshell • TOML-based • Nix language not needed

    for 80% use cases • Experimental https://github.com/numtide/devshell [devshell] packages = [ "go" ]
  42. Installing packages

  43. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER

  44. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks
  45. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts
  46. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped
  47. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager
  48. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager • Also experimental, see nix-env
  49. Recommendations

  50. Recommendations Before things get out of hand

  51. Recommendations • Use nix shells in your projects!

  52. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out
  53. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰
  54. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager
  55. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager • Use shells in CI
  56. None
  57. More use cases for Nix • System/user program management •

    Declarative system con f iguration • CI pipelines • Deployment • Building small docker images • ??? • ???
  58. Build system

  59. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied)
  60. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied) • No access to internet*, other f iles on disk • Build script has to output f iles to a speci f ic location
  61. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied) • No access to internet*, other f iles on disk • Build script has to output f iles to a speci f ic location * - Internet can be used if you know the output's hash beforehand
  62. Typical sbt build inputs (again)

  63. State of the art

  64. State of the art • Single dependency hash

  65. State of the art • Single dependency hash • Waste

    of space (duplicated deps)
  66. State of the art • Single dependency hash • Waste

    of space (duplicated deps) • Slow, non-incremental
  67. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  68. Inspiration

  69. Inspiration: Rust Nix-friendly lock f ile based on Cargo.lock

  70. Inspiration: Rust Dependencies are built in isolated, parallel Nix builds

  71. Hope

  72. Nix+Scala in a perfect world

  73. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line
  74. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches
  75. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse
  76. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules?
  77. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop?
  78. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop? • Replacing sbt's dependency resolution?
  79. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop? • Replacing sbt's dependency resolution? • New build tool?
  80. Nix+Scala work today

  81. Nix+Scala work today • sbt-derivation (not actively developed)

  82. Nix+Scala work today • sbt-derivation (not actively developed) • Open

    ticket in sbt-derivation to support lock f iles
  83. Nix+Scala work today • sbt-derivation (not actively developed) • Open

    ticket in sbt-derivation to support lock f iles • Possibly needed: new feature in sbt
  84. How to get started? sh <(curl -L https: // nixos.org/nix/install)

    -- daemon • https://nixos.org/guides/nix-pills + https://nixos.org/manual/nix/stable/ • https://nix.dev • I'm writing a book! Follow https://leanpub.com/nix-book The only time you'll ever have to sh from curl :)
  85. Thank you! 📰 blog.kubukoz.com 🐦 @kubukoz Slides: speakerdeck.com/kubukoz YouTube: yt.kubukoz.com