Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Nix for Scala folks

Nix for Scala folks

Quick intro to Nix and optimal usage with the Scala ecosystem in 2022.

08f642741fba006656cb86fb61c160b3?s=128

Jakub Kozłowski

May 11, 2022
Tweet

More Decks by Jakub Kozłowski

Other Decks in Programming

Transcript

  1. Nix for Scala folks Jakub Kozłowski, Disney Streaming | 11.05.2022

    | Warsaw
  2. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  3. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  4. What is Nix?

  5. What is Nix? • Package manager

  6. What is Nix? • Package manager • Build system

  7. What is Nix? • Package manager • Build system •

    Language
  8. What is Nix? • Package manager • Build system •

    Language • Linux distro (NixOS)
  9. Nix is a scalable tool for building and managing packages

    in a reproducible fashion. tl;dr
  10. The main repository with Nix package de f initions github.com/nixos/nixpkgs

    Nixpkgs
  11. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  12. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt)
  13. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) What needs to be compiled
  14. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) git, node, awscli, docker-client, ...
  15. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) JAVA_OPTS, SBT_OPTS, ...
  16. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) build.sbt, project/*
  17. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) *.graphql, *.proto, *.smithy
  18. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) runs sbt, compiler, tests...
  19. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) scala-compiler-2.13.6.jar
  20. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) cats-e ff ect-3.3.0.jar fs2-core-3.2.0.jar
  21. But f irst... typical sbt build inputs (let's be honest,

    everyone uses sbt) bin/sbt, sbt-1.6.2.jar
  22. Environment setup

  23. Environment setup with Nix let pkgs = import (builtins.fetchTarball {

    url = "https: / / github.com/NixOS/nixpkgs/archive/20eabe33864104ad9928450e5927ab7835f8f9e8.tar.gz"; sha256 = "0l00xjjd436ddq1llfvpcipz06b6s0zxqrmfm9mgj7c0ig4y4c0r"; }) { }; in pkgs.mkShell { packages = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }
  24. Environment setup with Nix pkgs.mkShell { packages = [ pkgs.sbt

    pkgs.coursier pkgs.nodejs ]; }
  25. Environment setup with Nix

  26. Why bother? • Doesn't pollute your environment 🌳

  27. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs
  28. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/...
  29. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/... • Share tools with the team (Git)
  30. Why bother? • Doesn't pollute your environment 🌳 • Doesn't

    con f lict with globally installed programs • Project-speci f ic version of the JVM/terraform/awscli/node/... • Share tools with the team (Git) • Same setup locally and in Continuous Integration
  31. Environment variables in Nix shell pkgs.mkShell { packages = [

    pkgs.sbt pkgs.coursier pkgs.nodejs ]; JAVA_OPTS = "-Xmx4G"; }
  32. Temporary shells

  33. Bonus: nix-direnv

  34. Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils";

    outputs = { self, nixpkgs, flake-utils, . .. }: }
  35. Bonus: f lakes flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs

    { inherit system; }; in );
  36. Bonus: f lakes { devShells.default = pkgs.mkShell { buildInputs =

    [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; }
  37. Bonus: f lakes { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils";

    outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  38. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  39. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories • Less boilerplate for pinning { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  40. Bonus: f lakes • Uses lock f ile to specify

    hash of Nixpkgs and other repositories • Less boilerplate for pinning • Still experimental, e ff orts to further simplify { inputs.nixpkgs.url = "github:nixos/nixpkgs"; inputs.flake-utils.url = "github:numtide/flake-utils"; outputs = { self, nixpkgs, flake-utils, . .. }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; in { devShells.default = pkgs.mkShell { buildInputs = [ pkgs.sbt pkgs.coursier pkgs.nodejs ]; }; } ); }
  41. See also: devshell • TOML-based • Nix language not needed

    for 80% use cases • Experimental https://github.com/numtide/devshell [devshell] packages = [ "go" ]
  42. Installing packages

  43. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER

  44. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks
  45. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts
  46. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped
  47. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager
  48. nix pro f ile • Drop-in replacement for $PACKAGE_MANAGER •

    Atomic installation, rollbacks • No transitive con f licts • User-scoped • For a declarative approach, see home-manager • Also experimental, see nix-env
  49. Recommendations

  50. Recommendations Before things get out of hand

  51. Recommendations • Use nix shells in your projects!

  52. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out
  53. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰
  54. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager
  55. Recommendations • Use nix shells in your projects! • Use

    one-time shells to try stu ff out • Flakes + shells + nix-direnv = 🥰 • Use nix as a normal package manager • Use shells in CI
  56. None
  57. More use cases for Nix • System/user program management •

    Declarative system con f iguration • CI pipelines • Deployment • Building small docker images • ??? • ???
  58. Build system

  59. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied)
  60. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied) • No access to internet*, other f iles on disk • Build script has to output f iles to a speci f ic location
  61. Nix as a build system def makePackage( dependencies: List[Package], src:

    Path, buildScript: String, ): Package = ??? (simpli f ied) • No access to internet*, other f iles on disk • Build script has to output f iles to a speci f ic location * - Internet can be used if you know the output's hash beforehand
  62. Typical sbt build inputs (again)

  63. State of the art

  64. State of the art • Single dependency hash

  65. State of the art • Single dependency hash • Waste

    of space (duplicated deps)
  66. State of the art • Single dependency hash • Waste

    of space (duplicated deps) • Slow, non-incremental
  67. Agenda Two words about Nix Nix in the Scala world

    Nix+Scala in the future
  68. Inspiration

  69. Inspiration: Rust Nix-friendly lock f ile based on Cargo.lock

  70. Inspiration: Rust Dependencies are built in isolated, parallel Nix builds

  71. Hope

  72. Nix+Scala in a perfect world

  73. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line
  74. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches
  75. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse
  76. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules?
  77. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop?
  78. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop? • Replacing sbt's dependency resolution?
  79. Nix+Scala in a perfect world • Lock f ile generated

    from sbt with a single line • Fast setup, using existing caches • Saving space with dependency reuse • Splitting sbt build into modules? • Direct integration with scalac/bloop? • Replacing sbt's dependency resolution? • New build tool?
  80. Nix+Scala work today

  81. Nix+Scala work today • sbt-derivation (not actively developed)

  82. Nix+Scala work today • sbt-derivation (not actively developed) • Open

    ticket in sbt-derivation to support lock f iles
  83. Nix+Scala work today • sbt-derivation (not actively developed) • Open

    ticket in sbt-derivation to support lock f iles • Possibly needed: new feature in sbt
  84. How to get started? sh <(curl -L https: // nixos.org/nix/install)

    -- daemon • https://nixos.org/guides/nix-pills + https://nixos.org/manual/nix/stable/ • https://nix.dev • I'm writing a book! Follow https://leanpub.com/nix-book The only time you'll ever have to sh from curl :)
  85. Thank you! 📰 blog.kubukoz.com 🐦 @kubukoz Slides: speakerdeck.com/kubukoz YouTube: yt.kubukoz.com