Tweet
Tweet

Slide 1

Slide 1 text

! # @zetavg
 fb.me/pokaichang72 Functional, Declarative and Modular System Environment 
 with Nix and NixOS

Slide 2

Slide 2 text

Hi !

Slide 3

Slide 3 text

! # @zetavg
 fb.me/pokaichang72
 皰玡㳓 Pokai Chang

Slide 4

Slide 4 text

! # @zetavg
 fb.me/pokaichang72
 speakerdeck.com/zetavg 皰玡㳓 Pokai Chang

Slide 5

Slide 5 text

Functional Programming

Slide 6

Slide 6 text

礍ᒊ

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

/ๅ碝懿袅.txt

Slide 10

Slide 10 text

/ๅ碝懿袅.txt

Slide 11

Slide 11 text

VPS

Slide 12

Slide 12 text

apt-get install blablabla vi /etc/nginx/conf.d/default.conf patch and recompile nginx 独 独

Slide 13

Slide 13 text

/ๅ碝懿袅.txt ݈憎ᶎЎ聲๏݋

Slide 14

Slide 14 text

Software Development

Slide 15

Slide 15 text

Develop Environment ▪ Your project still have some system dependencies. ▪ How to write instructions for setting up the develop environment? ▪ Write automated scripts instead of instructions. ▪ The script can act as documentation.
 (A kind of doc that you can get benefit from without reading it.) ▪ Forgot to check/install something...
 (Especially when you added or updated some dependencies) ▪ Script broken on a different OS distribution or package manager. ▪ Spend three days to onboard a new member with a working development environment.

Slide 16

Slide 16 text

Your Desktop/NB

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

Hidden Dependencies e.g. 
 Key Mapping Tool ↓ Window Manager ↓
 App Scripting

Slide 20

Slide 20 text

Patches Everywhere

Slide 21

Slide 21 text

???

Slide 22

Slide 22 text

Software Deployment Problems

Slide 23

Slide 23 text

Software Deployment Problems

Slide 24

Slide 24 text

Global Library Dependencies Libraries LibA Applications App1 App2 App3 LibB

Slide 25

Slide 25 text

Libraries Applications LibA App2 Global Library Dependencies Upgrade of App2 App1 App2' App3 LibB LibA' $

Slide 26

Slide 26 text

Global Library Dependencies Libraries LibA Applications App1 App2 App3 LibB

Slide 27

Slide 27 text

Global Library Dependencies Removal of App3 Libraries LibA Applications App1 App2 App3 LibB $

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

#PurelyFunctional #Immutable 
 #Declarative #Lazy
 #GarbageCollection
 #PersistedCache

Slide 30

Slide 30 text

Nix Store Nix Language / Derivation NixOS / nix-env /nix-shell

Slide 31

Slide 31 text

Nix Store Nix Language / Derivation NixOS / nix-env /nix-shell { च๜禊盢
 膏戔懯߽䋊 䋿褬ֵአ {

Slide 32

Slide 32 text

Nix Store

Slide 33

Slide 33 text

Nix Store Nix Store glibc-2.27 bash-4.4 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 ghc-8.4.4 nodejs-8.15.1 ruby-2.5.4 python-2.7.15

Slide 34

Slide 34 text

Nix Store Packages should not depend on global system stuff /usr/lib libssl.so App ruby-2.5.4 Global Variables Are Evil!

Slide 35

Slide 35 text

Nix Store Packages can depend on other packages in the Nix Store /usr/lib libssl.so App ruby-2.5.4 Nix Store ruby-2.5.4 openssl-1.0.2

Slide 36

Slide 36 text

Nix Store Example: bash bash-interactive-4.4 glibc-2.27 bash-4.4 readline-7.0 ncurses-6.1

Slide 37

Slide 37 text

Nix Store Example: wget wget-1.19.5 glibc-2.27 zlib-1.2.11 openssl-1.0.2 util-linux-2.32.1 libunistring-0.9.10 pcre-8.42 libidn2-2.0.5

Slide 38

Slide 38 text

Generated using: nix-store -q --graph $(which bash) Nix Store Real Dependency Graph Generated by Nix: bash

Slide 39

Slide 39 text

Generated using: nix-store -q --graph $(which gcc) Nix Store Real Dependency Graph Generated by Nix: gcc

Slide 40

Slide 40 text

Generated using: nix-store -q --graph $(which ghc) Nix Store Real Dependency Graph Generated by Nix: ghc

Slide 41

Slide 41 text

Nix Store Nix Store Packages Are Read-only upon Build glibc-2.27 bash-4.4 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 ghc-8.4.4 nodejs-8.15.1 ruby-2.5.4 python-2.7.15

Slide 42

Slide 42 text

Nix Store Reusable Packages bash-interactive-4.4 glibc-2.27 bash-4.4 readline-7.0 ncurses-6.1

Slide 43

Slide 43 text

Nix Store Reusable Packages bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 ncurses-6.1 readline-7.0 pcre-8.42

Slide 44

Slide 44 text

Nix Store Reusable Packages bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43

Slide 45

Slide 45 text

Nix Store Reusable Packages bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 #Immutable

Slide 46

Slide 46 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43

Slide 47

Slide 47 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 Unused

Slide 48

Slide 48 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 Unused #GarbageCollection

Slide 49

Slide 49 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 GC Roots GC Roots

Slide 50

Slide 50 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 GC Roots GC Roots

Slide 51

Slide 51 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.43 GC Roots GC Roots

Slide 52

Slide 52 text

Q: How to modify a package?

Slide 53

Slide 53 text

Q: How to modify a package? A: You don't. You create a new one.

Slide 54

Slide 54 text

Nix Store: Cryptographic Hash Hash
 Function Source Code Dependencies Build Script Build Arguments Environment Variables Build Shell Package Name mn4jdnhkz12a6yd6jg6wvb4mqpxf8q1f Cryptographic Hash

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

Cryptographic Hash

Slide 57

Slide 57 text

Nix Store This is Possible Same Package Name And "Version" bash-4.4 bash-4.4 With Patches

Slide 58

Slide 58 text

Nix Store This is Possible Same Package Name And "Version" bash-4.4 bash-4.4 Cryptographic Hashes are different With Patches

Slide 59

Slide 59 text

Nix Store: Cryptographic Hash Hash
 Function Source Code Dependencies Build Script Build Arguments Environment Variables Build Shell Package Name mn4jdnhkz12a6yd6jg6wvb4mqpxf8q1f { Build Inputs Cryptographic Hash

Slide 60

Slide 60 text

Nix Store: Cryptographic Hash Hash
 Function Source Code Dependencies Build Script Build Arguments Environment Variables Build Shell Package Name mn4jdnhkz12a6yd6jg6wvb4mqpxf8q1f { Build Inputs Cryptographic Hash #PurelyFunctional

Slide 61

Slide 61 text

Nix Language

Slide 62

Slide 62 text

Nix Language A pure, lazy, functional language

Slide 63

Slide 63 text

Nix Language The Language used to write expressions 
 that produce derivations

Slide 64

Slide 64 text

Derivation Nix-speak for a package build action, which derives the package from its inputs.

Slide 65

Slide 65 text

Packages are (special) variables in the Nix Language

Slide 66

Slide 66 text

Your whole software and its dependencies are declared as data structure in a programming language

Slide 67

Slide 67 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 default.nix let pkgs = import { }; in derivation { name = "hello"; system = "x86_64-linux"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" '' export PATH="${pkgs.coreutils}/bin:${pkgs.gcc}/bin" mkdir $out gcc "${./hello.c}" -o "$out/hello" '' ]; }

Slide 68

Slide 68 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 default.nix let pkgs = import { }; in derivation { name = "hello"; system = "x86_64-linux"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" '' export PATH="${pkgs.coreutils}/bin:${pkgs.gcc}/bin" mkdir $out gcc "${./hello.c}" -o "$out/hello" '' ]; }

Slide 69

Slide 69 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 default.nix let pkgs = import { }; in derivation { name = "hello"; system = "x86_64-linux"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" '' export PATH="${pkgs.coreutils}/bin:${pkgs.gcc}/bin" mkdir $out gcc "${./hello.c}" -o "$out/hello" '' ]; } Build Inputs

Slide 70

Slide 70 text

Nixpkgs

Slide 71

Slide 71 text

~= stdlib For Nix Language

Slide 72

Slide 72 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 default.nix let pkgs = import { }; in derivation { name = "hello"; system = "x86_64-linux"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" '' export PATH="${pkgs.coreutils}/bin:${pkgs.gcc}/bin" mkdir $out gcc "${./hello.c}" -o "$out/hello" '' ]; } bash from Nixpkgs gcc from Nixpkgs

Slide 73

Slide 73 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 default.nix let pkgs = import { }; in derivation { name = "hello"; system = "x86_64-linux"; builder = "${pkgs.bash}/bin/bash"; args = [ "-c" '' export PATH="${pkgs.coreutils}/bin:${pkgs.gcc}/bin" mkdir $out gcc "${./hello.c}" -o "$out/hello" '' ]; } bash from Nixpkgs gcc from Nixpkgs #Lazy

Slide 74

Slide 74 text

~= Channel For Nix Package Manager

Slide 75

Slide 75 text

You can create a custom package collection that adds or overrides packages in Nixpkgs https://github.com/zetavg/nix-packages

Slide 76

Slide 76 text

NixOS

Slide 77

Slide 77 text

NixOS Linux distribution, build via Nix

Slide 78

Slide 78 text

NixOS Packages, Kernal, Configurations... are built by Nix

Slide 79

Slide 79 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 configuration.nix { config, pkgs, lib, ... }: { fileSystems."/home" = { device = "/dev/disk/by-label/home"; }; environment.systemPackages = [ pkgs.git pkgs.vim ]; services.nginx = { enable = true; package = pkgs.nginx.overrideAttrs (oldAttrs: { patches = [ (builtins.toFile "patch" '' --- ./src/core/ngx_conf_file.c +++ ./src/core/ngx_conf_file.c @@ -2,10 +2,10 @@ -#define NGX_CONF_BUFFER 4096 +#define NGX_CONF_BUFFER 131072 '') ]; }); virtualHosts."example.com" = { serverAliases = [ "www.example.com" ]; enableACME = true; forceSSL = true; root = someSitePackage.root; }; }; }

Slide 80

Slide 80 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 configuration.nix { config, pkgs, lib, ... }: { fileSystems."/home" = { device = "/dev/disk/by-label/home"; }; environment.systemPackages = [ pkgs.git pkgs.vim ]; services.nginx = { enable = true; package = pkgs.nginx.overrideAttrs (oldAttrs: { patches = [ (builtins.toFile "patch" '' --- ./src/core/ngx_conf_file.c +++ ./src/core/ngx_conf_file.c @@ -2,10 +2,10 @@ -#define NGX_CONF_BUFFER 4096 +#define NGX_CONF_BUFFER 131072 '') ]; }); virtualHosts."example.com" = { serverAliases = [ "www.example.com" ]; enableACME = true; forceSSL = true; root = someSitePackage.root; }; }; } #Declarative

Slide 81

Slide 81 text

NixOS Whole System in the Nix Store Nix Store nixos-system bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 pcre-8.42 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 etc etc-user.conf nginx.service ... *Lots of stuff are omitted or simplified to fit in this limited space. nginx.conf

Slide 82

Slide 82 text

NixOS Switch Between System Versions: Update Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space. nixos rebuild --switch

Slide 83

Slide 83 text

NixOS Switch Between System Versions Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 84

Slide 84 text

NixOS Switch Between System Versions Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 85

Slide 85 text

NixOS Switch Between System Versions Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 86

Slide 86 text

NixOS Multiple Systems Coexist Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 87

Slide 87 text

Boot Into a Specific Version

Slide 88

Slide 88 text

nix-env

Slide 89

Slide 89 text

Your Shell Environment

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

Generations Switch between different nix-env generations

Slide 93

Slide 93 text

NixOS Switch Between System Versions Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 94

Slide 94 text

NixOS Switch Between System Versions Nix Store nixos-system-xxx-1 nixos-system-xxx-2 nixos-system-xxx-3 bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 pcre-8.42 nginx-1.14.2 passenger-6.0.2 pcre-8.43 openssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 ncurses-6.1 readline-7.0 zlib-1.2.11 *Lots of stuff are omitted or simplified to fit in this limited space.

Slide 95

Slide 95 text

No content

Slide 96

Slide 96 text

nix-shell

Slide 97

Slide 97 text

For
 Software Development

Slide 98

Slide 98 text

1. Get into the shell environment that builds a specific package

Slide 99

Slide 99 text

2. Write Nix expressions to declare a shell environment and jump into it

Slide 100

Slide 100 text

2. Write Nix expressions to declare a shell environment and jump into it #Declarative

Slide 101

Slide 101 text

Wrap Up

Slide 102

Slide 102 text

Nix Store: Cryptographic Hash Hash
 Function Source Code Dependencies Build Script Build Arguments Environment Variables Build Shell Package Name mn4jdnhkz12a6yd6jg6wvb4mqpxf8q1f { Build Inputs Cryptographic Hash #PurelyFunctional

Slide 103

Slide 103 text

Nix Store Reusable Packages bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 #Immutable

Slide 104

Slide 104 text

NixOS Whole System in the Nix Store nixos-system 42 penssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 zlib-1.2.11 etc etc-user.conf nginx.service ... *Lots of stuff are omitted or simplified to fit in this limited space. nginx.conf 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 configuration.nix { config, pkgs, lib, ... }: { fileSystems."/home" = { device = "/dev/disk/by-label/home"; }; environment.systemPackages = [ pkgs.git services.nginx = { enable = true; package = pkgs.nginx.overrideAttrs (o patches = [ (builtins.toFile "patch --- ./src/core/ngx_conf_file.c +++ ./src/core/ngx_conf_file.c @@ -2,10 +2,10 @@ -#define NGX_CONF_BUFFER 4096 +#define NGX_CONF_BUFFER 131072 '') ]; }); virtualHosts."example.com" = { serverAliases = [ "www.example.com" #Declarative

Slide 105

Slide 105 text

NixOS Whole System in the Nix Store nixos-system 42 penssl-1.1.1 libxslt-1.1.33 libxml2-2.9.9 nginx-1.14.2 zlib-1.2.11 etc etc-user.conf nginx.service ... *Lots of stuff are omitted or simplified to fit in this limited space. nginx.conf 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 configuration.nix { config, pkgs, lib, ... }: { fileSystems."/home" = { device = "/dev/disk/by-label/home"; }; environment.systemPackages = [ pkgs.git services.nginx = { enable = true; package = pkgs.nginx.overrideAttrs (o patches = [ (builtins.toFile "patch --- ./src/core/ngx_conf_file.c +++ ./src/core/ngx_conf_file.c @@ -2,10 +2,10 @@ -#define NGX_CONF_BUFFER 4096 +#define NGX_CONF_BUFFER 131072 '') ]; }); virtualHosts."example.com" = { serverAliases = [ "www.example.com" #Lazy #Declarative

Slide 106

Slide 106 text

Nix Store Package Removal bash-interactive-4.4 glibc-2.27 bash-4.4 zsh-5.6.2 zsh-5.7.1 ncurses-6.1 readline-7.0 pcre-8.42 pcre-8.43 GC Roots GC Roots #GarbageCollection

Slide 107

Slide 107 text

More ▪ Cachix: Nix binary cache as a service ▪ Hydra: A Nix build farm ▪ NixOps: Use Nix to declaratively deploy infrastructures

Slide 108

Slide 108 text

Stuff I Done ▪ NixOS on GCP ▪ https://github.com/zetavg/nixos-configs ▪ https://github.com/zetavg/nix-packages ▪ Phusion Passenger and nginx-mod-passenger ▪ "NPM" implemented with Nix ▪ Patched version of some programms

Slide 109

Slide 109 text

Recommend Materials ▪ Functional Thursday: Nix and NixOS牧9/5 at Mozilla מ嬝玟旰獍ਰ牐 ▪ Nix Pills: https://nixos.org/nixos/nix-pills/
 㮆Ո磧വ萃ጱ獈槹瞲ܖ牧و磪 20 缰Ꭸ缰෈ᒍ牧ℂਞ蕕牏च๜ֵአ牏ܻቘ牧Ӟ ᪠窩荠ک承᥺牏ই֜䌃ॺկ牏Ӟ犚礍䯤޾戔懯秇ୗ牧犋㰍硽֦ই֜አ Nix牧捝 ਠԏ盅螭胼犨఺ู硬֦ጱ羬翄犥现۱蕕֦ᛔ૩ጱ package牐 ▪ Continuous integration for iOS with Nix and Buildkite - Pinterest Engineering
 Pinterest ጱૡ纷䒍獤Ձ犢㮉ই֜አ Nix 㬵翄Ӟ犢㮉 mobile app ࣁ樄咳ᘏጱ秚 瑊犥现 CI Ӥጱ絑ह牧㪔አ Buildkite 蝡ॺ CI Interface as a Service 㬵಑蝨犢 㮉ጱ CI 絑ह牐 ▪ Nix: Haskell Concepts for Package Management - HackerNoon.com
 Օ奧 Nix ই֜ڥአ Functional牏Declarative ጱ粬௔牧薹究㯽翄羬翄絑ह਻ฃ ᤩ࿱礕牏篷ဩ粚๜矒ᓕ缛㺔氂牐 ▪ Eelco Dolstra ࣁ艰葦僂粬ۦඪय़䋊ጱ玡ॊ抷෈
 ฎ Nix ॺկᓕቘૡٍጱ獮蛪牐