Haskell >>= DevOps Vladimir Kirillov / @darkproger 

• Pla%orm So+ware / SRE at Zalora, CS undergrad at NTUU KPI • We're hiring to our distributed team! 

DevOps 

• configura*on management • running systems • monitoring • deployment / con*nuous delivery • immutable infrastructure • supervising failures • ... • system integra*on 

No content

Microgram • Applica(on realms (infra + configs) • Applica(on defini(ons (how to run / scale / monitor) • User APIs • Run(me converts defini(ons to real things (like infra) • Run(me handles opera(ons & automates labor • Fail 

DSLs everywhere 

systems as code 

/etc/passwd nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh daemon:*:1:1:System Services:/var/root:/usr/bin/false _uucp:*:4:4:Unix to Unix Copy Protocol:/var/spool/uucp:/usr/sbin/uucico _taskgated:*:13:13:Task Gate Daemon:/var/empty:/usr/bin/false _networkd:*:24:24:Network Services:/var/networkd:/usr/bin/false _installassistant:*:25:25:Install Assistant:/var/empty:/usr/bin/false _lp:*:26:26:Printing Services:/var/spool/cups:/usr/bin/false _postfix:*:27:27:Postfix Mail Server:/var/spool/postfix:/usr/bin/false _scsd:*:31:31:Service Configuration Service:/var/empty:/usr/bin/false 

/etc/pf.conf block return out on em0 inet all set queue std pass out on em0 inet proto tcp \ from $developerhosts to any port 80 \ set queue developers pass out on em0 inet proto tcp \ from $employeehosts to any port 80 \ set queue employees pass out on em0 inet proto tcp \ from any to any port 22 \ set queue(ssh_bulk, ssh_interactive) pass out on em0 inet proto tcp \ from any to any port 25 \ set queue mail 

sendmail.mc define(`confTO_CONNECT', `1m')dnl define(`confTO_IDENT', `0')dnl define(`confTO_COMMAND', `2m')dnl LOCAL_NET_CONFIG # This rule ensures that all local mail is delivered using the # smtp transport, everything else will go via the smart host. R$* < @ $* .$m. > $* $#smtp $@ $2.$m. $: $1 < @ $2.$m. > $3 

No content

JSON { "domain": "www.example.com", "mongodb": { "host": "localhost", "port": 27017 } } 

YAML tasks: - name: take out of load balancer pool command: /usr/bin/take_out_of_pool {{ inventory_hostname }} ## ^ string template delegate_to: 127.0.0.1 

"Programming" with JSON (CloudForma8on) "Outputs" : { "MyOutput" : { "Value" : { "Fn::Join" : [ "%", [ "A-string", {"Ref" : "AWS::StackName" } ] ] } } } 

"Programming" with YAML - command: /opt/application/upgrade_db.py when: inventory_hostname == webservers[0] 

Puppet Although Puppet’s language is built around describing resources (and the rela6onships between them) in a declara6ve way, several parts of the language do depend on evalua6on order case $operatingsystem { centos, redhat: { $service_name = 'ntpd' } debian, ubuntu: { $service_name = 'ntp' } } package { 'ntp': ensure => installed, } 

Chef ruby_block 'sleep30s' do block do sleep 30 end action :nothing end 

DSLs are like drugs 

Any insufficiently expressive DSL leads to the necessity of using an ul:mate systems integra:on tool. 

(drumroll) 

Which is bash. 

Bash • somewhat ubiquitous and mostly unportable • hard to scale • strings • more strings • unexpected EOF while looking for matching `"' • also strings 

Scaling Bash 

Scaling Bash with Nix 

No content

bash = "${pkgs.bash}/bin/bash"; base64 = "${pkgs.coreutils}/bin/base64"; jq = "${pkgs.jq}/bin/jq"; curl = "${pkgs.curl}/bin/curl -s --retry --fail"; awk = "${pkgs.gawk}/bin/awk"; openssl = "${pkgs.openssl}/bin/openssl"; 

No content

Nix • func&onal language • dynamically typed • lightweight "schema valida&on" (hard to say typing) • one side-effect (derivation, used to build a package manager framework) 

Nix mk-scope = paths: let self = { __nixPath = let to = k: v: { prefix = k; path = v; }; base = filterAttrs (p: _: p != "") scope-to-as; path = mapAttrsToList to (base // paths); in path; import = fn: self.scopedImport self fn; scopedImport = attrs: scopedImport (self // attrs); builtins = builtins // self; }; in self; 

Experiences with Nix 

• built a microgram implementa1on effec1vely compiled to "bash on steroids" • used a package manager toolchain for this • EDSL experiments in a dynamic language • hard to make private APIs • tried to add more side effects (like IO) • somewhat boring to work on 

The Haskell Journey 

Some OSS infra tools • zalora/upcast - declara1ve infra provisioning (like nixops/ terraform/fugue) • zalora/replicator - automated MySQL replica1on • zalora/sproxy - proxy that handles OAuth2 + ACL interface • zalora/aws-ec2 - EC2 extensions for aris1db/aws • unicron, a single-user cron • a lot more on Zalora's GitHub 

Lessons learned so far • a lot of effort spent on was/ng /me integra/ng with stringy world (e.g. to enable bash scrip/ng or SQL) • OTOH many of those tools integrate beAer with the UNIX world • ghci shell is a lot more fun to use • having to wait for builds to finish is a /me sink 

Deployment with nixpkgs • quite powerful • nixpkgs haskell infrastructure is hard to get to work • curated dependency sets before it was cool • lots of dependencies (disk space bloat) • especially bad with "distributed" builds (h

Deployment with anything else • people tend to build programs right on the target host • copy-paste of binaries between iden8cal systems (halcyon) • docker makes it easy to make distribu8on bloated (and generally sad) • high hopes for stack 

Deployment with anything else • hard to get builds fully sta2c (like golang does) • GHC with Musl + integer-simple to get rid of LGPL • btw GPL applies only for soAware redistribu2on to public :) • seriously consider GHCJS + node? 

The Grand Idea push dumb problems to solve fun problems 

hard to deploy haskell at scale - don't deploy it! :) don't write CLI tools! haskell is ideal for wri.ng glue to integrate various systems (control-plane services) 

The glue toolbox • parser combinators + quasi-quoters let you integrate with a lot of outside world • building and interpre:ng an AST is also nice • turn exis:ng CLI tools into libraries/DSLs (CLI integra:on to service integra:on) • wrap exis:ng interfaces with types 

Migra&ng from dynamically typed Nix • take all schemas and convert them to haskell • use Nix as a "query" language inside to access exis9ng code/data • Rebindable syntax is cool! Though you can't rebind let. types = { nullOr = fake-type-of "Maybe"; string = fake-type "Text"; int = fake-type "Integer"; bool = fake-type "Bool"; attrsOf = fake-type-of "Map"; listOf = fake-type-of "List"; unspecified = fake-type "Value"; }; 

Integra(ng with 3rd-party APIs • servant-client is awesome • not enough implementa1ons for things like swagger (remember WSDL?) • amazonka has a great tailor-made code generator for AWS • perhaps something like F# type providers one day? 

Interpre'ng effects • free / opera*onal / prompt / extensible-effects • Freer monads, More extensible effects, ICFP 2015 

Integra(ng with bash • can't fully replace bash with turtle • use a bash DSL! data Expr :: * -> * where E :: Executable -> e -> Expr e Pipe :: Expr e -> Expr e -> Expr e Seq :: Expr e -> Expr e -> Expr e Or :: Expr e -> Expr e -> Expr e Redir :: Expr e -> FilePath -> Expr e Env :: [Pair] -> Expr e -> Expr e Sudo :: Expr e -> Expr e SSH :: Hostname -> e -> Expr e -> Expr e 

Integra(ng with the rest of the stringy world • UNIX is a minefield for experimen5ng with parser combinators! • a lot of perf analysis or systems explora5on is done by analysing streams of text • use haskell if lost in awk+perl+sed • see proger/lxkit and zalora/gctuner 

Other notable Haskell tools • propellor • shake/bake • literate haskell (idea from org-mode with emacs) 

No content

Takeaways • define your domain • isolate your effects • apply transforms • stay away from boring 8me sinks • cheat • don't forget to bring added value from the start because nobody cares about rewrites 

There are infinitely many DSLs to do cool things There are not enough languages that are powerful enough to map those domains PS. Please don't use bash