Dhall Configuration Language

Dhall application to Kubernetes app deployment

Transcript

1. Dhall Configuration Language
   Use Case: deploy Kafka to Kubernetes
   Alexey Novakov, EPAM, Germany
   

   View Slide

2. Problem
   • App configuration in YAML is error-prone
   • Config error may cost you days of ops work
   • Code duplications leads to maintenance hell
   Typical mistakes:
   • Incorrect value semantic (string vs. number)
   • Incorrect object structure (no schema)
   • Incorrect indent (spaces)
   

   View Slide

3. Solution
   We need programmable configuration files
   Configuration Programming Laguage:
   • Non-turing complete -> better security
   • Deterministic
   • Staticly-typed
   • It should support YAML/JSON/XML
   

   View Slide

4. Dhall Configuration Language
   • JSON + functions + types + imports
   • Strong safety guarantees and more powerful tooling
   -- example.dhall
   let user = "bill"
   in { home = "/home/${user}"
   , privateKey = "/home/${user}/.ssh/id_ed25519"
   , publicKey = "/home/${user}/.ssh/id_ed25519.pub"
   }
   $ dhall-to-yaml < ./example.dhall
   home: /home/bill
   privateKey: /home/bill/.ssh/id_ed25519
   publicKey: /home/bill/.ssh/id_ed25519.pub
   Dhall programs never fail,
   hang, crash, leak secrets,
   or compromise your
   system.
   

   View Slide

5. Dhall ecosystem
   • CLI (static binaries):
   • dhall-to-yaml, dhall-to-json, dhall-to-
   text, others
   • yaml-to-dhall
   • dhall (REPL, format, + many other
   options)
   • Plugins: VSCode, Emacs, Vim
   • Documentation: docs.dhall-lang.org
   • Community:
   Packages Languages
   

   View Slide

6. Example: Kafka Deployment
   1. Customized Confluent Helm Charts (YAML)
   • Kafka Brokers
   • ZooKeeper
   • Custom volumes for jaas.conf, krb5.conf, keytabs
   2. Security Configs (Dhall)
   • ConfigMaps:
   • jaas.conf
   • client.properties
   • Client Pod spec
   https://github.com/novakov-alexey/dhall-kafka-kuberentes-deployment
   

   View Slide

7. Kafka Authentication
   • SSL
   • SASL:
   • GSSAPI (Kerberos)
   • OAUTHBEARER
   • SCRAM
   • PLAIN
   • Delegation Tokens
   • LDAP
   

   View Slide

8. Deploy Kafka
   $ helm install $(PLAIN_RELEASE) ./kafka/helm/cp-kafka \
   --values ./kafka/helm/cp-kafka/sasl-plain-values.yaml -n $(NAMESPACE)
   kubectl get po -n kafka -l 'app in (cp-kafka,cp-zookeeper)'
   NAME READY STATUS RESTARTS AGE
   plain-cp-kafka-0 1/1 Running 1 55s
   plain-cp-zookeeper-0 1/1 Running 0 55s
   $ dhall-to-yaml --documents < ./krb/krb5.dhall | kubectl create -n $(NAMESPACE) -f –
   $ dhall-to-yaml --documents < ./kafka/manifest/brokerConf.dhall | kubectl create -n $(NAMESPACE) -f -
   

   View Slide

9. Deploy Client
   $ SASL_MECHANISM=".PLAIN" \
   dhall-to-yaml < ./kafka/manifest/clientPod.dhall | kubectl create -n $(NAMESPACE) -f -
   $ kubectl get po -n kafka -l app=cp-kafka-client
   NAME READY STATUS RESTARTS AGE
   plain-kafka-client 1/1 Running 0 47s
   Consumer Producer
   

   View Slide

10. Links
    • Example: https://github.com/novakov-alexey/dhall-kafka-kuberentes-
    deployment
    • Main site: https://dhall-lang.org/
    • Wiki/Docs: https://docs.dhall-lang.org/index.html
    • Forum: https://discourse.dhall-lang.org/
    • Dhall-Kubernetes package: https://github.com/dhall-lang/dhall-
    kubernetes/blob/master/README.md
    

    View Slide