Dhall Configuration Language

Dhall application to Kubernetes app deployment

Transcript

1. Dhall Configuration Language Use Case: deploy Kafka to Kubernetes Alexey Novakov, EPAM, Germany
2. Problem • App configuration in YAML is error-prone • Config error may cost you days of ops work • Code duplications leads to maintenance hell Typical mistakes: • Incorrect value semantic (string vs. number) • Incorrect object structure (no schema) • Incorrect indent (spaces)
3. Solution We need programmable configuration files Configuration Programming Laguage: • Non-turing complete -> better security • Deterministic • Staticly-typed • It should support YAML/JSON/XML
4. Dhall Configuration Language • JSON + functions + types + imports • Strong safety guarantees and more powerful tooling -- example.dhall let user = "bill" in { home = "/home/${user}" , privateKey = "/home/${user}/.ssh/id_ed25519" , publicKey = "/home/${user}/.ssh/id_ed25519.pub" } $ dhall-to-yaml < ./example.dhall home: /home/bill privateKey: /home/bill/.ssh/id_ed25519 publicKey: /home/bill/.ssh/id_ed25519.pub Dhall programs never fail, hang, crash, leak secrets, or compromise your system.
5. Dhall ecosystem • CLI (static binaries): • dhall-to-yaml, dhall-to-json, dhall-to- text, others • yaml-to-dhall • dhall (REPL, format, + many other options) • Plugins: VSCode, Emacs, Vim • Documentation: docs.dhall-lang.org • Community: Packages Languages
6. Example: Kafka Deployment 1. Customized Confluent Helm Charts (YAML) • Kafka Brokers • ZooKeeper • Custom volumes for jaas.conf, krb5.conf, keytabs 2. Security Configs (Dhall) • ConfigMaps: • jaas.conf • client.properties • Client Pod spec https://github.com/novakov-alexey/dhall-kafka-kuberentes-deployment
7. Kafka Authentication • SSL • SASL: • GSSAPI (Kerberos) • OAUTHBEARER • SCRAM • PLAIN • Delegation Tokens • LDAP
8. Deploy Kafka $ helm install $(PLAIN_RELEASE) ./kafka/helm/cp-kafka \ --values ./kafka/helm/cp-kafka/sasl-plain-values.yaml -n $(NAMESPACE) kubectl get po -n kafka -l 'app in (cp-kafka,cp-zookeeper)' NAME READY STATUS RESTARTS AGE plain-cp-kafka-0 1/1 Running 1 55s plain-cp-zookeeper-0 1/1 Running 0 55s $ dhall-to-yaml --documents < ./krb/krb5.dhall | kubectl create -n $(NAMESPACE) -f – $ dhall-to-yaml --documents < ./kafka/manifest/brokerConf.dhall | kubectl create -n $(NAMESPACE) -f -
9. Deploy Client $ SASL_MECHANISM="<PLAIN|GSSAPI>.PLAIN" \ dhall-to-yaml < ./kafka/manifest/clientPod.dhall | kubectl create -n $(NAMESPACE) -f - $ kubectl get po -n kafka -l app=cp-kafka-client NAME READY STATUS RESTARTS AGE plain-kafka-client 1/1 Running 0 47s Consumer Producer
10. Links • Example: https://github.com/novakov-alexey/dhall-kafka-kuberentes- deployment • Main site: https://dhall-lang.org/ • Wiki/Docs: https://docs.dhall-lang.org/index.html • Forum: https://discourse.dhall-lang.org/ • Dhall-Kubernetes package: https://github.com/dhall-lang/dhall- kubernetes/blob/master/README.md