Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dhall Configuration Language

Dhall Configuration Language

Dhall application to Kubernetes app deployment

7a04b88e1469561db6da3818348d4b8f?s=128

Alexey Novakov

March 08, 2021
Tweet

Transcript

  1. Dhall Configuration Language Use Case: deploy Kafka to Kubernetes Alexey

    Novakov, EPAM, Germany
  2. Problem • App configuration in YAML is error-prone • Config

    error may cost you days of ops work • Code duplications leads to maintenance hell Typical mistakes: • Incorrect value semantic (string vs. number) • Incorrect object structure (no schema) • Incorrect indent (spaces)
  3. Solution We need programmable configuration files Configuration Programming Laguage: •

    Non-turing complete -> better security • Deterministic • Staticly-typed • It should support YAML/JSON/XML
  4. Dhall Configuration Language • JSON + functions + types +

    imports • Strong safety guarantees and more powerful tooling -- example.dhall let user = "bill" in { home = "/home/${user}" , privateKey = "/home/${user}/.ssh/id_ed25519" , publicKey = "/home/${user}/.ssh/id_ed25519.pub" } $ dhall-to-yaml < ./example.dhall home: /home/bill privateKey: /home/bill/.ssh/id_ed25519 publicKey: /home/bill/.ssh/id_ed25519.pub Dhall programs never fail, hang, crash, leak secrets, or compromise your system.
  5. Dhall ecosystem • CLI (static binaries): • dhall-to-yaml, dhall-to-json, dhall-to-

    text, others • yaml-to-dhall • dhall (REPL, format, + many other options) • Plugins: VSCode, Emacs, Vim • Documentation: docs.dhall-lang.org • Community: Packages Languages
  6. Example: Kafka Deployment 1. Customized Confluent Helm Charts (YAML) •

    Kafka Brokers • ZooKeeper • Custom volumes for jaas.conf, krb5.conf, keytabs 2. Security Configs (Dhall) • ConfigMaps: • jaas.conf • client.properties • Client Pod spec https://github.com/novakov-alexey/dhall-kafka-kuberentes-deployment
  7. Kafka Authentication • SSL • SASL: • GSSAPI (Kerberos) •

    OAUTHBEARER • SCRAM • PLAIN • Delegation Tokens • LDAP
  8. Deploy Kafka $ helm install $(PLAIN_RELEASE) ./kafka/helm/cp-kafka \ --values ./kafka/helm/cp-kafka/sasl-plain-values.yaml

    -n $(NAMESPACE) kubectl get po -n kafka -l 'app in (cp-kafka,cp-zookeeper)' NAME READY STATUS RESTARTS AGE plain-cp-kafka-0 1/1 Running 1 55s plain-cp-zookeeper-0 1/1 Running 0 55s $ dhall-to-yaml --documents < ./krb/krb5.dhall | kubectl create -n $(NAMESPACE) -f – $ dhall-to-yaml --documents < ./kafka/manifest/brokerConf.dhall | kubectl create -n $(NAMESPACE) -f -
  9. Deploy Client $ SASL_MECHANISM="<PLAIN|GSSAPI>.PLAIN" \ dhall-to-yaml < ./kafka/manifest/clientPod.dhall | kubectl

    create -n $(NAMESPACE) -f - $ kubectl get po -n kafka -l app=cp-kafka-client NAME READY STATUS RESTARTS AGE plain-kafka-client 1/1 Running 0 47s Consumer Producer
  10. Links • Example: https://github.com/novakov-alexey/dhall-kafka-kuberentes- deployment • Main site: https://dhall-lang.org/ •

    Wiki/Docs: https://docs.dhall-lang.org/index.html • Forum: https://discourse.dhall-lang.org/ • Dhall-Kubernetes package: https://github.com/dhall-lang/dhall- kubernetes/blob/master/README.md