Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dhall Configuration Language

Dhall Configuration Language

Dhall application to Kubernetes app deployment

Alexey Novakov

March 08, 2021
Tweet

More Decks by Alexey Novakov

Other Decks in Programming

Transcript

  1. Dhall Configuration Language
    Use Case: deploy Kafka to Kubernetes
    Alexey Novakov, EPAM, Germany

    View Slide

  2. Problem
    • App configuration in YAML is error-prone
    • Config error may cost you days of ops work
    • Code duplications leads to maintenance hell
    Typical mistakes:
    • Incorrect value semantic (string vs. number)
    • Incorrect object structure (no schema)
    • Incorrect indent (spaces)

    View Slide

  3. Solution
    We need programmable configuration files
    Configuration Programming Laguage:
    • Non-turing complete -> better security
    • Deterministic
    • Staticly-typed
    • It should support YAML/JSON/XML

    View Slide

  4. Dhall Configuration Language
    • JSON + functions + types + imports
    • Strong safety guarantees and more powerful tooling
    -- example.dhall
    let user = "bill"
    in { home = "/home/${user}"
    , privateKey = "/home/${user}/.ssh/id_ed25519"
    , publicKey = "/home/${user}/.ssh/id_ed25519.pub"
    }
    $ dhall-to-yaml < ./example.dhall
    home: /home/bill
    privateKey: /home/bill/.ssh/id_ed25519
    publicKey: /home/bill/.ssh/id_ed25519.pub
    Dhall programs never fail,
    hang, crash, leak secrets,
    or compromise your
    system.

    View Slide

  5. Dhall ecosystem
    • CLI (static binaries):
    • dhall-to-yaml, dhall-to-json, dhall-to-
    text, others
    • yaml-to-dhall
    • dhall (REPL, format, + many other
    options)
    • Plugins: VSCode, Emacs, Vim
    • Documentation: docs.dhall-lang.org
    • Community:
    Packages Languages

    View Slide

  6. Example: Kafka Deployment
    1. Customized Confluent Helm Charts (YAML)
    • Kafka Brokers
    • ZooKeeper
    • Custom volumes for jaas.conf, krb5.conf, keytabs
    2. Security Configs (Dhall)
    • ConfigMaps:
    • jaas.conf
    • client.properties
    • Client Pod spec
    https://github.com/novakov-alexey/dhall-kafka-kuberentes-deployment

    View Slide

  7. Kafka Authentication
    • SSL
    • SASL:
    • GSSAPI (Kerberos)
    • OAUTHBEARER
    • SCRAM
    • PLAIN
    • Delegation Tokens
    • LDAP

    View Slide

  8. Deploy Kafka
    $ helm install $(PLAIN_RELEASE) ./kafka/helm/cp-kafka \
    --values ./kafka/helm/cp-kafka/sasl-plain-values.yaml -n $(NAMESPACE)
    kubectl get po -n kafka -l 'app in (cp-kafka,cp-zookeeper)'
    NAME READY STATUS RESTARTS AGE
    plain-cp-kafka-0 1/1 Running 1 55s
    plain-cp-zookeeper-0 1/1 Running 0 55s
    $ dhall-to-yaml --documents < ./krb/krb5.dhall | kubectl create -n $(NAMESPACE) -f –
    $ dhall-to-yaml --documents < ./kafka/manifest/brokerConf.dhall | kubectl create -n $(NAMESPACE) -f -

    View Slide

  9. Deploy Client
    $ SASL_MECHANISM=".PLAIN" \
    dhall-to-yaml < ./kafka/manifest/clientPod.dhall | kubectl create -n $(NAMESPACE) -f -
    $ kubectl get po -n kafka -l app=cp-kafka-client
    NAME READY STATUS RESTARTS AGE
    plain-kafka-client 1/1 Running 0 47s
    Consumer Producer

    View Slide

  10. Links
    • Example: https://github.com/novakov-alexey/dhall-kafka-kuberentes-
    deployment
    • Main site: https://dhall-lang.org/
    • Wiki/Docs: https://docs.dhall-lang.org/index.html
    • Forum: https://discourse.dhall-lang.org/
    • Dhall-Kubernetes package: https://github.com/dhall-lang/dhall-
    kubernetes/blob/master/README.md

    View Slide