Dokku Internals Random acts of shell scripts 

What is Dokku Exactly? Or what isn’t it 

What is Dokku? • A set of executables acting as a pipeline • Dokku takes some input and produces one or more artifacts • A large collection of shell scripts 

What isn’t Dokku • Not a Platform as a Service, but commonly used as one • A promoter of whale sushi • Strictly docker-based technology 

Some History Because reading is fundamental 

History • First “killer app” for docker • Initially strung together Git + Nginx + Docker + Buildpacks with ~100 lines of bash • Supports deploys to Docker, Kubernetes, Nomad • 5k commits, 17k+ stars, 140 releases, 200+ community plugins 

History • 2013 : Started by Jeff Lindsay (@progrium) • 2014: Taken over by Jose Gonzalez • 2015: Official Datastore plugins • 2016: Zero downtime deploys • 2017: Started slow rewrite into golang • 2018: Herokuish 0.4.0/Ubuntu 16.04 base • 2019: Kubernetes/Nomad support 

Dokku Workflow Mechanism How does it know what to do? 

git push dokku master • On git push, associated ssh key triggers a command configured in /home/ dokku/.sshcommand • dokku binary implements git-receive-pack and writes code to /home/dokku/$APP • /home/dokku/$APP/hooks/pre-receive contains code that triggers the build process, and streams back output to stdout • dokku/git-http-backend (OSS) provides http(s) support, writing the code to the $APP directory, and chunks back output over http(s) to the client • This bypasses ssh key check, authentication must be performed via basic-auth or similar within a proxy in front of the git-http-backend. • Only master branch is built, but this can be configured • dokku git:set $APP deploy-branch $NEW_BRANCH 

Dokku command overview • Dokku commands provide the majority of the Heroku-like experience. They wrap common infrastructure tasks that are hard to remember/easy to mix up. • This is an alternative to using tools such as Ansible, Chef, or Puppet, though may be used in conjunction with them (e.g. ansible-dokku) • dokku binary takes many arguments. First argument is the subcommand, all others are passed to subcommand. Some simple flag parsing is also performed here. • Subcommands can be any executable code, but commonly shell. • Subcommands are grouped by common functionality • The git plugin contains triggers and subcommands related to git, while the config plugin handles environment variable management. 

Extending Subcommands • Subcommands can call any “exported” functions from other plugins • Convention is to place exportable functions in a functions file, while non-exportable functions should be in internal-functions • Subcommands can also “call” plugin triggers via plugn binary to avoid sourcing other plugins • Plugin triggers are commonly used to enable/disable functionality 

Plugin Internals • All plugin triggers are captured as events in a log via the events plugin (dokku events:on) • Core plugins can be swapped out for alternative implementations (dokku-scheduler-kubernetes) • Core plugins generally are not disabled, but extended via triggers • Alternative implementations may result in a user disabling a core plugin. A good example would be an alternative proxy implementation around caddy or haproxy • Core plugins provide helper functions for community plugins to utilize (config/property management) 

Plugin Examples • Smoke Test Plugin: https://github.com/dokku/ smoke-test-plugin • Postgres Datastore: https://github.com/dokku/ dokku-postgres • Kubernetes Scheduler: https://github.com/dokku/ dokku-scheduler-kubernetes • See http://dokku.viewdocs.io/dokku/development/ plugin-creation/ for plugin creation tips 

Simplest plugin post-deploy bash script 

Buildpacks The secret sauce to app building 

History • Started by Heroku in 2010/2011 for their PaaS • Two standards now • Heroku: Buildpack v2a • Cloud Foundry: Buildpack v2b • New joint standard (v3) under development • See https://buildpacks.io for more information 

Buildpack v2A • A bundle of executables • bin/detect: detects whether the buildpack applies • bin/compile: compiles dependencies and app code • bin/release: specifies default processes and platform addons for the app • Generally also includes supporting building code • Normally only targets a single language 

Who maintains buildpacks? • Heroku has a collection of 10 official buildpacks, any other unofficial ones • CloudFoundry maintains their own v2b buildpacks • Hundreds of Community Buildpacks • Dokku project maintains the static buildpack • Plotly maintains custom Conda, Python, and R buildpacks • Maybe you? 

Buildpack distribution • Generally distributed as git repositories • Buildpacks are installed on “stacks”, or base operating systems • gliderlabs/herokuish provides a docker- compatible layer for buildpacks (Dokku uses this) • Buildpack v3 will have officially provided stacks with built-in buildpacks 

Buildpack stacks • Provides ABI compatibility for compiled dependencies • Provides base OS + common utilities necessary to run an app • v2a Current options: cedar-10, cedar-14, heroku-16, heroku-18 • v3 options: bionic stack (heroku maintained) 

Herokuish • Herokuish is: • A collection of buildpacks • A binary that can act as an init for apps • A docker image • Released in a single stack flavor (currently heroku-18) • Can be further customized for dokku via DOKKU_IMAGE environment variable • Will be deprecated in favor of buildpack v3 pack-cli or equivalent 

Buildpack output • Binaries or source + dependencies • Golang/Java: Binary • R/Python: Source + dependencies • Export Formats • Cloud Foundry: Exports a tarball “droplet” of /app directory • Heroku: Exports a tarball “slug" of /app directory • Dokku: Provides a docker image of all changes • Dokku is the odd one here 

Docker In Dokku “What is it good for? Absolutely nothing!” 

Image Building Buildpacks avoid Dockerfiles, so how do we create a docker image? 

Intermediate Containers • docker run/wait/commit • Run a command inside of a container and commits output to docker image • Workflow allows for continually adding/removing from output image (e.g. dokku-apt) • Built image no different from any other docker image 

Scheduling • Default to docker-local scheduling • docker run --restart-policy $IMAGE… • Kubernetes support via plugin • Depends on the (external but official) registry plugin and an external registry • Supports both process and resource scaling • Currently requires a custom routing implementation • Nomad support via plugin • Similar requirements as Kubernetes support • Others? 

Datastores? • Datastores are separate containers running with mounted directories for storage • All common datastores are currently implemented as OSS plugins (Elasticsearch, MySQL, Postgres, Redis, etc.) • Upcoming Couchbase, Neo4j, Omnisci • Old docker --link flag used for communication • Sometimes does not withstand datastore restarts • Upcoming release will switch to non-deprecated networks • No clustering support on OSS datastores plugins • No resource limits specific on OSS datastores plugins • Should there be? 

Cross-container communication • Not possible except with: • intermediary (queuing through redis) • plain old http over open network • Commonly requested OSS feature (two apps talking to each other) • Maybe not as requested in commercial settings? • Upcoming release will implement official networking support • Strictly opt-in, but not multi-tenant 

Cross-server communication • If the server has access to it, so should the apps • No special docker0 rules in place to block inbound/outbound • This applies to IAM roles and other things living on the server • Outgoing access can be controlled via iptables • Many security softwares can break docker0, causing invalid networking for docker containers! • Other access control should be done via password or firewall 

thats it