A technical talk where I'll be going over what happens between when a user performs a "git push" and the app becomes available for viewing. Topics will include the Heroku buildpack lifecycle, docker container usage, and the Dokku plugin system.
2014: Taken over by Jose Gonzalez • 2015: Ofﬁcial Datastore plugins • 2016: Zero downtime deploys • 2017: Started slow rewrite into golang • 2018: Herokuish 0.4.0/Ubuntu 16.04 base • 2019: Kubernetes/Nomad support
key triggers a command conﬁgured in /home/ dokku/.sshcommand • dokku binary implements git-receive-pack and writes code to /home/dokku/$APP • /home/dokku/$APP/hooks/pre-receive contains code that triggers the build process, and streams back output to stdout • dokku/git-http-backend (OSS) provides http(s) support, writing the code to the $APP directory, and chunks back output over http(s) to the client • This bypasses ssh key check, authentication must be performed via basic-auth or similar within a proxy in front of the git-http-backend. • Only master branch is built, but this can be conﬁgured • dokku git:set $APP deploy-branch $NEW_BRANCH
the Heroku-like experience. They wrap common infrastructure tasks that are hard to remember/easy to mix up. • This is an alternative to using tools such as Ansible, Chef, or Puppet, though may be used in conjunction with them (e.g. ansible-dokku) • dokku binary takes many arguments. First argument is the subcommand, all others are passed to subcommand. Some simple ﬂag parsing is also performed here. • Subcommands can be any executable code, but commonly shell. • Subcommands are grouped by common functionality • The git plugin contains triggers and subcommands related to git, while the conﬁg plugin handles environment variable management.
other plugins • Convention is to place exportable functions in a functions ﬁle, while non-exportable functions should be in internal-functions • Subcommands can also “call” plugin triggers via plugn binary to avoid sourcing other plugins • Plugin triggers are commonly used to enable/disable functionality
in a log via the events plugin (dokku events:on) • Core plugins can be swapped out for alternative implementations (dokku-scheduler-kubernetes) • Core plugins generally are not disabled, but extended via triggers • Alternative implementations may result in a user disabling a core plugin. A good example would be an alternative proxy implementation around caddy or haproxy • Core plugins provide helper functions for community plugins to utilize (conﬁg/property management)
whether the buildpack applies • bin/compile: compiles dependencies and app code • bin/release: speciﬁes default processes and platform addons for the app • Generally also includes supporting building code • Normally only targets a single language
ofﬁcial buildpacks, any other unofﬁcial ones • CloudFoundry maintains their own v2b buildpacks • Hundreds of Community Buildpacks • Dokku project maintains the static buildpack • Plotly maintains custom Conda, Python, and R buildpacks • Maybe you?
are installed on “stacks”, or base operating systems • gliderlabs/herokuish provides a docker- compatible layer for buildpacks (Dokku uses this) • Buildpack v3 will have ofﬁcially provided stacks with built-in buildpacks
A binary that can act as an init for apps • A docker image • Released in a single stack ﬂavor (currently heroku-18) • Can be further customized for dokku via DOKKU_IMAGE environment variable • Will be deprecated in favor of buildpack v3 pack-cli or equivalent
Binary • R/Python: Source + dependencies • Export Formats • Cloud Foundry: Exports a tarball “droplet” of /app directory • Heroku: Exports a tarball “slug" of /app directory • Dokku: Provides a docker image of all changes • Dokku is the odd one here
$IMAGE… • Kubernetes support via plugin • Depends on the (external but ofﬁcial) registry plugin and an external registry • Supports both process and resource scaling • Currently requires a custom routing implementation • Nomad support via plugin • Similar requirements as Kubernetes support • Others?
for storage • All common datastores are currently implemented as OSS plugins (Elasticsearch, MySQL, Postgres, Redis, etc.) • Upcoming Couchbase, Neo4j, Omnisci • Old docker --link ﬂag used for communication • Sometimes does not withstand datastore restarts • Upcoming release will switch to non-deprecated networks • No clustering support on OSS datastores plugins • No resource limits speciﬁc on OSS datastores plugins • Should there be?
through redis) • plain old http over open network • Commonly requested OSS feature (two apps talking to each other) • Maybe not as requested in commercial settings? • Upcoming release will implement ofﬁcial networking support • Strictly opt-in, but not multi-tenant
so should the apps • No special docker0 rules in place to block inbound/outbound • This applies to IAM roles and other things living on the server • Outgoing access can be controlled via iptables • Many security softwares can break docker0, causing invalid networking for docker containers! • Other access control should be done via password or ﬁrewall