Wercker on CoreOS - Speaker Deck

Slide 1

Slide 1 text

wercker

Slide 2

Slide 2 text

wercker on CoreOS http://wercker.com @wercker

Slide 3

Slide 3 text

Wercker: automation driven development • Wercker is an automation platform • ci/cd not ambitious enough • continuous is assumed

Slide 4

Slide 4 text

Interface • wercker.yml • Describe job environment • Describe a work pipeline • Describe required services

Slide 5

Slide 5 text

Interface • wercker.yml • Describe job environment • Describe a work pipeline • Describe required services box: tcnksm/gox@0.1.0

Slide 6

Slide 6 text

Interface • wercker.yml • Describe job environment • Describe a work pipeline • Describe required services steps: -‐ script: name: install code: go get -‐v -‐d ./... -‐ golint

Slide 7

Slide 7 text

Interface • wercker.yml • Describe job environment • Describe a work pipeline • Describe required services services: -‐ mongodb

Slide 8

Slide 8 text

Output • Pipeline outputs container image • Hugely beneﬁcial • Immutable, tested object linked to commit • Repeatability

Slide 9

Slide 9 text

Under the hood • Standalone process dispatches work to ﬂeet • Called “kiddie-pool”

Slide 10

Slide 10 text

mhook • Needed simple package manager • downloads packages from S3 • indexed by project, branch, and commit • similar to Kelsey Hightower’s `pm` https://github.com/wercker/mhook

Slide 11

Slide 11 text

kiddie-pool uses mhook [Unit] Description=Wercker Kiddie-‐Pool %i After=install-‐mhook.service [Service] ExecStartPre=/tmp/mhook -‐-‐bucket \\ —project kiddie-‐pool \\ —branch master \\ —commit latest \\ /opt/bin/kiddie-‐pool-‐%i ExecStartPre=/usr/bin/chmod +x /tmp/kiddie-‐pool-‐%i ... [Install] WantedBy=multi-‐user.target [X-‐Fleet] Global=true

Slide 12

Slide 12 text

kiddie-pool • Sits between work queue and ﬂeet • Constructs Unit from job metadata • Unit invokes `wercker` cli tool • Sends to ﬂeet over http API

Slide 13

Slide 13 text

wercker cli • Understands how to actually run the job • You can download and run wercker locally • rapid iteration • don’t even need us • wow http://wercker.com/downloads/

Slide 14

Slide 14 text

What’s next • Smart job assignment • Fleet naively allocates work • Need a job scheduler that’s smarter • Colocate jobs, data, and services • Help us build this

Slide 15

Slide 15 text

What’s next • mhook • Push behavior • Signing packages • Wercker step

Slide 16

Slide 16 text

Logging • Logging • Use Graylog because they have a working docker image • `journalhook` sends application data to journal • `journal2gelf` container streams journal to graylog • persists journal cursor https://github.com/wercker/journalhook

Slide 17

Slide 17 text

Provisioning • Vagrant for development • AWS/Cloud Formation for user-facing • Single script to send application units to ﬂeet • Separation of concerns, no duplication

Slide 18

Slide 18 text

etcd • Used to publish service information • In the application to signal job aborts • Best way to run is as cluster of its own • should get easier with 2.0 with proxies • need to conﬁgure many things to have correct etcd url

Slide 19

Slide 19 text

References 1. http://wercker.com/downloads/ 2. https://github.com/wercker/mhook 3. https://github.com/wercker/journalhook 4. https://github.com/systemd/journal2gelf 5. https://registry.hub.docker.com/u/mwhooker/ journal2gelf/ 6. https://speakerdeck.com/mwhooker/wercker-on- coreos

Slide 20

Slide 20 text

Thank you Matt Hooker wercker Software Engineer matt@wercker.com @mwhooker http://wercker.com http://twitter.com/wercker Grab a shirt and/or sticker! wercker