APIs at Matt Cottingham http://microco.sm matt@microcosm.cc @mattrco Image credit: Renée French

Platform overview ● Hosted platform for community forums ● API is 18k lines of Go code ● API documentation: http: //microcosm-cc.github.io ● Alpha site: http://meta.microco.sm Microco.sm

Engineering overview Requirements ● API does everything: all clients (including ours) built on it ● Performance constraints (sub 50ms response times) ● Runtime configurable ● Scheduled tasks ● Up-front schema design ● Pluggable authentication ● Websocket support API DATA CLIENT AUTH

Why Go? Plusses Potential problems? ● Third-party packages (e.g. oAuth, database support) ● How much will change around 1.1? ● Who's using it? Will we be able to hire Go programmers? ● Type system ● Powerful standard library ● Small core language, easy to learn ● Concurrency primitives (this is how things looked before I knew much about Go, i. e. six months ago)

Implementation Stack ● PostgreSQL 9.3 ● Go 1.1 (we were following tip up to 1.1. We'll upgrade to 1.1.1 shortly) ● Memcached ● Nginx (TLS, SPDY, gzip, etc.)

Implementation ● Request processor inspired by goweb ● gorilla/mux: request routing ● lib/pq: pure Go PostgreSQL driver Go packages we use ● Memcached client: bradfitz/gomemcache ● Cron: robfig/cron ● Configuration: laurent22/toml-go

Example route declarations Implementation

Implementation Simple request Handler (switch on HTTP method)

Implementation Basic controller

Implementation Persistence ● What about struct-to-database-table mappings? ● Several ORM options, e.g. ● https://github.com/coopernurse/gorp/ ● https://github.com/eaigner/hood ● but with lib/pq we just talk to PostgreSQL through prepared statements (more maintenance but finer query control)

Implementation Storing data (receiver enables `m.Insert()` syntax)

Implementation Caveats/compromises ● Documented examples are quite sparse for database access ● database/sql could not (and should not) model all column datatypes, so some are provided by lib/pq ● Dealing with SQL NULL is quite simple, `thing.Valid` idiom:

S3 file storage ● We use the package from Juju (by Canonical) ● There are lots of packages, well worth reading the source: ● https://launchpad.net/juju-core ● Go and Juju at Canonical Implementation

Deployment ● `scp binary server` (yep...) ● upstart to manage processes (may move to supervisord) ● No graceful restarts. There's some work going on for this, but we'll likely just remove nodes from the load balancer, restart, add them back in. Systems

Systems Logging and monitoring ● stdlib logging package does syslog or user-defined formats ● fine for capturing panics etc. ● riemann or heka are possibilities for application monitoring ● must accept a payload over UDP, not Go specific

Flotsame & Jetsam Performance ● Web framework benchmarks (uses Go 1.1) ● Benchmark source ● Go compares favourably in tests that look realistic ● We haven't (yet) tested memory usage at e.g. high concurrency levels

Reading ● the official Go blog, Dave Cheney's blog, and go-nuts ● Go at Apcera ● Russ Cox on regex implementation is an interesting read ● Go Newsletter (disclaimer: by me!) Close