Devious Schemes

"Adventures in distributed programming with Scheme and PostgreSQL". A 5 minute lightning talk I gave at PGCon 2018 in Ottawa. This describes some hobbyist hacking. The code describes is very incomplete and a work in progress...

Transcript

1. Devious Schemes Adventures in distributed computing with PostgreSQL and Scheme lightning talk | pgcon 2018 Thomas Munro
2. What problem could I actually solve with PL/Scheme anyway…? We have
 PL/Scheme!? commit 2440c442d167d9d081a3e69c4fa78f3b6f8932e9 Author: Peter Eisentraut <peter_e@gmx.net> Date: Mon Jun 12 22:34:04 2017 -0400 doc: Update external PL list Add PL/Lua, PL/v8. Remove stale/unmaintained PL/PHP, PL/Py, PL/Ruby, PL/Scheme. Reported-by: Adam Sah <asah@midgard.net> We lost PL/Scheme!?
3. http 1 2 3 code
4. Move code to data using alien technology http 1 2 3 code Wormhole in space-time continuum
5. PostgreSQL Global Development Group Scheme • Minimalist dialect of the language in which the Gods wrought the universe* • Easy to rewrite code, sling code around as data • A “wormhole” system for other languages should be doable too, but that’d probably take more than one flight to Canada to prototype
6. Code push-down (remote (user-id) (string-append “Hello there, ” (query-single “SELECT …” user-id) “. Today’s top story is: ” (query-single “SELECT …”))) • Blocks of code transferred to run inside the database, compiled and cached for reuse • Named variables, return values, exceptions transferred (remote (x y) (query “UPDATE foo SET x = x + $1” x) (query “UPDATE bar SET x = x + $1” y)
 (query “UPDATE baz SET x = x + $1” (+ x y)))
7. Declarative transactions (with-transaction
 (execute! “INSERT INTO message VALUES ($1, $2, $3)” section title body) (execute! “UPDATE section_summary SET num_messages = num_messages + 1 WHERE section = $1” section)) • Automatic retry on retryable errors (serialization failure, deadlock, read- only standby, …) • Automatic connection pooling, routing and learning about read-only vs writable transactions
8. Combination = single round-trip (define (main-page user-id)
 (render-html “index.html”
 (with-remote-transaction (user-id)
 (list :latest-news (query “SELECT …” user-id) :new-messages (query “SELECT …” user-id) :last-login (query “SELECT …” user-id)))))
9. Security Nope
10. Assorted thoughts • Experimental half-baked hack, do not try this at work • Variables declared immutable can be cached in DB • Security model should probably be based on Scheme modules, perhaps controlled by a PGC_SUSET GUC? • Smarter code analysis -> implicit and/or adaptive code migration? • You could probably do this with other Lisps quite easily • You could probably do this with Python, Ruby, Java, … quite difficultly (source? AST? byte-code? …)
11. <EOF> Images borrowed from wikipedia.org, wikimedia.org, twitter.com, xkcd.com github.com/macdice/wormhole github.com/macdice/plscheme