Slide 1

Slide 1 text

Devious Schemes Adventures in distributed computing with PostgreSQL and Scheme lightning talk | pgcon 2018 Thomas Munro

Slide 2

Slide 2 text

What problem could I actually solve with PL/Scheme anyway…? We have
 PL/Scheme!? commit 2440c442d167d9d081a3e69c4fa78f3b6f8932e9 Author: Peter Eisentraut Date: Mon Jun 12 22:34:04 2017 -0400 doc: Update external PL list Add PL/Lua, PL/v8. Remove stale/unmaintained PL/PHP, PL/Py, PL/Ruby, PL/Scheme. Reported-by: Adam Sah We lost PL/Scheme!?

Slide 3

Slide 3 text

http 1 2 3 code

Slide 4

Slide 4 text

Move code to data using alien technology http 1 2 3 code Wormhole in space-time continuum

Slide 5

Slide 5 text

PostgreSQL Global Development Group Scheme • Minimalist dialect of the language in which the Gods wrought the universe* • Easy to rewrite code, sling code around as data • A “wormhole” system for other languages should be doable too, but that’d probably take more than one flight to Canada to prototype

Slide 6

Slide 6 text

Code push-down (remote (user-id) (string-append “Hello there, ” (query-single “SELECT …” user-id) “. Today’s top story is: ” (query-single “SELECT …”))) • Blocks of code transferred to run inside the database, compiled and cached for reuse • Named variables, return values, exceptions transferred (remote (x y) (query “UPDATE foo SET x = x + $1” x) (query “UPDATE bar SET x = x + $1” y)
 (query “UPDATE baz SET x = x + $1” (+ x y)))

Slide 7

Slide 7 text

Declarative transactions (with-transaction
 (execute! “INSERT INTO message VALUES ($1, $2, $3)” section title body) (execute! “UPDATE section_summary SET num_messages = num_messages + 1 WHERE section = $1” section)) • Automatic retry on retryable errors (serialization failure, deadlock, read- only standby, …) • Automatic connection pooling, routing and learning about read-only vs writable transactions

Slide 8

Slide 8 text

Combination = single round-trip (define (main-page user-id)
 (render-html “index.html”
 (with-remote-transaction (user-id)
 (list :latest-news (query “SELECT …” user-id) :new-messages (query “SELECT …” user-id) :last-login (query “SELECT …” user-id)))))

Slide 9

Slide 9 text

Security Nope

Slide 10

Slide 10 text

Assorted thoughts • Experimental half-baked hack, do not try this at work • Variables declared immutable can be cached in DB • Security model should probably be based on Scheme modules, perhaps controlled by a PGC_SUSET GUC? • Smarter code analysis -> implicit and/or adaptive code migration? • You could probably do this with other Lisps quite easily • You could probably do this with Python, Ruby, Java, … quite difficultly (source? AST? byte-code? …)

Slide 11

Slide 11 text

Images borrowed from wikipedia.org, wikimedia.org, twitter.com, xkcd.com github.com/macdice/wormhole github.com/macdice/plscheme