Devious Schemes

"Adventures in distributed programming with Scheme and PostgreSQL". A 5 minute lightning talk I gave at PGCon 2018 in Ottawa. This describes some hobbyist hacking. The code describes is very incomplete and a work in progress...

Transcript

1. Devious Schemes
   Adventures in distributed computing
   with PostgreSQL and Scheme
   lightning talk | pgcon 2018
   
   Thomas Munro
   

   View Slide

2. What problem
   could I actually solve
   with PL/Scheme
   anyway…?
   We have

   PL/Scheme!?
   commit 2440c442d167d9d081a3e69c4fa78f3b6f8932e9
   Author: Peter Eisentraut
   Date: Mon Jun 12 22:34:04 2017 -0400
   doc: Update external PL list
   Add PL/Lua, PL/v8.
   Remove stale/unmaintained PL/PHP, PL/Py, PL/Ruby, PL/Scheme.
   Reported-by: Adam Sah
   We lost
   PL/Scheme!?
   

   View Slide

3. http
   1
   2
   3
   code
   

   View Slide

4. Move code to data using
   alien technology
   http
   1
   2
   3
   code
   Wormhole in space-time continuum
   

   View Slide

5. PostgreSQL Global Development Group
   Scheme
   • Minimalist dialect of the language in which the
   Gods wrought the universe*
   
   • Easy to rewrite code, sling code around as data
   
   • A “wormhole” system for other languages should
   be doable too, but that’d probably take more than
   one flight to Canada to prototype
   

   View Slide

6. Code push-down
   (remote (user-id)
   (string-append “Hello there, ”
   (query-single “SELECT …” user-id)
   “. Today’s top story is: ”
   (query-single “SELECT …”)))
   • Blocks of code transferred to run inside the
   database, compiled and cached for reuse
   
   • Named variables, return values, exceptions
   transferred
   (remote (x y)
   (query “UPDATE foo SET x = x + $1” x)
   (query “UPDATE bar SET x = x + $1” y)

   (query “UPDATE baz SET x = x + $1” (+ x y)))
   

   View Slide

7. Declarative transactions
   (with-transaction

   (execute! “INSERT INTO message VALUES ($1, $2, $3)”
   section
   title
   body)
   (execute! “UPDATE section_summary
   SET num_messages = num_messages + 1
   WHERE section = $1”
   section))
   • Automatic retry on retryable errors (serialization failure, deadlock, read-
   only standby, …)
   
   • Automatic connection pooling, routing and learning about read-only vs
   writable transactions
   

   View Slide

8. Combination = single round-trip
   (define (main-page user-id)

   (render-html
   “index.html”

   (with-remote-transaction (user-id)

   (list :latest-news (query “SELECT …” user-id)
   :new-messages (query “SELECT …” user-id)
   :last-login (query “SELECT …” user-id)))))
   

   View Slide

9. Security
   Nope
   

   View Slide

10. Assorted thoughts
    • Experimental half-baked hack, do not try this at work
    
    • Variables declared immutable can be cached in DB
    
    • Security model should probably be based on Scheme modules,
    perhaps controlled by a PGC_SUSET GUC?
    
    • Smarter code analysis -> implicit and/or adaptive code migration?
    
    • You could probably do this with other Lisps quite easily
    
    • You could probably do this with Python, Ruby, Java, … quite
    difficultly (source? AST? byte-code? …)
    

    View Slide

11. 
    Images borrowed from wikipedia.org, wikimedia.org, twitter.com, xkcd.com
    github.com/macdice/wormhole
    github.com/macdice/plscheme
    

    View Slide