Sam Tobin-Hochstadt on Composable and Compilable Macros

"Composable and Compilable Macros" introduces the Racket module system, which addresses the following problem: When you have macros that run programs at compile-time, how does this interact with separate compilation and ahead-of-time compilation. The paper introduces "phases", which enable Racket to behave the same regardless of when and how you compile your program. It also introduces the idea of writing different modules in different languages, which is now used for systems like Typed Racket.

A few related papers:

- Syntactic Abstraction in Scheme, Dybvig Hieb & Bruggeman, Lisp andSymbolic Computation 1993

- Macros that Work Together, Flatt, Findler, Culpepper, & Darais, JFP 2012- Submodules in Racket, Flatt GPCE 2013

- Implicit phasing for R6RS libraries, Ghuloum & Dybvig, ICFP 2007

- Extending the scope of syntactic abstraction, Waddell & Dybvig, POPL 1999

Transcript

1. Matthew Flatt ICFP 2002 Sam Tobin-Hochstadt Papers �e Love NYC Composable & Compilable Macros You want it when?
2. Prehistory • Scheme No standard module system Implementations used Common Lisp approach • Macros Hygenic macros 15 years old Still considered esoteric • PLT Scheme Mostly designed for teaching IDE with multiple language levels
3. Composable and Compilable Macros Matthew Flatt University of Utah
4. A Scheme Programmer...
5. ... Programming in Scheme
6. ... in Macro-Extended Scheme!
7. ... Trying to Use a Compiler
8. ... Accomodating the Compiler
9. ... Trying a Different Compiler
10. ... Trying a Complex Library
11. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... m (load "ma.scm") (load "mb.scm") a b u ma mb = load effect
12. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... m (load "ma.scm") (load "mb.scm") a b u ma mb = dependency
13. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... m (load "ma.scm") (load "mb.scm") a b u ma mb
14. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... m (load "ma.scm") (load "mb.scm") a b u ma mb never loaded in compile mode
15. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... (load "u.scm") (load "ma.scm") (load "mb.scm") a b u ma mb
16. The Trouble with Scripting the Compiler (load "u.scm") (eval-when (compile load) (load "m.scm")) (load "a.scm") (load "b.scm") ... (load "u.scm") (load "ma.scm") (load "mb.scm") a b u ma mb loaded twice in interactive mode
17. Problem Solution • Declarative • Consistent success/failure in all compilation modes • Supports lexically scoped macros macro-defining macros Example Experience
18. Modules and Macros (module a (define-syntax m ...) (define f ...) ...)
19. Modules and Macros = run-time expressions (module a (define-syntax m ...) (define f ... ...) ... ...)
20. Modules and Macros = compile-time expressions (module a (define-syntax m ... ...) (define f ... ...) ... ...)
21. Modules and Macros #' in escapes back to (module a (define-syntax m ... ...) (define f ... ...) ... ...) #'(f ...) (f ...)
22. Modules and Macros (module a (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (define-syntax n ...) (define g ...) ...) #'(f ...) (f ...)
23. Modules and Macros (module a (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (define-syntax n ...) (define g ... ...) ... ...) #'(f ...) (f ...)
24. Modules and Macros (module a (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (define-syntax n ... ...) (define g ... ...) ... ...) #'(f ...) (f ...)
25. Modules and Macros (module u (define fold ...) ...) (module a (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (require-for-syntax "u.scm") (define-syntax n ... ...) (define g ... ...) ... ...) #'(f ...) (f ...)
26. Modules and Macros (module u (define fold ...) ...) (module a (require "u.scm") (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (require-for-syntax "u.scm") (define-syntax n ... ...) (define g ... ...) ... ...)
27. Modules and Macros (module u (define fold ...) ...) (module u (define fold ...) ...) (module a (require "u.scm") (define-syntax m ... ...) (define f ... ...) ... ...) (module b (require "a.scm") (require-for-syntax "u.scm") (define-syntax n ... ...) (define g ... ...) ... ...) different instance for each phase
28. Problem Solution Example • Record matching with static checks Experience
29. Records and Matching (define-record zebra (weight stripes))
30. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance
31. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance (define-record lizard (weight length color)) (lizard 2 5 'green) ; = a lizard instance
32. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance (define-record lizard (weight length color)) (lizard 2 5 'green) ; = a lizard instance (record-switch ... ((zebra w s) ...) ((lizard w l c) ...))
33. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance (define-record lizard (weight length color)) (lizard 2 5 'green) ; = a lizard instance (define (animal-weight a) (record-switch a ((zebra w s) w) ((lizard w l c) w))) (animal-weight (zebra 500 24)) ; = 500 (animal-weight (lizard 2 5 'green)) ; = 2
34. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance (define-record lizard (weight length color)) (lizard 2 5 'green) ; = a lizard instance (define (animal-weight a) (record-switch a ((zebra w s c) w) syntax error ((lizard w l c) w)))
35. Records and Matching (define-record zebra (weight stripes)) (zebra 500 24) ; = a zebra instance (define-record lizard (weight length color)) (lizard 2 5 'green) ; = a lizard instance (define (animal-weight a) (record-switch a ((zebra w s c) w) syntax error ((lizard w l c) w))) compile-time communication across module boundaries
36. Implementing Records (module record (define-syntax define-record ...) (define-syntax record-switch ...))
37. Implementing Records (module record (define-syntax define-record ...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...))
38. Implementing Records (module record (define-syntax define-record ...#'(define name (mkrec ...))...) (define-syntax record-switch ...#'(is-a name ...) ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...))
39. Implementing Records (module record (define-syntax define-record ...#'(define name (mkrec ...))...) (define-syntax record-switch ...#'(is-a name ...) ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) expands to (define zebra (mkrec ...))
40. Implementing Records (module record (define-syntax define-record ...#'(define name (mkrec ...))...) (define-syntax record-switch ...#'(is-a name ...) ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) expands to (define zebra (mkrec ...)) expands to (is-a zebra ...)
41. Implementing Records (module record (require "rt.scm") (define-syntax define-record ...#'(define name (mkrec mkrec ...))...) (define-syntax record-switch ...#'(is-a is-a name ...) ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) expands to (define zebra (mkrec mkrec ...)) expands to (is-a is-a zebra ...)
42. Implementing Records (module record (require "rt.scm") (define-syntax define-record ... table ...) (define-syntax record-switch ... table ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...))
43. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ... table table ...) (define-syntax record-switch ... table table ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...)
44. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ... table table ...) (define-syntax record-switch ... table table ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) adds zebra to table while expanding
45. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ... table table ...) (define-syntax record-switch ... table table ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) adds zebra to table while expanding finds zebra in table while expanding
46. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...)
47. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) (module client (require "zoo.scm") (record-switch ... ((zebra w s) ...)))
48. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) (module client (require "zoo.scm") (record-switch ... ((zebra w s) ...))) cannot find zebra in table
49. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...#'(for-syntax ...table table...)...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) (module client (require "zoo.scm") (record-switch ... ((zebra w s) ...)))
50. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...#'(for-syntax ...table table...)...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) (module client (require "zoo.scm") (record-switch ... ((zebra w s) ...))) now expands to (for-syntax ...table table...) (define zebra (mkrec ...))
51. Implementing Records (module record (require "rt.scm") (require-for-syntax "ct.scm") (define-syntax define-record ...#'(for-syntax ...table table...)...) (define-syntax record-switch ...)) (module zoo (require "record.scm") (define-record zebra (wgt sc)) (record-switch ... ((zebra w s) ...)) (module rt (define mkrec ...) (define is-a ...)) (module ct (define table ...) ...) (module client (require "zoo.scm") (record-switch ... ((zebra w s) ...))) now expands to (for-syntax ...table table...) (define zebra (mkrec ...)) finds zebra in table
52. Problem Solution Example Experience • The many languages & compilers of PLT Scheme
53. PLT Scheme
54. PLT Scheme DrScheme supports multiple languages that share libraries
55. PLT Scheme
56. PLT Scheme One compilation mode
57. PLT Scheme Another compilation mode
58. PLT Scheme Yet another compilation mode
59. PLT Scheme
60. �hat happened after • Modules Became the basis for Scheme standards Influenced other module systems • Macros syntax-parse, more phases, debugging Tons of features • Languages Typed Racket, datalog, Scribble
61. Other papers to read
62. Other papers to read
63. Other papers to read
64. Other papers to read