Upgrade to Pro — share decks privately, control downloads, hide ads and more …

nREPL Redux (DCD 2019)

nREPL Redux (DCD 2019)

Slide-deck from my presentation at Dutch Clojure Days 2019.

Bozhidar Batsov

April 06, 2019
Tweet

More Decks by Bozhidar Batsov

Other Decks in Programming

Transcript

  1. M-x

    View Slide

  2. M-x greet-audience

    View Slide

  3. Божидар

    View Slide

  4. Bug
    cool

    View Slide

  5. View Slide

  6. Sofia, Bulgaria
    Sofia, Bulgaria

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. We’re not hiring!

    View Slide

  11. View Slide

  12. @bbatsov

    View Slide

  13. metaredux.com

    View Slide

  14. emacsredux.com

    View Slide

  15. View Slide

  16. slide intentionally
    left blank

    View Slide

  17. nREPL
    REdux

    View Slide

  18. Dictionary Definitions

    View Slide

  19. nREPL

    View Slide

  20. network REPL

    View Slide

  21. redux

    View Slide

  22. adjective
    1. brought back, revived
    Emacs era redux
    2. (usually postpositive) (esp of
    an artistic work) presented in a new way
    Apocalypse Now Redux

    View Slide

  23. nREPL Redux?

    View Slide

  24. nREPL is legacy

    View Slide

  25. nREPL is complex

    View Slide

  26. (nREPL) middleware sucks

    View Slide

  27. Rich hates nREPL

    View Slide

  28. The Socket REPL is the future

    View Slide

  29. unrepl is the future

    View Slide

  30. prepl is the future

    View Slide

  31. LSP is the future

    View Slide

  32. notorious REPL

    View Slide

  33. nice REPL

    View Slide

  34. neo REPL

    View Slide

  35. noble REPL

    View Slide

  36. novel REPL

    View Slide

  37. mREPL

    View Slide

  38. mega REPL

    View Slide

  39. mighty REPL

    View Slide

  40. magic REPL

    View Slide

  41. magnificent REPL

    View Slide

  42. marvellous REPL

    View Slide

  43. majestic REPL

    View Slide

  44. Naming is hard!

    View Slide

  45. nREPL
    REdux

    View Slide

  46. nREPL is very
    actively maintained

    View Slide

  47. nREPL is not complex

    View Slide

  48. Simple by design

    View Slide

  49. nREPL is
    superbly documented

    View Slide

  50. View Slide

  51. Middleware is a
    very powerful concept

    View Slide

  52. A huge ecosystem of tools
    exists around nREPL

    View Slide

  53. lein repl

    View Slide

  54. boot repl

    View Slide

  55. vim-fireplace

    View Slide

  56. vim-iced

    View Slide

  57. acid.vim

    View Slide

  58. Calva (VS Code)

    View Slide

  59. Proto REPL (Atom)

    View Slide

  60. Cursive

    View Slide

  61. View Slide

  62. A REPL for (Great) Good

    View Slide

  63. 2010

    View Slide

  64. vim-clojure
    custom
    SLIME
    swank
    Enclojure
    custom
    CCW
    custom

    View Slide

  65. vim-clojure
    nREPL
    SLIME
    nREPL
    Enclojure
    nREPL
    CCW
    nREPL

    View Slide

  66. Project Goals
    •Common foundation for Clojure tooling
    •Trivial to embed
    •Zero runtime dependencies
    •Extremely simple wire format
    •Easy to extend with additional functionality

    View Slide

  67. vim-fireplace
    nREPL
    CIDER
    nREPL
    Cursive
    nREPL
    CCW
    nREPL
    vim-iced
    nREPL
    Calva
    nREPL
    acid.vim
    nREPL
    Proto REPL
    nREPL

    View Slide

  68. View Slide

  69. Clojure Contrib

    View Slide

  70. org.clojure/tools.nrepl

    View Slide

  71. •Contributor Agreement
    •No Pull Requests
    •JIRA for issue tracking
    •Hudson CI
    •Maven
    •Artifacts are deployed to Maven Central

    View Slide

  72. View Slide

  73. 010101010101010110

    View Slide

  74. The (REPL) Wasteland

    View Slide

  75. github.com/clojure/tools.nrepl
    Repo deprecation announcement

    View Slide

  76. View Slide

  77. 17 contributors in 8 years

    View Slide

  78. Almost no activity since 2015

    View Slide

  79. 25%
    5%
    70%
    Yes Maybe
    I already have Notepad No
    Do you want great Clojure development tools?

    View Slide

  80. 69%
    25%
    5%
    1%
    Yes Maybe
    I already have Notepad No
    Are you willing to work on Clojure development tools?

    View Slide

  81. SLIME -> CIDER

    View Slide

  82. CIDER loves nREPL

    View Slide

  83. View Slide

  84. View Slide

  85. Rogue One

    View Slide

  86. Challenges

    View Slide

  87. I didn’t really know what I
    had signed up for

    View Slide

  88. Breaking changes to one of the
    most widely used tools in the
    Clojure ecosystem

    View Slide

  89. org.clojure/tools.nrepl
    ->
    nrepl/nrepl

    View Slide

  90. clojure.tools.nrepl.*
    ->
    nrepl.*

    View Slide

  91. A ton of 3rd party tools and
    libraries needed to be updated

    View Slide

  92. •lein
    •boot
    •figwheel
    •shadow-cljs
    •REPLy
    •Ultra
    •All nREPL extensions in existence…

    View Slide

  93. •piggieback
    •cider-nrepl
    •refactor-nrepl
    •sayid
    •drawbridge
    •and many others…

    View Slide

  94. Migration Strategy
    • Teach most extensions about both versions of nREPL
    • Migrate tools like Lein and Boot to the new nREPL
    • Provide ways for users of older Boot/Lein to run the new nREPL
    • Drop support for tools.nrepl from the extensions
    • Document nREPL extensively in the process
    • Focus on actual nREPL improvements

    View Slide

  95. Building a community
    around the project

    View Slide

  96. A Catalyst for Change

    View Slide

  97. REPL (R)Evolution

    View Slide

  98. •nREPL 0.3 (initial compatibility release)
    •nREPL 0.4 (multiple feature releases)
    •nREPL 0.5
    •nREPL 0.6

    View Slide

  99. Better Versioning Policy

    View Slide

  100. 0.feature.patch

    View Slide

  101. Then
    •0.2.5 (Feature)
    •0.2.6 (Feature)
    •0.2.7 (Bug fix)
    •0.2.8 (Feature)

    View Slide

  102. Now
    •0.5.0 (Feature)
    •0.5.1 (Bug fix)
    •0.5.2 (Bug fix)
    •0.6.0 (Feature)

    View Slide

  103. Clear Compatibility Policy
    • All supported LTS JDKs (Currently Java 8 and Java 11)
    • Latest rapid cycle JDK (Currently Java 12)
    • Clojure 1.7+
    • Extreme dedication to protocol compatibility
    • Surgical changes at the implementation-level

    View Slide

  104. Fixed
    many long-standing bugs

    View Slide

  105. Complete overhaul of the
    internals

    View Slide

  106. •Simpler code evaluation model
    •Robust handling of Clojure’s special variables
    •Massively improved value printing logic
    •Better way to hook into evaluation errors
    •Global/project server configuration via configuration
    files
    •Reusable command-line API

    View Slide

  107. Exhaustive documentation
    (nrepl.org)

    View Slide

  108. Great integration with
    tools.deps

    View Slide

  109. Transit transport
    (https://github.com/nrepl/fastlane)

    View Slide

  110. EDN transport
    (coming in nREPL 0.7)

    View Slide

  111. Support for hot-loading
    libraries in a running REPL

    View Slide

  112. No breaking changes to the
    protocol

    View Slide

  113. Minimal breaking changes on
    the API level

    View Slide

  114. Team-work FTW

    View Slide

  115. Chas creates the new repo
    Rogue One

    View Slide

  116. View Slide

  117. 23 new contributors!

    View Slide

  118. Christophe Grand

    View Slide

  119. Michael Griffiths

    View Slide

  120. View Slide

  121. View Slide

  122. Monopoly is good

    View Slide

  123. Monopoly allows you to
    focus on innovation

    View Slide

  124. Meet The Competition

    View Slide

  125. The Socket REPL

    View Slide

  126. •bundled with Clojure & ClojureScript
    •streaming REPL
    •a socket based version of the default REPL
    •easy if you want to evaluate something in a remote app
    •can be started with a JVM property
    •hard to build tools on it

    View Slide

  127. $ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/repl}”
    $ nc localhost 5555
    user=> (+ 10 10)
    20
    user=> (println "Hello, world!")
    Hello, world!
    nil

    View Slide

  128. prepl

    View Slide

  129. $ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/io-prepl}”
    $ nc localhost 5555
    (+ 10 10)
    {:tag :ret, :val "20", :ns "user", :ms 23, :form "(+ 10 10)”}
    (println "Hello, world")
    {:tag :out, :val "Hello, world\n"}
    {:tag :ret, :val "nil", :ns "user", :ms 8, :form "(println \"Hello, world\")"}

    View Slide

  130. unrepl

    View Slide

  131. •based on the socket REPL
    •allows clients to customize the REPL
    •has a sophisticated rich data printer
    •streaming input, framed EDN output

    View Slide

  132. Language Server Protocol
    • Language agnostic
    • Doesn’t rely on a running REPL
    • A much broader protocol (completions, code lookup, etc)
    • Usually is implemented in terms of static analysis
    • Message oriented, JSON data format
    • Extensible protocol
    • Controlled by an evil corporation turned good

    View Slide

  133. View Slide

  134. Swiss army knife of REPLs

    View Slide

  135. • designed to power programmings tools
    • message oriented
    • can be extended with arbitrary operations
    • interruptible evals
    • session multiplexing
    • supports arbitrary communication protocol/data format
    • async in nature

    View Slide

  136. ;; request
    {:id "1"
    :op "eval"
    :code "(+ 1 1)”
    :ns "user"}
    ;; response
    {:id "1"
    :value "2"}

    View Slide

  137. ;; request
    {:id "1"
    :op "complete"
    :thing "map”
    :ns "user"}
    ;; response
    {:id "1"
    :candidates […]}

    View Slide

  138. ;; raw request
    {:id 1 :op "eval" :code "(+ 1 1)"}
    ;; encoded request
    “d4:code7:(+ 1 1)2:idi1e2:op4:evale"

    View Slide

  139. “d4:code7:(+ 1 1)2:idi1e2:op4:evale"
    •d (dictionary)
    •4:code - a 4 byte string “code”
    •7:(+ 1 1) - a 7 byte string “(+ 1 1)”
    •2:id - a 2 byte string “id”
    •i1e - an integer 1
    •…
    •e - end of dictionary

    View Slide

  140. Community Driven

    View Slide

  141. Open Source is not about You

    View Slide

  142. nREPL is
    all about YOU!!!

    View Slide

  143. Beyond Clojure

    View Slide

  144. nREPL as a protocol

    View Slide

  145. Simple by design

    View Slide

  146. Key Operations
    •eval
    •load-file
    •interrupt
    •stdin

    View Slide

  147. Bencode Data Types
    •strings
    •integers
    •lists
    •maps

    View Slide

  148. View Slide

  149. A light-weight alternative to
    LSP

    View Slide

  150. Easy to implement

    View Slide

  151. nREPL implementations
    •HyREPL - an nREPL for the Hy programming language
    •JeeJah - an nREPL server for Fennel and Lua
    •Ogion - an nREPL server for Racket
    •Chicken NREPL - an nREPL server for Chicken Scheme
    •cl-nrepl - an nREPL server for Common Lisp
    •cljs-noderepl - an nREPL server for ClojureScript running on Node.js
    •R-nREPL - an nREPL server for R
    •nREPL CLR - an nREPL server for ClojureCLR
    •Arcadia nREPL - an nREPL server implemented in C# for Arcadia.

    View Slide

  152. Dreams for the future
    • nREPL implemented in ClojureScript/ClojureCLR
    • nREPL for Erlang/Elixir
    • nREPL for Smalltalk
    • Better behaved nREPL clients (ones that don’t make assumptions
    they are dealing with a Clojure server)

    View Slide

  153. (next nrepl)

    View Slide

  154. Roadmap for nREPL 0.7
    •EDN Transport (done)
    •Client-side resource/class injection API (unrepl)
    •Upgrade socket REPL to nREPL (unrepl)
    •Rich data printer (unrepl)

    View Slide

  155. Personal Impact

    View Slide

  156. Extremely proud

    View Slide

  157. View Slide

  158. Frustrated

    View Slide

  159. Tired

    View Slide

  160. Burned out

    View Slide

  161. View Slide

  162. View Slide

  163. View Slide

  164. Sustainable

    View Slide

  165. Community Funded?

    View Slide

  166. Clojure’s Orchard

    View Slide

  167. $4000

    View Slide

  168. $5400

    View Slide

  169. We need to get Clojure companies
    invested
    in the Clojure tooling ecosystem

    View Slide

  170. 20 companies paying
    $500/month can make a
    massive difference

    View Slide

  171. Funding channels
    •https://opencollective.com/nrepl
    •https://opencollective.com/cider
    •https://patreon.com/bbatsov
    •https://paypal.me/bbatsov

    View Slide

  172. Help nREPL…

    View Slide

  173. …to help yourselves

    View Slide

  174. –Old Dutch Saying
    “You can’t build an epic Clojure
    programming experience on top of a
    poor REPL.”

    View Slide

  175. Felina

    View Slide

  176. Resources
    • https://metaredux.com/posts/2018/10/29/nrepl-redux.html
    • https://metaredux.com/posts/2019/01/12/nrepl-beyond-
    clojure.html
    • https://metaredux.com/posts/2019/03/29/nrepl-0-6.html
    • https://www.youtube.com/watch?v=4X-1fJm25Ww

    View Slide

  177. Credits
    twitter: @bbatsov
    github: @bbatsov
    https://metaredux.com
    https://emacsredux.com
    DCD 2019
    Amsterdam,
    The Netherlands
    06.04.2019

    View Slide