nREPL Redux (DCD 2019)

Slide-deck from my presentation at Dutch Clojure Days 2019.

Transcript

1. M-x
   

   View Slide

2. M-x greet-audience
   

   View Slide

3. Божидар
   

   View Slide

4. Bug
   cool
   

   View Slide

5. View Slide

6. Sofia, Bulgaria
   Sofia, Bulgaria
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. We’re not hiring!
    

    View Slide

11. View Slide

12. @bbatsov
    

    View Slide

13. metaredux.com
    

    View Slide

14. emacsredux.com
    

    View Slide

15. View Slide

16. slide intentionally
    left blank
    

    View Slide

17. nREPL
    REdux
    

    View Slide

18. Dictionary Definitions
    

    View Slide

19. nREPL
    

    View Slide

20. network REPL
    

    View Slide

21. redux
    

    View Slide

22. adjective
    1. brought back, revived
    Emacs era redux
    2. (usually postpositive) (esp of
    an artistic work) presented in a new way
    Apocalypse Now Redux
    

    View Slide

23. nREPL Redux?
    

    View Slide

24. nREPL is legacy
    

    View Slide

25. nREPL is complex
    

    View Slide

26. (nREPL) middleware sucks
    

    View Slide

27. Rich hates nREPL
    

    View Slide

28. The Socket REPL is the future
    

    View Slide

29. unrepl is the future
    

    View Slide

30. prepl is the future
    

    View Slide

31. LSP is the future
    

    View Slide

32. notorious REPL
    

    View Slide

33. nice REPL
    

    View Slide

34. neo REPL
    

    View Slide

35. noble REPL
    

    View Slide

36. novel REPL
    

    View Slide

37. mREPL
    

    View Slide

38. mega REPL
    

    View Slide

39. mighty REPL
    

    View Slide

40. magic REPL
    

    View Slide

41. magnificent REPL
    

    View Slide

42. marvellous REPL
    

    View Slide

43. majestic REPL
    

    View Slide

44. Naming is hard!
    

    View Slide

45. nREPL
    REdux
    

    View Slide

46. nREPL is very
    actively maintained
    

    View Slide

47. nREPL is not complex
    

    View Slide

48. Simple by design
    

    View Slide

49. nREPL is
    superbly documented
    

    View Slide

50. View Slide

51. Middleware is a
    very powerful concept
    

    View Slide

52. A huge ecosystem of tools
    exists around nREPL
    

    View Slide

53. lein repl
    

    View Slide

54. boot repl
    

    View Slide

55. vim-fireplace
    

    View Slide

56. vim-iced
    

    View Slide

57. acid.vim
    

    View Slide

58. Calva (VS Code)
    

    View Slide

59. Proto REPL (Atom)
    

    View Slide

60. Cursive
    

    View Slide

61. View Slide

62. A REPL for (Great) Good
    

    View Slide

63. View Slide

64. vim-clojure
    custom
    SLIME
    swank
    Enclojure
    custom
    CCW
    custom
    

    View Slide

65. vim-clojure
    nREPL
    SLIME
    nREPL
    Enclojure
    nREPL
    CCW
    nREPL
    

    View Slide

66. Project Goals
    •Common foundation for Clojure tooling
    •Trivial to embed
    •Zero runtime dependencies
    •Extremely simple wire format
    •Easy to extend with additional functionality
    

    View Slide

67. vim-fireplace
    nREPL
    CIDER
    nREPL
    Cursive
    nREPL
    CCW
    nREPL
    vim-iced
    nREPL
    Calva
    nREPL
    acid.vim
    nREPL
    Proto REPL
    nREPL
    

    View Slide

68. View Slide

69. Clojure Contrib
    

    View Slide

70. org.clojure/tools.nrepl
    

    View Slide

71. •Contributor Agreement
    •No Pull Requests
    •JIRA for issue tracking
    •Hudson CI
    •Maven
    •Artifacts are deployed to Maven Central
    

    View Slide

72. View Slide

73. View Slide

74. The (REPL) Wasteland
    

    View Slide

75. github.com/clojure/tools.nrepl
    Repo deprecation announcement
    

    View Slide

76. View Slide

77. 17 contributors in 8 years
    

    View Slide

78. Almost no activity since 2015
    

    View Slide

79. 25%
    5%
    70%
    Yes Maybe
    I already have Notepad No
    Do you want great Clojure development tools?
    

    View Slide

80. 69%
    25%
    5%
    1%
    Yes Maybe
    I already have Notepad No
    Are you willing to work on Clojure development tools?
    

    View Slide

81. SLIME -> CIDER
    

    View Slide

82. CIDER loves nREPL
    

    View Slide

83. View Slide

84. View Slide

85. Rogue One
    

    View Slide

86. Challenges
    

    View Slide

87. I didn’t really know what I
    had signed up for
    

    View Slide

88. Breaking changes to one of the
    most widely used tools in the
    Clojure ecosystem
    

    View Slide

89. org.clojure/tools.nrepl
    ->
    nrepl/nrepl
    

    View Slide

90. clojure.tools.nrepl.*
    ->
    nrepl.*
    

    View Slide

91. A ton of 3rd party tools and
    libraries needed to be updated
    

    View Slide

92. •lein
    •boot
    •figwheel
    •shadow-cljs
    •REPLy
    •Ultra
    •All nREPL extensions in existence…
    

    View Slide

93. •piggieback
    •cider-nrepl
    •refactor-nrepl
    •sayid
    •drawbridge
    •and many others…
    

    View Slide

94. Migration Strategy
    • Teach most extensions about both versions of nREPL
    • Migrate tools like Lein and Boot to the new nREPL
    • Provide ways for users of older Boot/Lein to run the new nREPL
    • Drop support for tools.nrepl from the extensions
    • Document nREPL extensively in the process
    • Focus on actual nREPL improvements
    

    View Slide

95. Building a community
    around the project
    

    View Slide

96. A Catalyst for Change
    

    View Slide

97. REPL (R)Evolution
    

    View Slide

98. •nREPL 0.3 (initial compatibility release)
    •nREPL 0.4 (multiple feature releases)
    •nREPL 0.5
    •nREPL 0.6
    

    View Slide

99. Better Versioning Policy
    

    View Slide

100. 0.feature.patch
     

     View Slide

101. Then
     •0.2.5 (Feature)
     •0.2.6 (Feature)
     •0.2.7 (Bug fix)
     •0.2.8 (Feature)
     

     View Slide

102. Now
     •0.5.0 (Feature)
     •0.5.1 (Bug fix)
     •0.5.2 (Bug fix)
     •0.6.0 (Feature)
     

     View Slide

103. Clear Compatibility Policy
     • All supported LTS JDKs (Currently Java 8 and Java 11)
     • Latest rapid cycle JDK (Currently Java 12)
     • Clojure 1.7+
     • Extreme dedication to protocol compatibility
     • Surgical changes at the implementation-level
     

     View Slide

104. Fixed
     many long-standing bugs
     

     View Slide

105. Complete overhaul of the
     internals
     

     View Slide

106. •Simpler code evaluation model
     •Robust handling of Clojure’s special variables
     •Massively improved value printing logic
     •Better way to hook into evaluation errors
     •Global/project server configuration via configuration
     files
     •Reusable command-line API
     

     View Slide

107. Exhaustive documentation
     (nrepl.org)
     

     View Slide

108. Great integration with
     tools.deps
     

     View Slide

109. Transit transport
     (https://github.com/nrepl/fastlane)
     

     View Slide

110. EDN transport
     (coming in nREPL 0.7)
     

     View Slide

111. Support for hot-loading
     libraries in a running REPL
     

     View Slide

112. No breaking changes to the
     protocol
     

     View Slide

113. Minimal breaking changes on
     the API level
     

     View Slide

114. Team-work FTW
     

     View Slide

115. Chas creates the new repo
     Rogue One
     

     View Slide

116. View Slide

117. 23 new contributors!
     

     View Slide

118. Christophe Grand
     

     View Slide

119. Michael Griffiths
     

     View Slide

120. View Slide

121. View Slide

122. Monopoly is good
     

     View Slide

123. Monopoly allows you to
     focus on innovation
     

     View Slide

124. Meet The Competition
     

     View Slide

125. The Socket REPL
     

     View Slide

126. •bundled with Clojure & ClojureScript
     •streaming REPL
     •a socket based version of the default REPL
     •easy if you want to evaluate something in a remote app
     •can be started with a JVM property
     •hard to build tools on it
     

     View Slide

127. $ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/repl}”
     $ nc localhost 5555
     user=> (+ 10 10)
     20
     user=> (println "Hello, world!")
     Hello, world!
     nil
     

     View Slide

128. prepl
     

     View Slide

129. $ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/io-prepl}”
     $ nc localhost 5555
     (+ 10 10)
     {:tag :ret, :val "20", :ns "user", :ms 23, :form "(+ 10 10)”}
     (println "Hello, world")
     {:tag :out, :val "Hello, world\n"}
     {:tag :ret, :val "nil", :ns "user", :ms 8, :form "(println \"Hello, world\")"}
     

     View Slide

130. unrepl
     

     View Slide

131. •based on the socket REPL
     •allows clients to customize the REPL
     •has a sophisticated rich data printer
     •streaming input, framed EDN output
     

     View Slide

132. Language Server Protocol
     • Language agnostic
     • Doesn’t rely on a running REPL
     • A much broader protocol (completions, code lookup, etc)
     • Usually is implemented in terms of static analysis
     • Message oriented, JSON data format
     • Extensible protocol
     • Controlled by an evil corporation turned good
     

     View Slide

133. View Slide

134. Swiss army knife of REPLs
     

     View Slide

135. • designed to power programmings tools
     • message oriented
     • can be extended with arbitrary operations
     • interruptible evals
     • session multiplexing
     • supports arbitrary communication protocol/data format
     • async in nature
     

     View Slide

136. ;; request
     {:id "1"
     :op "eval"
     :code "(+ 1 1)”
     :ns "user"}
     ;; response
     {:id "1"
     :value "2"}
     

     View Slide

137. ;; request
     {:id "1"
     :op "complete"
     :thing "map”
     :ns "user"}
     ;; response
     {:id "1"
     :candidates […]}
     

     View Slide

138. ;; raw request
     {:id 1 :op "eval" :code "(+ 1 1)"}
     ;; encoded request
     “d4:code7:(+ 1 1)2:idi1e2:op4:evale"
     

     View Slide

139. “d4:code7:(+ 1 1)2:idi1e2:op4:evale"
     •d (dictionary)
     •4:code - a 4 byte string “code”
     •7:(+ 1 1) - a 7 byte string “(+ 1 1)”
     •2:id - a 2 byte string “id”
     •i1e - an integer 1
     •…
     •e - end of dictionary
     

     View Slide

140. Community Driven
     

     View Slide

141. Open Source is not about You
     

     View Slide

142. nREPL is
     all about YOU!!!
     

     View Slide

143. Beyond Clojure
     

     View Slide

144. nREPL as a protocol
     

     View Slide

145. Simple by design
     

     View Slide

146. Key Operations
     •eval
     •load-file
     •interrupt
     •stdin
     

     View Slide

147. Bencode Data Types
     •strings
     •integers
     •lists
     •maps
     

     View Slide

148. View Slide

149. A light-weight alternative to
     LSP
     

     View Slide

150. Easy to implement
     

     View Slide

151. nREPL implementations
     •HyREPL - an nREPL for the Hy programming language
     •JeeJah - an nREPL server for Fennel and Lua
     •Ogion - an nREPL server for Racket
     •Chicken NREPL - an nREPL server for Chicken Scheme
     •cl-nrepl - an nREPL server for Common Lisp
     •cljs-noderepl - an nREPL server for ClojureScript running on Node.js
     •R-nREPL - an nREPL server for R
     •nREPL CLR - an nREPL server for ClojureCLR
     •Arcadia nREPL - an nREPL server implemented in C# for Arcadia.
     

     View Slide

152. Dreams for the future
     • nREPL implemented in ClojureScript/ClojureCLR
     • nREPL for Erlang/Elixir
     • nREPL for Smalltalk
     • Better behaved nREPL clients (ones that don’t make assumptions
     they are dealing with a Clojure server)
     

     View Slide

153. (next nrepl)
     

     View Slide

154. Roadmap for nREPL 0.7
     •EDN Transport (done)
     •Client-side resource/class injection API (unrepl)
     •Upgrade socket REPL to nREPL (unrepl)
     •Rich data printer (unrepl)
     

     View Slide

155. Personal Impact
     

     View Slide

156. Extremely proud
     

     View Slide

157. View Slide

158. Frustrated
     

     View Slide

159. Tired
     

     View Slide

160. Burned out
     

     View Slide

161. View Slide

162. View Slide

163. View Slide

164. Sustainable
     

     View Slide

165. Community Funded?
     

     View Slide

166. Clojure’s Orchard
     

     View Slide

167. $4000
     

     View Slide

168. $5400
     

     View Slide

169. We need to get Clojure companies
     invested
     in the Clojure tooling ecosystem
     

     View Slide

170. 20 companies paying
     $500/month can make a
     massive difference
     

     View Slide

171. Funding channels
     •https://opencollective.com/nrepl
     •https://opencollective.com/cider
     •https://patreon.com/bbatsov
     •https://paypal.me/bbatsov
     

     View Slide

172. Help nREPL…
     

     View Slide

173. …to help yourselves
     

     View Slide

174. –Old Dutch Saying
     “You can’t build an epic Clojure
     programming experience on top of a
     poor REPL.”
     

     View Slide

175. Felina
     

     View Slide

176. Resources
     • https://metaredux.com/posts/2018/10/29/nrepl-redux.html
     • https://metaredux.com/posts/2019/01/12/nrepl-beyond-
     clojure.html
     • https://metaredux.com/posts/2019/03/29/nrepl-0-6.html
     • https://www.youtube.com/watch?v=4X-1fJm25Ww
     

     View Slide

177. Credits
     twitter: @bbatsov
     github: @bbatsov
     https://metaredux.com
     https://emacsredux.com
     DCD 2019
     Amsterdam,
     The Netherlands
     06.04.2019
     

     View Slide