nREPL Redux (DCD 2019) - Speaker Deck

Slide 1

Slide 1 text

M-x

Slide 2

Slide 2 text

M-x greet-audience

Slide 3

Slide 3 text

Божидар

Slide 4

Slide 4 text

Bug cool

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Sofia, Bulgaria Soﬁa, Bulgaria

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

We’re not hiring!

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

@bbatsov

Slide 13

Slide 13 text

metaredux.com

Slide 14

Slide 14 text

emacsredux.com

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

slide intentionally left blank

Slide 17

Slide 17 text

nREPL REdux

Slide 18

Slide 18 text

Dictionary Definitions

Slide 19

Slide 19 text

nREPL

Slide 20

Slide 20 text

network REPL

Slide 21

Slide 21 text

redux

Slide 22

Slide 22 text

adjective 1. brought back, revived Emacs era redux 2. (usually postpositive) (esp of an artistic work) presented in a new way Apocalypse Now Redux

Slide 23

Slide 23 text

nREPL Redux?

Slide 24

Slide 24 text

nREPL is legacy

Slide 25

Slide 25 text

nREPL is complex

Slide 26

Slide 26 text

(nREPL) middleware sucks

Slide 27

Slide 27 text

Rich hates nREPL

Slide 28

Slide 28 text

The Socket REPL is the future

Slide 29

Slide 29 text

unrepl is the future

Slide 30

Slide 30 text

prepl is the future

Slide 31

Slide 31 text

LSP is the future

Slide 32

Slide 32 text

notorious REPL

Slide 33

Slide 33 text

nice REPL

Slide 34

Slide 34 text

neo REPL

Slide 35

Slide 35 text

noble REPL

Slide 36

Slide 36 text

novel REPL

Slide 37

Slide 37 text

mREPL

Slide 38

Slide 38 text

mega REPL

Slide 39

Slide 39 text

mighty REPL

Slide 40

Slide 40 text

magic REPL

Slide 41

Slide 41 text

magniﬁcent REPL

Slide 42

Slide 42 text

marvellous REPL

Slide 43

Slide 43 text

majestic REPL

Slide 44

Slide 44 text

Naming is hard!

Slide 45

Slide 45 text

nREPL REdux

Slide 46

Slide 46 text

nREPL is very actively maintained

Slide 47

Slide 47 text

nREPL is not complex

Slide 48

Slide 48 text

Simple by design

Slide 49

Slide 49 text

nREPL is superbly documented

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Middleware is a very powerful concept

Slide 52

Slide 52 text

A huge ecosystem of tools exists around nREPL

Slide 53

Slide 53 text

lein repl

Slide 54

Slide 54 text

boot repl

Slide 55

Slide 55 text

vim-ﬁreplace

Slide 56

Slide 56 text

vim-iced

Slide 57

Slide 57 text

acid.vim

Slide 58

Slide 58 text

Calva (VS Code)

Slide 59

Slide 59 text

Proto REPL (Atom)

Slide 60

Slide 60 text

Cursive

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

A REPL for (Great) Good

Slide 63

Slide 63 text

2010

Slide 64

Slide 64 text

vim-clojure custom SLIME swank Enclojure custom CCW custom

Slide 65

Slide 65 text

vim-clojure nREPL SLIME nREPL Enclojure nREPL CCW nREPL

Slide 66

Slide 66 text

Project Goals •Common foundation for Clojure tooling •Trivial to embed •Zero runtime dependencies •Extremely simple wire format •Easy to extend with additional functionality

Slide 67

Slide 67 text

vim-ﬁreplace nREPL CIDER nREPL Cursive nREPL CCW nREPL vim-iced nREPL Calva nREPL acid.vim nREPL Proto REPL nREPL

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

Clojure Contrib

Slide 70

Slide 70 text

org.clojure/tools.nrepl

Slide 71

Slide 71 text

•Contributor Agreement •No Pull Requests •JIRA for issue tracking •Hudson CI •Maven •Artifacts are deployed to Maven Central

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

010101010101010110

Slide 74

Slide 74 text

The (REPL) Wasteland

Slide 75

Slide 75 text

github.com/clojure/tools.nrepl Repo deprecation announcement

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

17 contributors in 8 years

Slide 78

Slide 78 text

Almost no activity since 2015

Slide 79

Slide 79 text

25% 5% 70% Yes Maybe I already have Notepad No Do you want great Clojure development tools?

Slide 80

Slide 80 text

69% 25% 5% 1% Yes Maybe I already have Notepad No Are you willing to work on Clojure development tools?

Slide 81

Slide 81 text

SLIME -> CIDER

Slide 82

Slide 82 text

CIDER loves nREPL

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

Rogue One

Slide 86

Slide 86 text

Challenges

Slide 87

Slide 87 text

I didn’t really know what I had signed up for

Slide 88

Slide 88 text

Breaking changes to one of the most widely used tools in the Clojure ecosystem

Slide 89

Slide 89 text

org.clojure/tools.nrepl -> nrepl/nrepl

Slide 90

Slide 90 text

clojure.tools.nrepl.* -> nrepl.*

Slide 91

Slide 91 text

A ton of 3rd party tools and libraries needed to be updated

Slide 92

Slide 92 text

•lein •boot •ﬁgwheel •shadow-cljs •REPLy •Ultra •All nREPL extensions in existence…

Slide 93

Slide 93 text

•piggieback •cider-nrepl •refactor-nrepl •sayid •drawbridge •and many others…

Slide 94

Slide 94 text

Migration Strategy • Teach most extensions about both versions of nREPL • Migrate tools like Lein and Boot to the new nREPL • Provide ways for users of older Boot/Lein to run the new nREPL • Drop support for tools.nrepl from the extensions • Document nREPL extensively in the process • Focus on actual nREPL improvements

Slide 95

Slide 95 text

Building a community around the project

Slide 96

Slide 96 text

A Catalyst for Change

Slide 97

Slide 97 text

REPL (R)Evolution

Slide 98

Slide 98 text

•nREPL 0.3 (initial compatibility release) •nREPL 0.4 (multiple feature releases) •nREPL 0.5 •nREPL 0.6

Slide 99

Slide 99 text

Better Versioning Policy

Slide 100

Slide 100 text

0.feature.patch

Slide 101

Slide 101 text

Then •0.2.5 (Feature) •0.2.6 (Feature) •0.2.7 (Bug ﬁx) •0.2.8 (Feature)

Slide 102

Slide 102 text

Now •0.5.0 (Feature) •0.5.1 (Bug ﬁx) •0.5.2 (Bug ﬁx) •0.6.0 (Feature)

Slide 103

Slide 103 text

Clear Compatibility Policy • All supported LTS JDKs (Currently Java 8 and Java 11) • Latest rapid cycle JDK (Currently Java 12) • Clojure 1.7+ • Extreme dedication to protocol compatibility • Surgical changes at the implementation-level

Slide 104

Slide 104 text

Fixed many long-standing bugs

Slide 105

Slide 105 text

Complete overhaul of the internals

Slide 106

Slide 106 text

•Simpler code evaluation model •Robust handling of Clojure’s special variables •Massively improved value printing logic •Better way to hook into evaluation errors •Global/project server configuration via configuration files •Reusable command-line API

Slide 107

Slide 107 text

Exhaustive documentation (nrepl.org)

Slide 108

Slide 108 text

Great integration with tools.deps

Slide 109

Slide 109 text

Transit transport (https://github.com/nrepl/fastlane)

Slide 110

Slide 110 text

EDN transport (coming in nREPL 0.7)

Slide 111

Slide 111 text

Support for hot-loading libraries in a running REPL

Slide 112

Slide 112 text

No breaking changes to the protocol

Slide 113

Slide 113 text

Minimal breaking changes on the API level

Slide 114

Slide 114 text

Team-work FTW

Slide 115

Slide 115 text

Chas creates the new repo Rogue One

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

23 new contributors!

Slide 118

Slide 118 text

Christophe Grand

Slide 119

Slide 119 text

Michael Grifﬁths

Slide 120

Slide 120 text

No content

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

Monopoly is good

Slide 123

Slide 123 text

Monopoly allows you to focus on innovation

Slide 124

Slide 124 text

Meet The Competition

Slide 125

Slide 125 text

The Socket REPL

Slide 126

Slide 126 text

•bundled with Clojure & ClojureScript •streaming REPL •a socket based version of the default REPL •easy if you want to evaluate something in a remote app •can be started with a JVM property •hard to build tools on it

Slide 127

Slide 127 text

$ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/repl}” $ nc localhost 5555 user=> (+ 10 10) 20 user=> (println "Hello, world!") Hello, world! nil

Slide 128

Slide 128 text

prepl

Slide 129

Slide 129 text

$ clj -J-Dclojure.server.jvm="{:port 5555 :accept clojure.core.server/io-prepl}” $ nc localhost 5555 (+ 10 10) {:tag :ret, :val "20", :ns "user", :ms 23, :form "(+ 10 10)”} (println "Hello, world") {:tag :out, :val "Hello, world\n"} {:tag :ret, :val "nil", :ns "user", :ms 8, :form "(println \"Hello, world\")"}

Slide 130

Slide 130 text

unrepl

Slide 131

Slide 131 text

•based on the socket REPL •allows clients to customize the REPL •has a sophisticated rich data printer •streaming input, framed EDN output

Slide 132

Slide 132 text

Language Server Protocol • Language agnostic • Doesn’t rely on a running REPL • A much broader protocol (completions, code lookup, etc) • Usually is implemented in terms of static analysis • Message oriented, JSON data format • Extensible protocol • Controlled by an evil corporation turned good

Slide 133

Slide 133 text

No content

Slide 134

Slide 134 text

Swiss army knife of REPLs

Slide 135

Slide 135 text

• designed to power programmings tools • message oriented • can be extended with arbitrary operations • interruptible evals • session multiplexing • supports arbitrary communication protocol/data format • async in nature

Slide 136

Slide 136 text

;; request {:id "1" :op "eval" :code "(+ 1 1)” :ns "user"} ;; response {:id "1" :value "2"}

Slide 137

Slide 137 text

;; request {:id "1" :op "complete" :thing "map” :ns "user"} ;; response {:id "1" :candidates […]}

Slide 138

Slide 138 text

;; raw request {:id 1 :op "eval" :code "(+ 1 1)"} ;; encoded request “d4:code7:(+ 1 1)2:idi1e2:op4:evale"

Slide 139

Slide 139 text

“d4:code7:(+ 1 1)2:idi1e2:op4:evale" •d (dictionary) •4:code - a 4 byte string “code” •7:(+ 1 1) - a 7 byte string “(+ 1 1)” •2:id - a 2 byte string “id” •i1e - an integer 1 •… •e - end of dictionary

Slide 140

Slide 140 text

Community Driven

Slide 141

Slide 141 text

Open Source is not about You

Slide 142

Slide 142 text

nREPL is all about YOU!!!

Slide 143

Slide 143 text

Beyond Clojure

Slide 144

Slide 144 text

nREPL as a protocol

Slide 145

Slide 145 text

Simple by design

Slide 146

Slide 146 text

Key Operations •eval •load-ﬁle •interrupt •stdin

Slide 147

Slide 147 text

Bencode Data Types •strings •integers •lists •maps

Slide 148

Slide 148 text

No content

Slide 149

Slide 149 text

A light-weight alternative to LSP

Slide 150

Slide 150 text

Easy to implement

Slide 151

Slide 151 text

nREPL implementations •HyREPL - an nREPL for the Hy programming language •JeeJah - an nREPL server for Fennel and Lua •Ogion - an nREPL server for Racket •Chicken NREPL - an nREPL server for Chicken Scheme •cl-nrepl - an nREPL server for Common Lisp •cljs-noderepl - an nREPL server for ClojureScript running on Node.js •R-nREPL - an nREPL server for R •nREPL CLR - an nREPL server for ClojureCLR •Arcadia nREPL - an nREPL server implemented in C# for Arcadia.

Slide 152

Slide 152 text

Dreams for the future • nREPL implemented in ClojureScript/ClojureCLR • nREPL for Erlang/Elixir • nREPL for Smalltalk • Better behaved nREPL clients (ones that don’t make assumptions they are dealing with a Clojure server)

Slide 153

Slide 153 text

(next nrepl)

Slide 154

Slide 154 text

Roadmap for nREPL 0.7 •EDN Transport (done) •Client-side resource/class injection API (unrepl) •Upgrade socket REPL to nREPL (unrepl) •Rich data printer (unrepl)

Slide 155

Slide 155 text

Personal Impact

Slide 156

Slide 156 text

Extremely proud

Slide 157

Slide 157 text

No content

Slide 158

Slide 158 text

Frustrated

Slide 159

Slide 159 text

Tired

Slide 160

Slide 160 text

Burned out

Slide 161

Slide 161 text

No content

Slide 162

Slide 162 text

No content

Slide 163

Slide 163 text

No content

Slide 164

Slide 164 text

Sustainable

Slide 165

Slide 165 text

Community Funded?

Slide 166

Slide 166 text

Clojure’s Orchard

Slide 167

Slide 167 text

$4000

Slide 168

Slide 168 text

$5400

Slide 169

Slide 169 text

We need to get Clojure companies invested in the Clojure tooling ecosystem

Slide 170

Slide 170 text

20 companies paying $500/month can make a massive difference

Slide 171

Slide 171 text

Funding channels •https://opencollective.com/nrepl •https://opencollective.com/cider •https://patreon.com/bbatsov •https://paypal.me/bbatsov

Slide 172

Slide 172 text

Help nREPL…

Slide 173

Slide 173 text

…to help yourselves

Slide 174

Slide 174 text

–Old Dutch Saying “You can’t build an epic Clojure programming experience on top of a poor REPL.”

Slide 175

Slide 175 text

Felina

Slide 176

Slide 176 text

Resources • https://metaredux.com/posts/2018/10/29/nrepl-redux.html • https://metaredux.com/posts/2019/01/12/nrepl-beyond- clojure.html • https://metaredux.com/posts/2019/03/29/nrepl-0-6.html • https://www.youtube.com/watch?v=4X-1fJm25Ww

Slide 177

Slide 177 text

Credits twitter: @bbatsov github: @bbatsov https://metaredux.com https://emacsredux.com DCD 2019 Amsterdam, The Netherlands 06.04.2019