Slide 1
Slide 1 text
No content
Slide 2
Slide 2 text
π Marco Roth
t
@marcoroth_
M
@[email protected]
g
marcoroth.dev
g
@marcoroth
Full-Stack Developer & Open Source Contributor
Slide 3
Slide 3 text
Jim Remsik π
Slide 4
Slide 4 text
Developer Tooling For The Modern
Hotwire & Rails Era
Marco Roth
Full-Stack Developer & Open Source Contributor
Slide 5
Slide 5 text
CableReady
Core Team
Maintainer
Slide 6
Slide 6 text
I β€ Open Source
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Started to actively contribute in 2019
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Had enough
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
X
Slide 14
Slide 14 text
Superpower
β‘
Slide 15
Slide 15 text
The future of modern
Rails apps
Slide 16
Slide 16 text
I want to help build the
modern tooling needed
Slide 17
Slide 17 text
Went full-time on
OSS in January 2022
Slide 18
Slide 18 text
To work on Hotwire
and related projects
Slide 19
Slide 19 text
But, I noticed something
Slide 20
Slide 20 text
The tooling and DX is
awesome
Slide 21
Slide 21 text
We didn
β
t really have anything
like it in the Ruby world
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
I started to miss these tools
while working in Ruby/Rails
Slide 24
Slide 24 text
No IDE
Slide 25
Slide 25 text
They felt clunky, heavyweight
and cumbersome to use
Slide 26
Slide 26 text
Made me
less productive
Slide 27
Slide 27 text
But this changed
in the last few years
Slide 28
Slide 28 text
Editor Evolution
Slide 29
Slide 29 text
Language Server Protocol
(LSP)
Slide 30
Slide 30 text
What is it and
why do we want to use it?
Slide 31
Slide 31 text
Language Intelligence
without the need for an IDE
Slide 32
Slide 32 text
Source: https://code.visualstudio.com/api/language-extensions/language-server-extension-guide
Slide 33
Slide 33 text
How does it work?
Slide 34
Slide 34 text
Your editor starts a one
background process per
language
Slide 35
Slide 35 text
Communicates over
STDIN/STDOUT or TCP/IP
Slide 36
Slide 36 text
JSON-RPC
Slide 37
Slide 37 text
Similar to HTTP
Slide 38
Slide 38 text
Content-Length: ...\r\n
\r\n
{
"jsonrpc": "2.0",
"id": 1,
"method": "textDocument/completion",
"params": {
...
}
}
Slide 39
Slide 39 text
Stimulus LSP
What are we trying to solve?
Slide 40
Slide 40 text
Greet
Slide 41
Slide 41 text
Greet
Slide 42
Slide 42 text
Slide 43
Slide 43 text
data-controller
data-action
data-[identifier]-target
data-[identifier]-[class]-class
data-[identifier]-[value]-value
data-[identifier]-[outlet]-outlet
Slide 44
Slide 44 text
Slide 45
Slide 45 text
Slide 46
Slide 46 text
Slide 47
Slide 47 text
These mistakes
happen so easily
Slide 48
Slide 48 text
I thought:
"We can do better!"
Slide 49
Slide 49 text
Jun 23, 2021
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
Let
β
s take a look
how it works
Slide 55
Slide 55 text
{
id: 1,
method: "textDocument/completion",
params: {
"textDocument": {
"uri": "file:///.../lsp-example/.../test.txt"
},
"position": {
"line": 1,
"character": 7
},
"context": {
"triggerKind": 1
}
}
}
Slide 56
Slide 56 text
{
id: 1,
method: "textDocument/completion",
params: {
"textDocument": {
"uri": "file:///.../lsp-example/.../test.txt"
},
"position": {
"line": 1,
"character": 7
},
"context": {
"triggerKind": 1
}
}
}
Slide 57
Slide 57 text
{
id: 1,
method: "textDocument/completion",
params: {
"textDocument": {
"uri": "file:///.../lsp-example/.../test.txt"
},
"position": {
"line": 1,
"character": 7
},
"context": {
"triggerKind": 1
}
}
}
Slide 58
Slide 58 text
{
id: 1,
method: "textDocument/completion",
params: {
"textDocument": {
"uri": "file:///.../lsp-example/.../test.txt"
},
"position": {
"line": 1,
"character": 7
},
"context": {
"triggerKind": 1
}
}
}
Slide 59
Slide 59 text
{
id: 1,
result: [
{
"label": "TypeScript",
"kind": 1,
"data": 1
},
{
"label": "JavaScript",
"kind": 1,
"data": 2
}
]
}
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
Greet
Slide 62
Slide 62 text
Embedded Languages
Slide 63
Slide 63 text
The Stimulus grammar is not
part of the HTML spec
Slide 64
Slide 64 text
So we need to extend the
handling for HTML
Slide 65
Slide 65 text
Language services
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
Data providers
Slide 68
Slide 68 text
import { IHTMLDataProvider } from "vscode-html-languageservice"
export class StimulusHTMLDataProvider implements IHTMLDataProvider {
provideTags() {
...
}
provideAttributes(tag: string) {
...
}
provideValues(tag: string, attribute: string) {
...
}
}
Slide 69
Slide 69 text
Which language do
you write it in?
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
Existing Data Providers
Portability
Documentation/Integration
Existing Language Services
Slide 73
Slide 73 text
import { IHTMLDataProvider } from "vscode-html-languageservice"
export class StimulusHTMLDataProvider implements IHTMLDataProvider {
provideTags() {
...
}
provideAttributes(tag: string) {
...
}
provideValues(tag: string, attribute: string) {
...
}
}
Slide 74
Slide 74 text
Let
β
s implement it
Slide 75
Slide 75 text
provideTags() {
}
Slide 76
Slide 76 text
provideTags() {
return []
}
Slide 77
Slide 77 text
provideAttributes(tag: string) {
}
Slide 78
Slide 78 text
data-controller
data-action
data-[identifier]-target
data-[identifier]-[class]-class
data-[identifier]-[value]-value
data-[identifier]-[outlet]-outlet
Slide 79
Slide 79 text
data-controller
data-action
data-[identifier]-target
data-[identifier]-[class]-class
data-[identifier]-[value]-value
data-[identifier]-[outlet]-outlet
Slide 80
Slide 80 text
provideAttributes(_tag: string) {
return [
{ name: "data-controller" },
{ name: "data-action" },
]
}
Slide 81
Slide 81 text
data-controller
data-action
data-[identifier]-target
data-[identifier]-[class]-class
data-[identifier]-[value]-value
data-[identifier]-[outlet]-outlet
Slide 82
Slide 82 text
Parse JavaScript
Slide 83
Slide 83 text
Parse Stimulus Controllers
Slide 84
Slide 84 text
class ControllerDefinition {
readonly path: string
methods: Array = []
targets: Array = []
classes: Array = []
values: { [key: string]: Value } = {}
}
Slide 85
Slide 85 text
Acorn
Slide 86
Slide 86 text
import { Parser as AcornParser } from "acorn"
import { simple as walk } from "acorn-walk"
class Parser {
parseController(code: string, filename: string) {
...
}
}
Slide 87
Slide 87 text
f
app/javascript/controllers/hello_controller.js
Slide 88
Slide 88 text
export default class extends Controller {
static targets = ["name", "output"]
connect() {
...
}
greet() {
...
}
disconnect() {
...
}
}
Slide 89
Slide 89 text
export default class extends Controller {
static targets = ["name", "output"]
connect() {
...
}
greet() {
...
}
disconnect() {
...
}
}
Slide 90
Slide 90 text
const pattern = "app/javascript/controllers/**/*_controller.js"
const controllerFiles = await glob(pattern)
controllerFiles.forEach(async path => {
const code = await fs.readFile(path, "utf8")
parser.parseController(code, path)
})
Slide 91
Slide 91 text
parseController(code: string, filename: string) {
const ast = this.parse(code)
const controller = new ControllerDefinition(filename)
walk(ast, {
MethodDefinition(node) {
if (node.kind === "method") {
controller.methods.push(node.key.name)
}
},
})
}
Slide 92
Slide 92 text
parseController(code: string, filename: string) {
const ast = this.parse(code)
const controller = new ControllerDefinition(filename)
walk(ast, {
MethodDefinition(node) {
if (node.kind === "method") {
controller.methods.push(node.key.name)
}
},
})
}
Slide 93
Slide 93 text
parseController(code: string, filename: string) {
const ast = this.parse(code)
const controller = new ControllerDefinition(filename)
walk(ast, {
MethodDefinition(node) {
if (node.kind === "method") {
controller.methods.push(node.key.name)
}
},
})
}
Slide 94
Slide 94 text
parseController(code: string, filename: string) {
const ast = this.parse(code)
const controller = new ControllerDefinition(filename)
walk(ast, {
MethodDefinition(node) {
if (node.kind === "method") {
controller.methods.push(node.key.name)
}
},
})
}
Slide 95
Slide 95 text
We do something similar for
the Targets
Slide 96
Slide 96 text
Slide 97
Slide 97 text
data-controller
data-action
data-[identifier]-target
data-[identifier]-[class]-class
data-[identifier]-[value]-value
data-[identifier]-[outlet]-outlet
Slide 98
Slide 98 text
provideAttributes(_tag: string) {
return [
{ name: "data-controller" },
{ name: "data-action" },
]
}
Slide 99
Slide 99 text
provideAttributes(_tag: string) {
const targets = controllers.map(controller =>
`data-${controller.identifier}-target`
)
return [
{ name: "data-controller" },
{ name: "data-action" },
...targets,
]
}
Slide 100
Slide 100 text
provideAttributes(_tag: string) {
const targets = controllers.map(controller =>
`data-${controller.identifier}-target`
)
return [
{ name: "data-controller" },
{ name: "data-action" },
...targets,
]
}
Slide 101
Slide 101 text
provideAttributes(_tag: string) {
const targets = controllers.map(controller =>
`data-${controller.identifier}-target`
)
return [
{ name: "data-controller" },
{ name: "data-action" },
...targets,
]
}
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
Slide 104
Slide 104 text
provideValues(_tag: string, attribute: string) {
}
Slide 105
Slide 105 text
provideValues(_tag: string, attribute: string) {
if (attribute === "data-controller") {
return this.controllers.map(controller =>
controller.identifier
)
}
const match = attribute.match(/data-(.+)-target/)
const controller = this.controllers.find(controller =>
controller.identifier == match[1]
)
return controller.targets
}
Slide 106
Slide 106 text
provideValues(_tag: string, attribute: string) {
if (attribute === "data-controller") {
return this.controllers.map(controller =>
controller.identifier
)
}
const match = attribute.match(/data-(.+)-target/)
const controller = this.controllers.find(controller =>
controller.identifier == match[1]
)
return controller.targets
}
Slide 107
Slide 107 text
No content
Slide 108
Slide 108 text
https://github.com/marcoroth/stimulus-lsp/commit/a8ec2cb4cd430ffe5ccc08c44249c12513ae3ce2
Slide 109
Slide 109 text
Jun 25, 2021
Slide 110
Slide 110 text
No content
Slide 111
Slide 111 text
What else can we do?
Slide 112
Slide 112 text
Diagnostics
Slide 113
Slide 113 text
No content
Slide 114
Slide 114 text
No content
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
No content
Slide 117
Slide 117 text
No content
Slide 118
Slide 118 text
What else can we do?
Slide 119
Slide 119 text
Code Actions to
guide the user in the right
direction
Slide 120
Slide 120 text
No content
Slide 121
Slide 121 text
Fast forward to
October 2023
Slide 122
Slide 122 text
Available Now
Stimulus LSP
Slide 123
Slide 123 text
I asked the community for
feedback
Slide 124
Slide 124 text
And got a lot of awesome
feedback and ideas
Slide 125
Slide 125 text
Some of the naive approaches
we used were starting to fall
short
Slide 126
Slide 126 text
Especially the way we were parsing
the JavaScript with Acorn
Slide 127
Slide 127 text
We hard-coded
app/javascript/controllers/
Slide 128
Slide 128 text
import { application } from "controllers/application"
import { eagerLoadControllersFrom } from "@hotwired/stimulus-loading"
eagerLoadControllersFrom("controllers", application)
Slide 129
Slide 129 text
No content
Slide 130
Slide 130 text
The Language Server works
Slide 131
Slide 131 text
But produced a lot of false
positives and inaccurate results
Slide 132
Slide 132 text
Which is really
annoying as a user
Slide 133
Slide 133 text
We addressed and
fixed a lot of issues
Slide 134
Slide 134 text
Which technically ended up in a
rewrite of the parser
Slide 135
Slide 135 text
Fast forward to
May 2024
Slide 136
Slide 136 text
No content
Slide 137
Slide 137 text
No content
Slide 138
Slide 138 text
What
β
s next?
Slide 139
Slide 139 text
No content
Slide 140
Slide 140 text
None of is this is
exclusive to the LSP
Slide 141
Slide 141 text
The diagnostics could
be used independently
Slide 142
Slide 142 text
Kind of like RuboCop is
an independent tool
Slide 143
Slide 143 text
Coming Soon
Stimulus Lint
Slide 144
Slide 144 text
Errors
Diagnostics
Best Practices
Recommendations
Slide 145
Slide 145 text
$ stimulus-lint
Stimulus Lint is inspecting 25 files
........................
25 files inspected, 0 offenses detected, 0 offenses autocorrectable
Slide 146
Slide 146 text
<%= tag.div(
data: {
controller: "filter",
filter_open_class: "border-white",
filter_close_class: "hover:bg-gray-100 border-gray-300"
}
) %>
Full support for HTML+ERB and Rails-specific helpers
Slide 147
Slide 147 text
Diagnostics
Code Actions
Code Completion
Refactoring Tools
Slide 148
Slide 148 text
Pre-Release
Turbo LSP
Slide 149
Slide 149 text
No content
Slide 150
Slide 150 text
Turbo also makes heavy use
of ERB and view helpers
Slide 151
Slide 151 text
So it
β
s only really useful if we
can provide intelligence
Slide 152
Slide 152 text
And that only works if
we actually understand ERB
Slide 153
Slide 153 text
Stimulus LSP understands
HTML in ERB documents
Slide 154
Slide 154 text
But it doesn
β
t understand
Embedded Ruby that outputs
HTML
Slide 155
Slide 155 text
No content
Slide 156
Slide 156 text
No content
Slide 157
Slide 157 text
Source: https://twitter.com/vinistock/status/1710280529889620038
Slide 158
Slide 158 text
ERB Support in Ruby LSP
Slide 159
Slide 159 text
RubyConf 2023
Hack Day
Slide 160
Slide 160 text
No content
Slide 161
Slide 161 text
We quickly realized that we need
something more powerful to make
things work beyond the basics
Slide 162
Slide 162 text
<%= @user.firstname %>
<%= @user.lastname %>
Slide 163
Slide 163 text
<%= @user.firstname %>
<%= @user.lastname %>
Slide 164
Slide 164 text
<%= @user.firstname %>
<%= @user.lastname %>
Slide 165
Slide 165 text
Source: https://github.com/Shopify/ruby-lsp/pull/2235
Slide 166
Slide 166 text
Source: https://railsatscale.com/2024-07-18-mastering-ruby-code-navigation-major-enhancements-in-ruby-lsp-2024
Slide 167
Slide 167 text
<%= @user.firstname %>
<%= @user.lastname %>
Slide 168
Slide 168 text
We lost the HTML-context
in the process
Slide 169
Slide 169 text
<%= "Hello World" %>
>
Slide 170
Slide 170 text
Experiment
HTML-aware ERB Parser
Slide 171
Slide 171 text
Stimulus LSP
Turbo LSP
Stimulus Lint
Ruby LSP (maybe?)
Slide 172
Slide 172 text
type="text" <% end %> />
<% @posts.each do |post| %>
<%= post.title %>
<% end %> <%= content_tag(:p, "Hello world!") %> <%= tag.div tag.p("Hello world!") %> <%= tag.p do %> Hello world! <% end %> <%= tag.div( data: { controller: "hello", action: "click->hello#greet" } ) %>Slide 173
Slide 173 text
Which made me think that it
could be useful to have an
HTML abstraction
Slide 174
Slide 174 text
In which the ERB view helpers
also have an HTML tag node
for Rails View Helpers
Slide 175
Slide 175 text
<%= tag.p do %>
Hello World!
<% end %>
Hello World!
==Slide 176
Slide 176 text
==
<%= tag.div(
data: {
controller: "hello",
action: "click->hello#greet"
}
) %>
Slide 177
Slide 177 text
Hypertext Abstract Syntax Tree
Slide 178
Slide 178 text
{
"type": "element",
"tagName": "div",
"properties": {
"data-controller": "hello",
"data-action": "click->hello#greet",
},
"children": []
}
Slide 179
Slide 179 text
But we would have to adapt it to
be compatible with ERB
Slide 180
Slide 180 text
Needs more exploration
Slide 181
Slide 181 text
This might also open to door to
support Haml, Slim, Liquid, Phlex,
Blade and other template
languages
Slide 182
Slide 182 text
As long as we can
transform the template
syntax into HAST
Slide 183
Slide 183 text
Then we could build the
whole logic for the Stimulus
LSP on top of that
Slide 184
Slide 184 text
erb-languageservice
html-templating-languageservice
Slide 185
Slide 185 text
If the parser can understand any ERB
context and Rails-render calls it is super
powerful
Slide 186
Slide 186 text
This could power advanced
diagnostics
Slide 187
Slide 187 text
Slide 188
Slide 188 text
Slide 189
Slide 189 text
Modern Rails apps are composed of
multiple views/partials/components
Slide 190
Slide 190 text
<%= render partial: "input" %>
Slide 191
Slide 191 text
But if the parser understands the
render logic
Slide 192
Slide 192 text
<%= render partial: "input" %>
Slide 193
Slide 193 text
<%= render partial: "input" %>
Slide 194
Slide 194 text
I think an advanced HTML-aware ERB
parser could proof super helpful
Slide 195
Slide 195 text
Stimulus LSP
Stimulus Lint
Turbo LSP
Ruby LSP
Slide 196
Slide 196 text
html_press gem
phlexing gem
deface gem
better_html gem
erb_lint gem
Probably even more
Slide 197
Slide 197 text
HTML+ERB Formatter
HTML+ERB Linter
and more?
Slide 198
Slide 198 text
Prism has a big effect on Ruby
internals and the tooling landscape
Slide 199
Slide 199 text
An HTML-aware ERB parser could
have a similar effect for HTML
template tooling
Slide 200
Slide 200 text
In Progress
Hotwire Browser Extension
Slide 201
Slide 201 text
No content
Slide 202
Slide 202 text
No content
Slide 203
Slide 203 text
No content
Slide 204
Slide 204 text
It
β
s a work in progress
Slide 205
Slide 205 text
No content
Slide 206
Slide 206 text
Highlight Stimulus Controllers & Turbo Frames
Slide 207
Slide 207 text
Turbo Event Logging
Slide 208
Slide 208 text
Sneak Peak
Slide 209
Slide 209 text
No content
Slide 210
Slide 210 text
No content
Slide 211
Slide 211 text
Similarly for Stimulus Targets
Slide 212
Slide 212 text
No content
Slide 213
Slide 213 text
Turbo Streams Debug Bar
Pause Streams Processing
Step through Streams
Revert Turbo Stream Actions
Show Diff of what an action changed
and moreβ¦
Slide 214
Slide 214 text
Turbo Morphing Debug
Turbo Morphing Visual Diffs
Turbo Form Submission Debug
Turbo Streams Header Debug
ActionCable / Turbo Rails Cable Debug
Turbo Frame Lazy Loading Debug
Stimulus Controller Tree-View Panel
β¦
Ideas & Roadmap
Slide 215
Slide 215 text
Let us know if you have any ideas,
we are just getting started
Slide 216
Slide 216 text
Conclusion
Slide 217
Slide 217 text
Developer Experience and
Developer Ergonomics matter.
Slide 218
Slide 218 text
This is the kind of tooling we
are currently missing.
Slide 219
Slide 219 text
There is so much potential to
level up
Slide 220
Slide 220 text
I am super excited for the
future of these tools.
Slide 221
Slide 221 text
I believe that tools like these are part of
the reason why a language/framework
keeps it
β
s relevancy
Slide 222
Slide 222 text
Or put differently
Slide 223
Slide 223 text
These tools are necessary and
expected from a modern
language and framework
Slide 224
Slide 224 text
X
Slide 225
Slide 225 text
I want to keep building with
Ruby and Rails
Slide 226
Slide 226 text
I want to help build the tools
needed for the future of Rails
applications.
Slide 227
Slide 227 text
WEEKLY
Slide 228
Slide 228 text
Hotwire Weekly
β’ A new Hotwire-focused newsletter
β’ Delivered Weekly
β’ Explore what
β
s happening in the world of Hotwire
β’ Progress and updates while we are building out hotwire.io
β’ Been running since Oct 2023
β’ Over 1350+ subscribers and growing
WEEKLY
Slide 229
Slide 229 text
t
@Hotwire_Weekly
M
@[email protected]
Hotwire Weekly
Sign up on hotwireweekly.com
Slide 230
Slide 230 text
A more productive Hotwire
ecosystem
Slide 231
Slide 231 text
A more unified Hotwire
ecosystem
Slide 232
Slide 232 text
Feedback Ideas Thoughts
Improvements
Come
fi
nd me!
Slide 233
Slide 233 text
Stickers
Slide 234
Slide 234 text
Thank you π
t
@marcoroth_
M
@[email protected]
g
marcoroth.dev
g
@marcoroth