search & analysis engine for Mozilla codebases. ! It’s a code browser, full text & regex searcher, structural query language understands the code it indexes. ! There is so much cool stuff to do. Laura & I would both love to see contributions from other webeng folks. My job… Convince you it’s so much fun, you won’t be able to help but start hacking on it.
Reference Da Cross Reference? Dino Cross Reference? Disco Cross Reference? Past & present of DXR—our search & analysis engine for Mozilla codebases. ! It’s a code browser, full text & regex searcher, structural query language understands the code it indexes. ! There is so much cool stuff to do. Laura & I would both love to see contributions from other webeng folks. My job… Convince you it’s so much fun, you won’t be able to help but start hacking on it.
into 3 milestones. When I started, couldn’t even type a word into the query field without the JS scrambling the letters. Squarely at the ashamed-of-it milestone. Now, we are within a hair’s breadth of the no-longer-ashamed-of-it milestone, ! causing the famously hard-to-please Taras to say things like dink and dink.
can be divided into 3 milestones. When I started, couldn’t even type a word into the query field without the JS scrambling the letters. Squarely at the ashamed-of-it milestone. Now, we are within a hair’s breadth of the no-longer-ashamed-of-it milestone, ! causing the famously hard-to-please Taras to say things like dink and dink.
DXR’s evolution during my stewardship can be divided into 3 milestones. When I started, couldn’t even type a word into the query field without the JS scrambling the letters. Squarely at the ashamed-of-it milestone. Now, we are within a hair’s breadth of the no-longer-ashamed-of-it milestone, ! causing the famously hard-to-please Taras to say things like dink and dink.
“Much better than any other code browser I use” DXR’s evolution during my stewardship can be divided into 3 milestones. When I started, couldn’t even type a word into the query field without the JS scrambling the letters. Squarely at the ashamed-of-it milestone. Now, we are within a hair’s breadth of the no-longer-ashamed-of-it milestone, ! causing the famously hard-to-please Taras to say things like dink and dink.
“Much better than any other code browser I use” “Damn it feels good” DXR’s evolution during my stewardship can be divided into 3 milestones. When I started, couldn’t even type a word into the query field without the JS scrambling the letters. Squarely at the ashamed-of-it milestone. Now, we are within a hair’s breadth of the no-longer-ashamed-of-it milestone, ! causing the famously hard-to-please Taras to say things like dink and dink.
it. ! ▼ ❑ 27 structural filters • ❑ Callers • ❑ Subclasses, superclasses • ❑ Uses & definitions of types & macros • ❑ Drill in using browse interface ▼ ❑ moz-central & comm-central • ❑ Both updating every 6 hours • ❑ 50 more trees to come, quite possibly including webdev stuff ▼ ❑ All of this is real-time. • ❑ We just did a regex search over a 2.5GB codebase in under a sec. • ❑ Trigram indices
it. ! ▼ ❑ 27 structural filters • ❑ Callers • ❑ Subclasses, superclasses • ❑ Uses & definitions of types & macros • ❑ Drill in using browse interface ▼ ❑ moz-central & comm-central • ❑ Both updating every 6 hours • ❑ 50 more trees to come, quite possibly including webdev stuff ▼ ❑ All of this is real-time. • ❑ We just did a regex search over a 2.5GB codebase in under a sec. • ❑ Trigram indices
soon we will start chasing the proud-of-it milestone. I want you to help with that. It’s too much fun not to share. Toward that, I want to give you an overview of how DXR’s structured….
it proud! of it milestones Very soon we will start chasing the proud-of-it milestone. I want you to help with that. It’s too much fun not to share. Toward that, I want to give you an overview of how DXR’s structured….
Questions? • ❑ Now, this structure hasn't changed much since I started on the project. But everything inside the boxes has changed like crazy. ! When I first started working on DXR, it was kind of a mess.
whole doggone thing yourself, fight with versions of sqlite dev headers, tame the linker and loader. ▼ ❑ A Vagrant box (one of Lonnen's early contributions)
Vagrant. • ❑ That way, we don't have to get LLVM installed on the host and continually pester anybody for updates. • ❑ As a bonus, the suitability of the box as a fast spin-up platform for contribs stays validated.
user groups ask if I use Flask. • ❑ I don't know what to say. • ❑ There are maybe 20 Flask-specific lines of code in DXR. • ❑ Part of that is a credit to Flask and its use of plain data types. • ❑ But a lot is that the interesting stuff in DXR happens at indexing and search. • ❑ There are only 5 HTTP endpoints. • ❑ There is zero state mutable at request time. • ❑ It's barely a web project. • ❑ In fact, I'd like to see us add a well-documented JSON endpoint for search and spin off a CLI and editor integration.
hiding all the time Consistent query lang. Proper parser based on Parsimonious All fresh JS from Schalk and contribs. Features like multi-line highlights. Pushstate/popstate. ! Used to look like this. Now looks like this.
hiding all the time Consistent query lang. Proper parser based on Parsimonious All fresh JS from Schalk and contribs. Features like multi-line highlights. Pushstate/popstate. ! Used to look like this. Now looks like this.
hiding all the time Consistent query lang. Proper parser based on Parsimonious All fresh JS from Schalk and contribs. Features like multi-line highlights. Pushstate/popstate. ! Used to look like this. Now looks like this.
entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
the entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
gradually rewriting the entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
noticed, we're gradually rewriting the entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
you haven't noticed, we're gradually rewriting the entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
In case you haven't noticed, we're gradually rewriting the entire thing. ! UI: been through one major revision http interface: been replaced render pipeline: rewritten query machinery: rewriting now storage backend: up next ! It's an aggressively 4-dimensional project.
feel more like an air traffic controller than anything else. ! In fact, it was 4-dimensionality that led to the second of only 2 meetings we’ve ever had about DXR. [Story: landing rust support] ! ▼ ❑ That's another thing that makes the project fun: • ❑ You get to talk to your fellow contribs a lot to figure out what order you can do all this stuff in • ❑ all the while keeping it deployable and not ending up in merge hell. ! Nice things that give us freedom & help toward that…
•! ❑!No migrations to worry about! ! ▼!❑!We change schemas all the time! ! ▼!❑!When we do, we increment a "format version" so the deploy waits.! ! •! ❑!An example of the overriding philosophy I try to take on the project: do the simplest thing that could possibly work.! ! •! ❑!More specifically, do the simplest thing that works and doesn't prevent us from making it better.! ! •! ❑!We were going to have this complicated system of build triggers, set off by a duo of commits and timers.! ! •! ❑!We still may someday, but, until then, YAGNI.!
in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
to welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
like to welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
I would like to welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
Laura & I would like to welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
result mixing Laura & I would like to welcome everyone in this room to hack on DXR. One of the neat things about the project is the diversity of things to do. Whether you like UI design or mucking about with compilers, there’s something really important for you. Here’s what people are already working on. ! • ❑ Line-based searching ▼ ❑ SQLite → ES ▼ ❑ Common misconception that concurrency is bad with SQLite • ❑ Only for writes ▼ ❑ Will enable older-rev display • ❑ Major thing toward killing off MXR, which is our prime directive • ❑ Will enable dynamic rendering • ❑ Rust • ❑ More trees • ❑ incremental indexing. Much lower hardware requirements • ❑ result mixing ! I think it’s really important(…) to work on what really turns you on: better work.
! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
could use help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
stuff we could use help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
more stuff we could use help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
harnesses Here’s more stuff we could use help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
harnesses config system Here’s more stuff we could use help on: ! • ❑ Polishing off a few unfinished pieces of the UI refresh (CSS, template DRY) so we can be really proud of it. If you like little things or front-end things, this would be good for you. • ❑ Multi-language support in single codebase • ❑ Chasing reports of bad analysis (might span C++ and Python work). Great for a grand tour of the codebase. ▼ ❑ Little refactorings • ❑ Package DXR as a Python package. Almost there. • ❑ Getting the last of the boilerplate makefiles out of the test harness • ❑ Rewrite config system • ❑ Not on this list but implicit: The thing that interests you the most. That's the most important consideration. ! I hope I’ve expressed how much fun I find DXR to work on. Admittedly, I’m a little crazy, but I welcome you to join me in my craziness.
erikrose
yanagii
yusukebe
pokutuna
sakahukamaki
yahonda
biacco42
hal_spidernight
satorunooshie
d_endo
minodriven
sucitw
sgash708
matthewcrist
garrettdimon
michaelherold
chriscoyier
danielanewman
trallard
bluesmoon
sferik
swwweet
akosma
mza
addyosmani
