How To Quickly Understand Millions of Lines of Code

Reading source code is not really doable once you hit the tens of thousands of lines of code. It's even more hopeless at millions of them. Yet, analysis tools that can summarise this information struggle just as much as humans do. So how do we build tools that can handle such ginormous codebases, anyway?

In this talk we'll take a practical (but superficial) look at some of the algorithms involved in the making of Glass, a static analysis tool developed at Klarna, and the optimisations that allow providing answers to analysis in real-time for IDEs, and reasonable-time for build/CI tools.

Links:
- Glass' GitHub repository: https://github.com/klarna-incubator/glass

- µKanren: A Minimal Functional Core for Relational Programming: http://webyrd.net/scheme-2013/papers/HemannMuKanren2013.pdf
- How Developers Search for Code: A Case Study: https://research.google/pubs/pub43835/
- Adapton: Composable, Demand-Driven Incremental Computation: http://matthewhammer.org/adapton/adapton-pldi2014.pdf

Transcript

1. How to Quickly Understand Millions of Lines of Code? (a case for static analysis) http://talks.robotlolita.me
2. Hi, I’m Quil. I like designing programming languages and tools. (aka @robotlolita)
3. And I want to work on tools to help people understand their programs.
4. Why?
5. Systems begin small and cute. meow~
6. And then, they grow. A lot. Hello, tiny human.
7. Reading the source to understand it isn’t feasible anymore. Just 9000 books to go...
8. This is the perfect case for software analysis tools! We already rely on Dialyzer, Cover, Xref, etc
9. And so I came up with an idea for a project that I called Glass. it means “ice cream” in Swedish.
10. it’s also a backronym: Generic Language Analysis & Search System
11. One of the primary use cases is “semantic code search” Where’s “lists:map/2” used?
12. Let me have my cake and eat it, too! “I want code search that is fast enough to be used in an IDE, easy to use, and scalable for codebases of any size!” Let me have my cake and eat it, too!
13. Glass’s approach is based on pattern matching & relational logic. influenced by miniKanren
14. Let’s say you looking for where ct:pal/2 is called: “I’m looking for code like this”
15. But what if you want very specific calls? “I’m looking for code like this” “But only if this holds”
16. To evaluate queries, we unify patterns against every node in our AST. progress: 1 of 9 million
18. + Ignore this match
19. This approach is generally efficient, and queries are easier to write.
20. Compare to the Datalog approach:
21. “If only I had time to work on this cool idea…”
22. A colleague and I built a prototype during an OSS hackathon at Klarna.
23. We ran queries against Glass’ own codebase… They completed in less than 1 second.
24. But we want Glass to work on BIG codebases, too...
25. So we ran it on our biggest codebase: • > 1 million lines of code • > 6000 Erlang modules • > 250 Erlang applications • Maintained by several teams over many years
26. ...and things didn’t go so smooothly. Some queries didn’t even finish!
27. Can we make it go faster?
28. ONE BIG PROBLEM: Search time is proportional to the size of the codebase.
29. I know! Let’s use indexes! I know! Let’s use indexes!
30. SECOND BIG PROBLEM: Code changes frequently, and indexing is expensive.
31. Visiting every node is bad!!! Oh no, the code changed AGAIN! Indexing!!! Invalidation :(
32. Let’s take a step back, How do we want people to use Glass?
33. We expect users to search from their IDE, while editing code.
34. “I’m looking for code that looks like this, but only in some contexts.”
35. “Where do we still have debug logs?” “fully qualified function references!”
36. “Are we still accessing record_a directly?” “record names look good for indexes?”
37. “Have I seen this pattern before…?” “fully qualified function references will likely be here.”
38. “Did I forget type specs anywhere?” “function definitions and other static ops might not be so common...”
39. Queries have plenty of static information we can infer indexes from!
40. Caitlin et al made similar observations at Google.
41. Caitlin et al’s work also adds: Queries are generally bounded to “known” locations, rather than spanning all code.
42. New knowledge! • Static information in queries --- infer indexes from it! • Queries are generally localised --- try to use location for ranking results! --- stream results to improve perceived performance!
43. But what about code changes?!
44. Changes are frequent, but localised.
45. Make our indexes incremental!
46. INCREMENTAL COMPUTATION A technique for efficiently updating a program’s output when part of its inputs change.
47. A common approach is to maintain a dependency graph and a cache.
48. Running with a cold cache will compute everything.
49. Running with a hot cache will only compute what has changed.
50. For Erlang, we have module-level dependencies (because of erlc).
51. So our dependency graph looks like this: module header files parser transforms
52. Header files and parser transforms don’t change as often! Amortised cost is small!
53. Incremental Pattern indexes + queries = correct and fast searches, even in an IDE.
54. One thing I’ve learned: Having a clear UX vision and design constraints was very helpful. As did experience, of course.
55. Glass’ design constraints were: • The time to think of a query matters --- make it tangible; • The perceived query performance matters --- show useful results quickly; • The user effort to get fast queries matters --- make queries easy to optimise;
56. Glass is still a work in progress! @klarna-incubator/glass
57. PAPER RECOMMENDATIONS µKanren: A Minimal Functional Core for Relational Programming Jason Hemann and Daniel P. Friedman (2013) How Developers Search for Code: A Case Study Caitlin Sadowski, Kathryn T. Stolee, and Sebastian Elbaum (2015) Adapton: Composable, Demand-Driven Incremental Computation Matthew A. Hammer, Khoo Yit Phang, Michael Hicks, and Jeffrey S. Foster (2014)
58. THAT’S ALL, FOLKS http://talks.robotlolita.me