CodeScene: Get Deep Insights into Your (Clojure) Code [Clojure Vienna meetup - March 2018 ]

Come and meet CodeScene, a unique behavioral code analysis tool written in Clojure. This tool goes beyond traditional static analysis by looking for patterns in version control data to understand the history and evolution of a code base: unraveling things like hotspots, temporal coupling between modules and an interesting social view of the code. I'll briefly describe the ideas behind CodeScene, how it works and then show you an analysis of real Clojure code.

Transcript

1. CodeScene Get Deep Insights into Your (Clojure) Code 1
2. About (me) • Java • Clojure • curiousprogrammer.net • Empear - CodeScene Brno, CZ Malmo, SE 2
3. Agenda • Motivation • What is CodeScene • Demo • “Under the hood” • Why (not) Clojure 3
4. Motivation • Developers spend most of their time on doing modifications to existing code • Technical Debt is inevitable and must be proactively eliminated. 4
6. Technical Debt 6
7. Motivation (2) • Clojure benefits - dynamic, quick experiments => rapid development • Technical Debt can accumulate quickly 7
8. Motivation (3) • Not all technical debt is created equal - Return On Investment • Static analysis ignores history • Relying solely on human’s expertise is ineffective and expensive • We need concrete data to be able to deal with Technical debt effectively 8
9. What is CodeScene? • Web-based code analysis tool that lets you prioritise technical debt, identify organizational inefficiences, and detect code that’s expensive to maintain. • Language neutral • On-prem & cloud version (codescene.io) 9
10. Demo: clojure.core 10
11. Demo: Technical Debt • Hotspots - commit frequency or code churn • Refactoring Targets - prioritised hotspots • X-Ray • Programming languages 11
12. Demo: Architecture • Hotspots • Conway’s Law 12
13. Demo: Social Analyses • Individuals • Owners • Knowledge loss (Rich Hickey) • Coordination needs - gvec.clj • Authors 13
14. Demo: Project Management • Branch statistics (re-frame) • Delivery risk • Lead Time To Merge 14
15. Demo: Early Warnings • metabase 15
16. “Under the Hood” • git log -> parse -> analyse -> store to disk -> … present (UI) • Pipes & Filters • Core “analysis” module reused by cloud and on- prem applications • Performance • Libraries 16
17. Pipes & Filters • Git -> git log -> parse -> analyze -> persist on disk -> load from disk -> present (UI) • Pipes & Filters 17
18. Parsing 18
19. Analyses • Language neutral - based on universal LOC metric • X-Ray (ANTLR) • Hotspots • Temporal Coupling 19
20. Analyses (2) • Core analysis module is completely separated and reused from both on-prem and cloud applications • Results are stored in plain CSV files - no DB! 20
21. Performance • Memory is the bottleneck => serialisation of analyses • Linux ~45 min on i7 machine 21
22. Libraries • ANTLR - microgrammars (X-Ray) • Incanter - buggy • ring & compojure • selmer • spec • clojure.java.jdbc - H2 • core.async - simple scheduler • 3rd party integrations - clj-http, proxy-vole, clj-ldap, clj-slack • etaoin - UI tests • cprops - configuration 22
23. Why Clojure? • It’s fun • Productivity • Quick feedback • Fast innovations • Data transformations 23
24. “That’s it? Just these guys?!” — Carl Gustaf 24
25. Why Not Clojure(Script)? • Server-side rendering with thin JS layer • Jenkins plugin - Java • Haskell - git cloning service • Kotlin? (IDEA plugin) 25
26. Conclusion • Technical debt is a real problem regardless of programming language • There’s a huge amount of useful information stored in your version control system • Ultimately, you need to rely on human expertise • Support your developer’s judgment and experience with data to get the highest ROI 26
27. Resources • CodeScene Introduction (15 min) screencast • CodeScene Product Sheet • CodeScene.io Showcase - clojure, erlang, React, etc. • Adam’s talks & books • Talk A Crystal Ball to Prioritize Technical Debt • EuroClojure 2015 talk Beyond Code: Repository Mining with Clojure • book Software Design X-Rays: Fix Technical Debt with Behavioral Code Analysis • book Your Code as a Crime Scene • CodeScene enterprise documentation • Empear’s blog - e.g. The Day I Parsed a Monster • code-maat repository 27
28. Go and Try! 28 https://codescene.io/