Static Analysis: enforcing code conventions

It’s well known that as a team grows, the communication overhead increases geometrically until it takes up most of the time of the members. Static analysis can help us reduce the time cost, streamline the code review process, as well as to keep big teams synchronized.

Transcript

1. References • https://engineering.linecorp.com/en/blog/synchronizing-geographically- distributed-teams-with-lint/ • https://groups.google.com/forum/#!forum/lint-dev • https://docs.google.com/presentation/d/ 1Sr-6E3Tk1lBguUob0GigqSEsfN-04qS56Whj0UEr0AE/ edit#slide=id.g29100eff00_3_126 • https://en.wikipedia.org/wiki/Static_program_analysis
2. Static Analysis: Enforcing code conventions Daniel Vila Teissiere @danivteissiere
3. Big teams
4. Big teams Big teams bring challenges:
5. Big teams Big teams bring challenges: • Onboarding of new members.
6. Big teams Big teams bring challenges: • Onboarding of new members. • Code consistency.
7. Big teams Big teams bring challenges: • Onboarding of new members. • Code consistency. • Code conventions.
8. Big teams Big teams bring challenges: • Onboarding of new members. • Code consistency. • Code conventions. • Internal APIs.
9. Big teams
10. Code conventions fun getInt()k= (0..10).map { it.also { "test" } }.also { “Test2" }.apply { size }.let { it.last() }.takeIf { it > 10 }
11. Code conventions fun getInt(): Int?k= (0..10).map { it.also { "test" } }.also { “Test2" }.apply { size }.let { it.last() }.takeIf { it > 10 }
12. What can we do? k
13. What can we do? Static analysis!
14. What can we do? Static analysis! A=Analyse code without executing it.
16. Why Lint? Compatible with Java/Kotlin (UAST).
17. Why Lint? Compatible with Java/Kotlin (UAST). Lint client is implemented by Android Studio.
18. Why Lint? Compatible with Java/Kotlin (UAST). Lint client is implemented by Android Studio. Quick fixes in Android Studio.
19. UAST UAST
20. UAST Universal Abstract Syntactic Tree
21. UAST package com.test class Test { fun test() = 1 }k
22. UAST package com.test class Test { fun test() = 1 }k println(context.uastFile?.asRecursiveLogString())
23. UAST package com.test class Test { fun test() = 1 }k println(context.uastFile?.asRecursiveLogString()) UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
24. UAST package com.test class Test { fun test() = 1 }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
25. UAST package com.test class Test { fun test() = 1 }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
26. UAST package com.test class Test { fun test() = 1 }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
27. UAST package com.test class Test { fun test() = 1 }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
28. UAST package com.test class Test { fun test() = 1 }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
29. UAST package com.test class Test { fun test() = 1 } UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
30. UAST package com.test; final class Test { int test() { return 1; }k }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
31. UAST package com.test; final class Test { int test() { return 1; }k }k UFile (package = com.test) UClass (name = Test) UMethod (name = test) UBlockExpression UReturnExpression ULiteralExpression (value = 1)
32. Lint Lint has two APIs
33. Lint Lint has two APIs Client API
34. Lint Lint has two APIs Client API Detector API
35. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
36. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
37. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
38. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
39. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
40. Lint Issue First we have to create an issue explaining what’s the problem and how to fix it: class MyDetector : Detector(), SourceCodeScanner { companion object { @JvmField val ISSUE: Issue = Issue.create( “MyDetectorID", “MyDetector short description", "My detector **very** *long* `description`.", Category.CORRECTNESS, 6, Severity.WARNING, Implementation(MyDetector::class.java, Scope.JAVA_FILE_SCOPE) )k }k …k }k
41. Lint Registry Now we have to add the issue to the lint issue registry: class MyIssueRegistry : IssueRegistry() {k override val issues: List<Issue> = listOf(MyDetector.ISSUE) override val api: Int = CURRENT_API override val minApi: Int = MIN_API companion object {k const val MIN_API = 2 // Corresponds to android gradle plugin 3.2+ }k }k
42. Lint Registry Now we have to add the issue to the lint issue registry: @AutoService(IssueRegistry::class) class MyIssueRegistry : IssueRegistry() {k override val issues: List<Issue> = listOf(MyDetector.ISSUE) override val api: Int = CURRENT_API override val minApi: Int = MIN_API companion object {k const val MIN_API = 2 // Corresponds to android gradle plugin 3.2+ }k }k
43. Detectors A Detector is responsible for detecting occurrences of an Issue in the source code, multiple Issues can use the same Detector.
44. Specialized Detectors Lint has a number of Detector specializations which makes a lot easier to write checks. • XmlScanner: lets you visit the XML file with DOM. • SourceCodeScanner: useful for Kotlin/Java analysis. • ClassScanner, BinaryResourceScanner, ResorceFolderScanner…
45. Specialized Detectors Lint has a number of Detector specializations which makes a lot easier to write checks. • XmlScanner: lets you visit the XML file with DOM. • SourceCodeScanner: useful for Kotlin/Java analysis. • ClassScanner, BinaryResourceScanner, ResorceFolderScanner…
46. Specialized Detectors Lint has a number of Detector specializations which makes a lot easier to write checks. • XmlScanner: lets you visit the XML file with DOM. • SourceCodeScanner: useful for Kotlin/Java analysis. • ClassScanner, BinaryResourceScanner, ResorceFolderScanner…
47. Specialized Detectors Lint has a number of Detector specializations which makes a lot easier to write checks. • XmlScanner: lets you visit the XML file with DOM. • SourceCodeScanner: useful for Kotlin/Java analysis. • ClassScanner, BinaryResourceScanner, ResorceFolderScanner…
48. SourceCodeScanner Has many useful callback functions, to name a few: fun applicableSuperClasses(): List<String>? fun getApplicableMethodNames(): List<String>? fun getApplicableUastTypes(): List<Class<out UElement>>?
49. SourceCodeScanner Has many useful callback functions, to name a few: fun applicableSuperClasses(): List<String>? fun getApplicableMethodNames(): List<String>? fun getApplicableUastTypes(): List<Class<out UElement>>?
50. SourceCodeScanner Has many useful callback functions, to name a few: fun applicableSuperClasses(): List<String>? fun getApplicableMethodNames(): List<String>? fun getApplicableUastTypes(): List<Class<out UElement>>?
51. SourceCodeScanner Has many useful callback functions, to name a few: fun applicableSuperClasses(): List<String>? fun getApplicableMethodNames(): List<String>? fun getApplicableUastTypes(): List<Class<out UElement>>?
52. SourceCodeScanner fun getInt()k= (0..10).map { it.also { "test" } }.also { “Test2" }.apply { size }.let { it.last() }.takeIf { it > 10 }
53. SourceCodeScanner fun getInt(): Int?k= (0..10).map { it.also { "test" } }.also { “Test2" }.apply { size }.let { it.last() }.takeIf { it > 10 }
54. SourceCodeScanner override fun getApplicableUastTypes(): MutableList<Class<out UElement>> = mutableListOf(UMethod::class.java) override fun createUastHandler(context: JavaContext): UElementHandler = InferredTypeHandler(context) class InferredTypeHandler(private val context: JavaContext) : UElementHandler() { ... override fun visitMethod(node: UMethod) { if (!isKotlin(node)) return if (isInferredMethod(node)){ report(context) } k } k ... } k
55. SourceCodeScanner override fun getApplicableUastTypes(): MutableList<Class<out UElement>> = mutableListOf(UMethod::class.java) override fun createUastHandler(context: JavaContext): UElementHandler = InferredTypeHandler(context) class InferredTypeHandler(private val context: JavaContext) : UElementHandler() { ... override fun visitMethod(node: UMethod) { if (!isKotlin(node)) return if (isInferredMethod(node)){ report(context) } k } k ... } k
56. SourceCodeScanner override fun getApplicableUastTypes(): MutableList<Class<out UElement>> = mutableListOf(UMethod::class.java) override fun createUastHandler(context: JavaContext): UElementHandler = InferredTypeHandler(context) class InferredTypeHandler(private val context: JavaContext) : UElementHandler() { ... override fun visitMethod(node: UMethod) { if (!isKotlin(node)) return if (isInferredMethod(node)){ report(context) } k } k ... } k
57. SourceCodeScanner private fun isInferredMethod(node: UMethod): Boolean { val function = node.sourcePsi as? KtNamedFunction ?: return false val isFunction = function.funKeyword != null val hasColon = function.colon != null val isVoid = node.returnType?.presentableText == "void" return isFunction && !hasColon && !isVoid }
58. PSI
59. Program Structure Interface
60. Program Structure Interface Used in IntelliJ to model Java code, XML, etc Many UElements also implement PSI interfaces: interface UClass : UDeclaration, PsiClass interface UMethod : UDeclaration, PsiMethod interface UField : UVariable, PsiField interface UParameter : UVariable, PsiParameter
61. SourceCodeScanner override fun getApplicableUastTypes(): MutableList<Class<out UElement>> = mutableListOf(UMethod::class.java) override fun createUastHandler(context: JavaContext): UElementHandler = InferredTypeHandler(context) class InferredTypeHandler(private val context: JavaContext) : UElementHandler() { ... override fun visitMethod(node: UMethod) { if (!isKotlin(node)) return if (isInferredMethod(node)){ report(context) } k } k ... } k
62. Report javaContext.report(ISSUE, uMethod, javaContext.getNameLocation(uMethod), getReportMessage(canonicalText))
63. JavaContext Lint provides a JavaContext with information about the project: • Information about the given file in which lint is running. • Report errors to the given client (Android Studio, Gradle)
64. JavaContext Lint provides a JavaContext with information about the project: • Information about the given file in which lint is running. • Report errors to the given client (Android Studio, Gradle)
65. JavaContext Lint provides a JavaContext with information about the project: • Information about the given file in which lint is running. • Report errors to the given client (Android Studio, Gradle)
66. Report javaContext.report(ISSUE, uMethod, javaContext.getNameLocation(uMethod), getReportMessage(canonicalText))
67. Report javaContext.report(ISSUE, uMethod, javaContext.getNameLocation(uMethod), getReportMessage(canonicalText))
68. Quick Fixes Provide a fix when the user presses cmd+enter in the IDE: LintFix.create().replace() .name("Add inferred type") .range(getLocation(node)) .with(fixString) .reformat(true) .build()
69. Report javaContext.report(ISSUE, uMethod, javaContext.getNameLocation(uMethod), getReportMessage(canonicalText) k ) k
70. Report javaContext.report(ISSUE, uMethod, javaContext.getNameLocation(uMethod), getReportMessage(canonicalText) k , fix) k
71. Report
72. Report
73. Report
74. Report
75. Report
76. Report
77. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambas. • Nesting in custom LiveData. • Top level class comments
78. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambas. • Nesting in custom LiveData. • Top level class comments.
79. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambas. • Nesting in custom LiveData. • Top level class comments.
80. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambdas. • Nesting in custom LiveData. • Top level class comments.
81. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambdas. • Nesting in custom LiveData. • Top level class comments.
82. Examples of checks Some examples of checks we are currently using: • BaseFragment within a package. • Internal APIs to recommend to use a custom subscriber. • Nesting in lambdas. • Nesting in custom LiveData. • Top level class comments.
83. Pretend to remember all your code conventions and internal APIs writing custom lint checks.
84. Gist https://gist.github.com/dtvc87/ c4118d25575d91a0e59603b0cf5724a9
85. We are hiring!