Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Static Code Analysis with Kotlin

Mohit S
August 28, 2018

Static Code Analysis with Kotlin

Detekt is a static code analysis tool for Kotlin. It can analyze your Kotlin projects for various types of code smells. The tool contains rules for style, performance and potential bugs. The tool is fully configurable so that you may turn off and set thresholds for these rules. It is also extensible so that you could create your own custom rules from your style guide. This tool goes a long way to enforcing good practices in your code base.

Under the hood, the library performs checks against these rules by analyzing the abstract syntax tree provided by the Kotlin Compiler. In this talk, we will learn about how to setup Detekt and how to create custom rules. We will also take a deep dive into how the library enforces these rules by using the abstract syntax tree.

Mohit S

August 28, 2018
Tweet

More Decks by Mohit S

Other Decks in Programming

Transcript

  1. Static Code Analysis with Kotlin • Why Static Code Analysis?

    • Setup & Configure Detekt • Program Structure Interface (PSI) • How Detekt uses the PSI to find code smells • Write your own Detekt rules
  2. • 100,000+ lines of Java • Started 5 years ago

    • Legacy code base • Migrating to Kotlin Problem
  3. Legacy code problems • Maintainability • Performance • Code smells

    • Defects • Warnings • Security • Inconsistent styles
  4. Maintainability class MainActivity : AppCompatActivity() { fun loadPost() { …

    } fun loadComments() { … } fun showComments() { … } fun likePost() { … } fun comment() { … } … }
  5. Potential Bugs class FeedActivity extends AppCompatActivity { @Nullable FeedAdapter feedAdapter;

    public void showFeed() { feedAdapter.getItemCount(); feedAdapter.registerAdapterDataObserver(…); feedAdapter.submitList(…); } }
  6. Potential Bugs class FeedActivity : AppCompatActivity() { var feedAdapter: FeedAdapter?

    = null fun showFeed() { feedAdapter!!!.itemCount feedAdapter!!!.registerAdapterDataObserver(…) feedAdapter!!!.submitList(…) }
  7. Potential Bugs class FeedActivity : AppCompatActivity() { var feedAdapter: FeedAdapter?

    = null fun showFeed() { feedAdapter!?.itemCount feedAdapter!?.registerAdapterDataObserver(…) feedAdapter!?.submitList(…) }
  8. Kotlin Idioms public class Post { private Integer id; private

    String title; private String message; 
 …
 
 }
  9. Kotlin Idioms public class Post { public Integer getId() {

    return id; } public String getTitle() { return title; } public String getMessage() { return message; }
 
 … }
  10. Kotlin Idioms public class Post { @Override public boolean equals(Object

    o) { … } @Override public int hashCode() { … } 
 }
  11. Kotlin Idioms class Post { var id: Int? = null

    var title: String? = null var message: String? = null override fun equals(o: Any?): Boolean { … } override fun hashCode(): Int { … } }
  12. Kotlin Idioms data class Post( val id: Int? = null,

    val title: String? = null, val message: String? = null )
  13. Static Code Analysis with Kotlin • Why Static Code Analysis?

    • Setup & Configure Detekt • Program Structure Interface (PSI) • How Detekt uses the PSI to find code smells • Write your own Detekt rules
  14. Add Detekt Plugin plugins { id "io.gitlab.arturbosch.detekt" version "1.0.0.RC8" }

    detekt { version = "1.0.0.RC8" profile(“main") { input = "$projectDir" config = "$project.rootDir/detekt-config.yml" filters = ".!*/resources/.*,.!*/build/.*" } }
  15. YML Configs complexity: active: true MethodOverloading: active: true threshold: 3

    potential-bugs: active: true UnsafeCallOnNullableType: active: true Rule Rule Set
  16. Report 182 kotlin files were analyzed. Ruleset: complexity
 MethodOverloading -

    [createPost] at ApiClient.kt:3:1 Ruleset: comments Ruleset: empty-blocks Ruleset: exceptions Ruleset: naming Ruleset: performance Ruleset: potential-bugs Ruleset: style
  17. Report Complexity Report: - 16662 lines of code (loc) -

    9633 source lines of code (sloc) - 6380 logical lines of code (lloc) - 4605 comment lines of code (cloc) - 1884 McCabe complexity (mcc) - 2 number of total code smells - 47 % comment source ratio - 295 mcc per 1000 lloc - 0 code smells per 1000 lloc
  18. YML Configs console-reports: active: true exclude: # - 'ProjectStatisticsReport' #

    - 'ComplexityReport' # - 'NotificationReport' # - 'FindingsReport' # - 'BuildFailureReport'
  19. YML Configs failFast: false build: maxIssues: 10 weights: # complexity:

    2 # LongParameterList: 1 # style: 1 # comments: 1
  20. class FeedActivity : AppCompatActivity() { var feedAdapter: FeedAdapter? = null

    !// Non-Compliant fun showFeed() { feedAdapter!!!.itemCount feedAdapter!!!.registerAdapterDataObserver(…) feedAdapter!!!.submitList(…) } Nullable Types
  21. class FeedActivity : AppCompatActivity() { var feedAdapter: FeedAdapter? = null

    !// Compliant fun showFeed() { feedAdapter!?.itemCount feedAdapter!?.registerAdapterDataObserver(…) feedAdapter!?.submitList(…) } Nullable Types
  22. Rule: Unsafe Call on Nullable Type Description: • Guard your

    code to prevent NullPointerExceptions Severity: • Defect
  23. Detekt Check Report 5 kotlin files were analyzed. Ruleset: potential-bugs

    UnsafeCallOnNullableType - [showFeed] at FeedActivity.kt:8:17 Ruleset: comments Ruleset: complexity Ruleset: empty-block Ruleset: exceptions Ruleset: naming Ruleset: performance Ruleset: style
  24. Use Optional Parameters !// Non-Compliant fun createPost(title: String) {} fun

    createPost(title: String, message: String) {}
 fun createPost(title: String, message: String, images: List<Images>) { }
  25. Use Optional Parameters !// Compliant fun createPost(title: String = “Untitled",

    message: String? = null, images: List<Image>? = null) { }
  26. Use Data Classes public class Post { @Override public boolean

    equals(Object o) { … } @Override public int hashCode() { … } 
 }
  27. Use Data Classes !// Non-Compliant class Post { var id:

    Int? = null var title: String? = null var message: String? = null override fun equals(o: Any?): Boolean { … } override fun hashCode(): Int { … } }
  28. Use Data Classes !// Compliant data class Post( val id:

    Int? = null, val title: String? = null, val message: String? = null )
  29. Rule: Use Data Class Description: • Enforce the usage of

    data classes for POJOs. Severity: • Style
  30. Rule Sets • Comments • Complexity • Empty-blocks • Exceptions

    • Formatting • Naming • Performance • Potential-Bugs • Style
  31. Git Pre-commit Hook ##!/usr/bin/env bash
 echo "Running detekt check##..." OUTPUT="/tmp/detekt-$(date

    +%s)" ./gradlew detektCheck > $OUTPUT EXIT_CODE=$? if [ $EXIT_CODE -ne 0 ]; then cat $OUTPUT rm $OUTPUT echo "***********************************************" echo " Detekt failed " echo " Please fix the above issues before committing " echo "***********************************************" exit $EXIT_CODE fi rm $OUTPUT
  32. Static Code Analysis with Kotlin • Why Static Code Analysis?

    • Setup & Configure Detekt • Program Structure Interface (PSI) • Finding Code Smells with PSI • Write your own Detekt rules.
  33. Program Structure Interface Program Structure Interface is a layer in

    the Intellij Platform for parsing files into a semantic model. 
 Source PsiElement PsiElement PsiElement PsiElement Java, Kotlin, Python, etc..
  34. Intellij Platform public interface PsiBuilder { Project getProject(); @NotNull Marker

    mark(); @NotNull ASTNode getTreeBuilt(); } PsiElement PsiElement PsiElement PsiElement intellij-community/platform/core-api/src/com/intellij/lang/PsiBuilder.java
  35. Kotlin Embeddable Compiler public class KotlinParsing { void parseFile() {

    PsiBuilder.Marker fileMarker = mark(); parsePreamble(); while (!eof()) { parseTopLevelDeclaration(); } checkUnclosedBlockComment(); fileMarker.done(KT_FILE); } } KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction kotlin/compiler/psi/src/org/jetbrains/kotlin/parsing/KotlinParsing.java
  36. class ApiClient { fun createPost(
 title: String, 
 message: String)

    { } } Parse into Tokens CLASS_KEYWORD LBRACE FUN_KEYWORD WHITE_SPACE RBRACE IDENTIFIER
  37. Parsing Source to Tokens … “typealias" { return KtTokens.TYPE_ALIAS_KEYWORD ;}

    "object" { return KtTokens.OBJECT_KEYWORD ;} "while" { return KtTokens.WHILE_KEYWORD ;} "break" { return KtTokens.BREAK_KEYWORD ;} "class" { return KtTokens.CLASS_KEYWORD ;} "throw" { return KtTokens.THROW_KEYWORD ;} "false" { return KtTokens.FALSE_KEYWORD ;} … kotlin/compiler/psi/src/org/jetbrains/kotlin/lexer/Kotlin.flex
  38. PSI Tree KtFile KtClass KtNamedFunction KtNamedFunction class ApiClient { fun

    createPost(
 title: String, 
 message: String) { } }
  39. Creating/Traverse PSI Tree 1. Configure the compiler.
 2. Get PSI

    Tree for a Kotlin file.
 3. Use visitor pattern to traverse PSI Tree
  40. Configure Compiler !// Configure Compiler val configuration = CompilerConfiguration() configuration.put(

    CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, PrintingMessageCollector(System.err, MessageRenderer.PLAIN_FULL_PATHS, false)) !// Create Project val psiProject: Project = KotlinCoreEnvironment.createForProduction( Disposer.newDisposable(), configuration, EnvironmentConfigFiles.JVM_CONFIG_FILES ).project
  41. Create PSI Tree val psiFactory = PsiFileFactory(psiProject) !// Get Kotlin

    File Content val path = Paths.get(“/path/to/kotlin/file.kt”) val content = path.toFile().readText()
  42. Create PSI Tree val psiFactory = PsiFileFactory(psiProject) !// Get Kotlin

    File Content val path = Paths.get(“/path/to/kotlin/file.kt”) val content = path.toFile().readText() !// Create PSI Tree val ktFile: PsiFile = psiFactory.createFileFromText( path.fileName.toString(), KotlinLanguage.INSTANCE, content,
 …)
  43. Create PSI Tree !// Create PSI Tree val ktFile: PsiFile

    = psiFactory.createFileFromText( path.fileName.toString(), KotlinLanguage.INSTANCE, content,
 …) KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  44. How do I traverse it? class ApiClient { fun createPost()

    { … } fun deletePost() { … } } KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  45. Visitor Provided by Compiler public class KtTreeVisitorVoid extends KtVisitor<Void, Void>

    { void visitKtElement(@NotNull KtElement element) { } void visitClass(@NotNull KtClass klass) { } void visitNamedFunction(@NotNull KtNamedFunction function) { } … } kotlin/compiler/psi/src/org/jetbrains/kotlin/KtVisitorVoid.java
  46. Create Visitor class MyVisitor: KtTreeVisitorVoid() { override fun visitKtFile(file: KtFile)

    { } override fun visitClass(klass: KtClass) { } override fun visitNamedFunction(function: KtNamedFunction) { } }
  47. Create Visitor !// Create PSI Tree val ktFile: PsiFile =

    psiFactory.createFileFromText( path.fileName.toString(), KotlinLanguage.INSTANCE, content,
 …) !// Specify visitor to traverse tree val visitor = MyVisitor()
 psiFile.accept(visitor) Start Traversing Tree
  48. Order of Traversal of PSI Tree class MyVisitor: KtTreeVisitorVoid() {

    override fun visitKtFile(file: KtFile) { println("visitKtFile") } override fun visitClass(klass: KtClass) { println("visitClass") } override fun visitNamedFunction(function: KtNamedFunction) { println(“visitNamedFunction") } }
  49. Output: visitNamedFunction visitNamedFunction visitClassBody visitClass visitKtFile Order of Traversal of

    PSI Tree KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  50. Output: visitNamedFunction visitNamedFunction visitClassBody visitClass visitKtFile Order of Traversal of

    PSI Tree KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  51. Output: visitNamedFunction visitNamedFunction visitClassBody visitClass visitKtFile Order of Traversal of

    PSI Tree KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  52. Output: visitNamedFunction visitNamedFunction visitClassBody visitClass visitKtFile Order of Traversal of

    PSI Tree KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  53. Static Code Analysis with Kotlin • Why Static Code Analysis?

    • Setup & Configure Detekt • Program Structure Interface (PSI) • How Detekt uses the PSI to find code smells • Write your own Detekt rules.
  54. Rule: Keep Data Classes Simple Description: • Data classes should

    ONLY hold data. • Data classes should NOT contain extra functions. • Overridden methods in data classes are OK.
  55. Rule: Keep Data Classes Simple !// Non-Compliant data class Post(val

    id: Int, val message: String) { fun create() { … } fun delete() { … } }
  56. Rule: Keep Data Classes Simple !// Non-Compliant data class Post(val

    id: Int, val message: String) { fun create() { … } fun delete() { … } } Should only store data
  57. Rule: Keep Data Classes Simple !// Compliant data class Post(val

    id: Int, val message: String) fun Post.create(apiClient: ApiClient) { … } fun Post.delete(apiClient: ApiClient) { … }
  58. Rule: Keep Data Classes Simple !// Compliant data class Post(val

    id: Int, val message: String) { override fun hashCode(): Int { … } override fun equals(other: Any?): Boolean { … } override fun toString(): String { … } }
  59. Strategy for Implementation • Determine class is data class •

    Does it any methods? • Are the methods are overridden? KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  60. Implementation Check type of KtClass • Enum • Object •

    Data class • Sealed class KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  61. KtClass API public open class KtClass : KtClassOrObject { public

    final fun isData(): Boolean { … } public final fun isEnum(): Boolean { … } public final fun isSealed(): Boolean { … } …
 } kotlin/compiler/psi/src/org/jetbrains/kotlin/parsing/KtClass.kt
  62. Implementation Get all declaration in the class • Properties •

    Companion Objects • Functions • Inner classes • etc… KtFile KtClass KtClassBody KtNamedFunction KtNamedFunction
  63. KtClassBody API public final class KtClassBody { public open fun

    getDeclarations(): MutableList<KtDeclaration> { … } } kotlin/compiler/psi/src/org/jetbrains/kotlin/parsing/KtClassBody.kt
  64. KtNamedFunction API public class KtNamedFunction extends KtModifierListOwner { boolean hasModifier(@NotNull

    KtModifierKeywordToken token); … } kotlin/compiler/psi/src/org/jetbrains/kotlin/parsing/KtNamedFunction.kt
  65. class DataClassContainsFunctions: KtTreeVisitorVoid() { override fun visitClass(klass: KtClass) { if

    (klass.isData()) { klass.getBody()!?.declarations } } } Implementation
  66. class DataClassContainsFunctions: KtTreeVisitorVoid() { override fun visitClass(klass: KtClass) { if

    (klass.isData()) { klass.getBody()!?.declarations !?.filterIsInstance<KtNamedFunction>() } } } Implementation
  67. class DataClassContainsFunctions: KtTreeVisitorVoid() { override fun visitClass(klass: KtClass) { 


    if (klass.isData()) {
 klass.getBody()!?.declarations !?.filterIsInstance<KtNamedFunction>() !?.filterNot { it.hasModifier(KtTokens.OVERRIDE_KEYWORD) } } } } Implementation
  68. class DataClassContainsFunctions: KtTreeVisitorVoid() { override fun visitClass(klass: KtClass) { 


    if (klass.isData()) {
 klass.getBody()!?.declarations !?.filterIsInstance<KtNamedFunction>() !?.filterNot { it.hasModifier(KtTokens.OVERRIDE_KEYWORD) } !?.forEach { reportCodeSmell(it.name) } } } } Implementation
  69. Rule Set: Complexity Description: • Enforce single responsibility principle in

    conditions, functions and classes. Rules: • Too Many Functions
  70. Rule: Too Many Functions Description: • Set a limit to

    # of functions you could have in a file, class, interfaces, etc… Threshold (default 11): • Files • Classes • Interfaces • Objects • Enums • Private
  71. Threshold: 11 !// Non-Compliant fun isPostPublished(…) { }
 
 fun

    isCommentingEnabled(…) { }
 
 fun sharePost(…) { }
 
 30+ methods 
 Rule: Too Many Functions
  72. Threshold: 11 !// Non-Compliant fun isPostPublished(…) { }
 
 @Deprecated


    fun isCommentingEnabled(…) { }
 
 fun sharePost(…) { }
 
 30+ methods
 Rule: Too Many Functions Do not count deprecated methods
  73. Threshold: 11 !// Non-Compliant fun isPostPublished(…) { }
 
 @Deprecated


    fun isCommentingEnabled(…) { }
 
 fun sharePost(…) { }
 
 
 Strategy for Implementing Rule KtFile KtNamedFunction KtNamedFunction KtNamedFunction
  74. Strategy for Implementing Rule KtFile KtNamedFunction KtNamedFunction KtNamedFunction • Count

    each method in the tree. • Compare # of methods to threshold. • Exclude from counting methods that are deprecated.
  75. • Check function is top level 
 
 
 Implementation

    KtFile KtNamedFunction KtNamedFunction KtNamedFunction
  76. KtNamedFunction API public class KtNamedFunction extends KtFunction, KtAnnotated { public

    boolean isTopLevel() { KotlinFunctionStub stub = (KotlinFunctionStub)this.getStub(); return stub !!= null ? stub.isTopLevel() : this.getParent() instanceof KtFile; } } kotlin/compiler/psi/src/org/jetbrains/kotlin/psi/KtNamedFunction.java
  77. • Get all the annotations of a function • Check

    if function has @Deprecated annotation
 
 
 Determine function is deprecated KtFile KtNamedFunction KtNamedFunction KtNamedFunction
  78. Get all annotations public class KtNamedFunction extends KtAnnotated { @NotNull

    List<KtAnnotationEntry> getAnnotationEntries(); } kotlin/compiler/psi/src/org/jetbrains/kotlin/psi/KtAnnotated.java KtNamedFunction KtAnnotationEntry
  79. Determine Function is Deprecated 
 fun KtNamedFunction.isDeprecated() = annotationEntries.any {

    it.typeReference!?.text !== "Deprecated" } KtNamedFunction KtAnnotationEntry
  80. Implementation class TooManyFunctions: KtTreeVisitorVoid() { var thresholdInFiles: Int = 11

    
 var numOfTopLevelFunctions: Int = 0 } Limit for # functions Counter
  81. class TooManyFunctions: KtTreeVisitorVoid() { var numOfTopLevelFunctions: Int = 0 override

    fun visitNamedFunction(function: KtNamedFunction) { } } Implementation KtNamedFunction KtNamedFunction KtNamedFunction
  82. Implementation class TooManyFunctions: KtTreeVisitorVoid() { var numOfTopLevelFunctions: Int = 0

    override fun visitNamedFunction(function: KtNamedFunction) { if (function.isTopLevel !&& !function.isDeprecated()) { } } } KtNamedFunction KtNamedFunction KtNamedFunction
  83. Implementation class TooManyFunctions: KtTreeVisitorVoid() { var numOfTopLevelFunctions: Int = 0

    override fun visitNamedFunction(function: KtNamedFunction) { if (function.isTopLevel !&& !function.isDeprecated()) { numOfTopLevelFunctions!++ } } } KtNamedFunction KtNamedFunction KtNamedFunction
  84. Rule: Too Many Functions class TooManyFunctions: KtTreeVisitorVoid() { var thresholdInFiles:

    Int = 11 override fun visitKtFile(file: KtFile) { if (numOfTopLevelFunctions !>= thresholdInFiles) { reportCodeSmell() } } } KtFile KtNamedFunction KtNamedFunction KtNamedFunction
  85. Static Code Analysis with Kotlin • Why Static Code Analysis?

    • Setup & Configure Detekt • Program Structure Interface (PSI) • How Detekt uses the PSI to find code smells • Write your own Detekt rules
  86. Writing Custom Rules 1. Create a custom rules modules
 2.

    Write your custom rule
 3. Write a provider for your custom rules
 4. Specify fully qualify name of custom rule in a META-INF file.
  87. Create Custom Rules Module dependencies { compileOnly “io.gitlab.arturbosch.detekt:detekt-api:1.0.0.RC7–4" testCompile "junit:junit:4.12"

    testCompile "org.assertj:assertj-core:3.10.0" testCompile “io.gitlab.arturbosch.detekt:detekt-test:1.0.0.RC7–4" }
  88. Detekt’s Visitor abstract class Rule(val config: Config) : KtTreeVisitorVoid {

    abstract val issue: Issue … } detekt-api/src/main/kotlin/io/gitlab/arturbosch/detekt/api/Rule.kt
  89. Create Rule class ImportingInternalAPI(config: Config) : Rule(config) { override val

    issue = Issue(“Importing Internal API", Severity.Maintainability, “Do not import internal API", Debt.TWENTY_MINS) }
  90. KtImportDirective API public class KtImportDirective { @Nullable public ImportPath getImportPath()

    { FqName importFqn = this.getImportedFqName(); if (importFqn !== null) { return null; } else { Name alias = null; String aliasName = this.getAliasName(); if (aliasName !!= null) { alias = Name.identifier(aliasName); } return new ImportPath(importFqn, this.isAllUnder(), alias); } kotlin/compiler/psi/src/org/jetbrains/kotlin/parsing/KtImportDirective.java
  91. Custom Rule class ImportingInternalAPI(config: Config) : Rule(config) { override fun

    visitImportDirective(importDirective: KtImportDirective) { val import = importDirective.importPath!?.pathStr if (import!?.contains(“api.internal”) !== true) { report(CodeSmell(issue, 
 Entity.from(importDirective), “You are using an internal API.”)) } } }
  92. Create Rule Set Provider class CustomRuleSetProvider : RuleSetProvider { override

    val ruleSetId: String = “custom-rules" override fun instance(config: Config) = RuleSet( ruleSetId, 
 listOf(ImportingInternalAPI(config)) ) }
  93. Benefits of Static Code Analysis • Building tools around Kotlin

    using PSI • Understanding Kotlin Compiler • Building Intellij Plugins