Data Flow Analysis in IntelliJ IDEA

Slide 1

Slide 1 text

Data Flow Analysis in IntelliJ IDEA __ How the IDE Perceives Your Code Tagir Valeev

Slide 2

Slide 2 text

2014-2015: Contributed to FindBugs static analyzer Since 2016: In JetBrains, Java team, data-flow analysis 2

Slide 3

Slide 3 text

Slide 4

Slide 4 text

void visitMethodCall(Call call) { Method m = call.resolve(); if (m.equals(STREAM_FOR_EACH)) { Expression qualifier = call.getQualifier(); if (qualifier instanceof Call) { Method m1 = ((Call) qualifier).resolve(); if (m1.equals(COLLECTION_STREAM)) { reportWarning(call, "stream().forEach() can be replaced with forEach()"); } } } } 4

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Constant conditions & exceptions warnings 6

Slide 7

Slide 7 text

Condition is always true/false 7

Slide 8

Slide 8 text

idx < 4 -> true idx >= 4 -> exception 8

Slide 9

Slide 9 text

> 5 5 if statement if body … 9

Slide 10

Slide 10 text

Slide 11

Slide 11 text

> 3 2 - * 12 * % / 6 19 4 Math.abs -3 11

Slide 12

Slide 12 text

> 3 2 - * 12 * % / 6 19 4 Math.abs -3 =36 =3 =3 =3 =9 =27 =true 12

Slide 13

Slide 13 text

1. PUSH 12 2. PUSH 3 3. BINOP * 4. PUSH 19 5. PUSH 6 6. BINOP / 7. PUSH 4 8. BINOP % 9. PUSH -3 10.CALL Math.abs 11.BINOP * 12.BINOP – 13.PUSH 2 14.BINOP > 13

Slide 14

Slide 14 text

1. PUSH 12 12 2. PUSH 3 12 3 3. BINOP * 36 4. PUSH 19 36 19 5. PUSH 6 36 19 6 6. BINOP / 36 3 7. PUSH 4 36 3 4 8. BINOP % 36 3 9. PUSH -3 36 3 -3 10.CALL Math.abs 36 3 3 11.BINOP * 36 9 12.BINOP – 27 13.PUSH 2 27 2 14.BINOP > true 14

Slide 15

Slide 15 text

1. PUSH x 2. PUSH 5 3. BINOP % 4. PUSH y 5. PUSH 10 6. BINOP % 7. PUSH 20 8. BINOP + 9. BINOP > 15

Slide 16

Slide 16 text

1. PUSH x ? 2. PUSH 5 ? 5 3. BINOP % 4. PUSH y 5. PUSH 10 6. BINOP % 7. PUSH 20 8. BINOP + 9. BINOP > 16

Slide 17

Slide 17 text

1. PUSH x ? 2. PUSH 5 ? 5 3. BINOP % [-4..4] 4. PUSH y 5. PUSH 10 6. BINOP % 7. PUSH 20 8. BINOP + 9. BINOP > 17

Slide 18

Slide 18 text

1. PUSH x ? 2. PUSH 5 ? 5 3. BINOP % [-4..4] 4. PUSH y [-4..4] ? 5. PUSH 10 [-4..4] ? 10 6. BINOP % [-4..4] [-9..9] 7. PUSH 20 [-4..4] [-9..9] 20 8. BINOP + [-4..4] [11..29] 9. BINOP > false 18

Slide 19

Slide 19 text

1. PUSH x ? 2. PUSH 2 ? 2 3. BINOP * [even] 4. PUSH y [even] ? 5. PUSH 4 [even] ? 4 6. BINOP * [even] [even] 7. PUSH 1 [even] [even] 1 8. BINOP + [even] [odd] 9. BINOP == false 19

Slide 20

Slide 20 text

Ctrl+Shift+P (Expression type) Ctrl+Shift+P twice (Advanced expression type) 20

Slide 21

Slide 21 text

Slide 22

Slide 22 text

1. PUSH x 2. PUSH 3 3. ASSIGN 4. POP int x = 3 22

Slide 23

Slide 23 text

1. PUSH x 2. PUSH 3 3. ASSIGN 4. POP 5. PUSH y 6. PUSH x 7. PUSH x 8. BINOP * 9. ASSIGN 10.POP 11.PUSH z 12.PUSH x 13.PUSH 2 14.BINOP * 15.ASSIGN 16.POP 17.PUSH x 18.PUSH z 19.BINOP + 20.PUSH y 21.BINOP > int x = 3 int y = x * x int z = x * 2 x + z > y 23

Slide 24

Slide 24 text

1. PUSH x x 2. PUSH 3 x 3 3. ASSIGN x ; x=3 4. POP ; x=3 5. PUSH y 6. PUSH x 7. PUSH x 8. BINOP * 9. ASSIGN 10.POP 11.PUSH z 12.PUSH x 13.PUSH 2 14.BINOP * 15.ASSIGN 16.POP 17.PUSH x 18.PUSH z 19.BINOP + 20.PUSH y 21.BINOP > 24

Slide 25

Slide 25 text

1. PUSH x x 2. PUSH 3 x 3 3. ASSIGN x ; x=3 4. POP ; x=3 5. PUSH y y ; x=3 6. PUSH x y x ; x=3 7. PUSH x y x x ; x=3 8. BINOP * y 9 ; x=3, y=9 9. ASSIGN y ; x=3, y=9 10.POP ; x=3, y=9 11.PUSH z z ; x=3, y=9 12.PUSH x z x ; x=3, y=9 13.PUSH 2 z x 2 ; x=3, y=9 14.BINOP * z 6 ; x=3, y=9 15.ASSIGN z ; x=3, y=9, z=6 16.POP ; x=3, y=9, z=6 17.PUSH x x ; x=3, y=9, z=6 18.PUSH z x z ; x=3, y=9, z=6 19.BINOP + 9 ; x=3, y=9, z=6 20.PUSH y 9 y ; x=3, y=9, z=6 21.BINOP > false ; x=3, y=9, z=6 25

Slide 26

Slide 26 text

1. PUSH x 2. PUSH 0 3. BINOP < 4. IF_FALSE 10 5. PUSH x 6. PUSH 1 7. BINOP > 8. IF_FALSE 10 9. … 10.END 26

Slide 27

Slide 27 text

1. PUSH x x 2. PUSH 0 x 0 3. BINOP < true ; x < 0 false; x >= 0 4. IF_FALSE 10 5. PUSH x 6. PUSH 1 7. BINOP > 8. IF_FALSE 10 9. … 10.END 27

Slide 28

Slide 28 text

1. PUSH x x 2. PUSH 0 x 0 3. BINOP < true ; x < 0 false; x >= 0 4. IF_FALSE 10 ; x < 0 5. PUSH x 6. PUSH 1 7. BINOP > 8. IF_FALSE 10 9. … 10.END ; x >= 0 28

Slide 29

Slide 29 text

1. PUSH x x 2. PUSH 0 x 0 3. BINOP < true ; x < 0 false; x >= 0 4. IF_FALSE 10 ; x < 0 5. PUSH x x ; x < 0 6. PUSH 1 x 1 ; x < 0 7. BINOP > false; x < 0 // always false! 8. IF_FALSE 10 9. … 10.END ; x < 0 ; x >= 0 29

Slide 30

Slide 30 text

void test(int a, int b, int c, int d, int e, int f, int g, int h) { if (a > 0) System.out.println("a > 0"); // a>0; a<=0 if (b > 0) System.out.println("b > 0"); // a>0, b>0; a>0, b<=0; a<=0, b>0; a<=0, b<=0 if (c > 0) System.out.println("c > 0"); // 8 states if (d > 0) System.out.println("d > 0"); // 16 states if (e > 0) System.out.println("e > 0"); // 32 states if (f > 0) System.out.println("f > 0"); // 64 states if (g > 0) System.out.println("g > 0"); // 128 states if (h > 0) System.out.println("h > 0"); // 256 states } 30

Slide 31

Slide 31 text

; x < 0, y < 0 ; x > 0, y < 0 ; x != 0, y < 0 join 31

Slide 32

Slide 32 text

Slide 33

Slide 33 text

int a = x > 0 ? 1 : 0; // State#1: x > 0, a = 1 // State#2: x <= 0, a = 0 int b = y > 0 ? 1 : 0; // State#1: x > 0, a = 1, y > 0, b = 1 // State#2: x > 0, a = 1, y <= 0, b = 0 // State#3: x <= 0, a = 0, y > 0, b = 1 // State#4: x <= 0, a = 0, y <= 0, b = 0 if (a + b == 2) { // State#1: x > 0, a = 1, y > 0, b = 1 if (x < 0) { ... } } 33

Slide 34

Slide 34 text

Slide 35

Slide 35 text

void printNumbers(int length) { for (int i = 0; i < length; ++i) { System.out.println(i); } } 1. PUSH i 2. PUSH 0 3. ASSIGN 4. POP 5. PUSH i 6. PUSH length 7. BINOP < 8. IF_FALSE 19 9. PUSH System.out 10.PUSH i 11.CALL println 12.PUSH i 13.PUSH i 14.PUSH 1 15.BINOP + 16.ASSIGN 17.POP 18.GOTO 5 19.END int i = 0 i < length ++i 35

Slide 36

Slide 36 text

void printNumbers(int length) { for (int i = 0; i < length; ++i) { System.out.println(i); } } Iteration#1: i = 0; length > 0; Iteration#2: i = 0; length > 0 | i =1; length > 1 Iteration#3: i = 0; length > 0 | i =1; length > 1 | i =2; length > 2 Iteration#4: i = 0; length > 0 | i =1; length > 1 | i =2; length > 2 | i = 3; length > 3 … 36

Slide 37

Slide 37 text

void printNumbers(int length) { for (int i = 0; i < length; ++i) { System.out.println(i); } } Iteration#1: i = 0; length > 0; Iteration#2: i = [0..1]; length > 0 Iteration#3: i = [0..2]; length > 0 Iteration#4: i = [0..3]; length > 0 … 37

Slide 38

Slide 38 text

Type constraints _ 38

Slide 39

Slide 39 text

Type constraints _ 39

Slide 40

Slide 40 text

Type constraints _ 40

Slide 41

Slide 41 text

Nullness _ 41

Slide 42

Slide 42 text

Nullness _ 42

Slide 43

Slide 43 text

Nullness kinds: ✓ not-null ✓ unknown ✓ nullable ✓ null 43

Slide 44

Slide 44 text

A B A B A B A ∪ B join A ∩ B meet Algebraic lattice 44

Slide 45

Slide 45 text

Nullness domain Not-null Null Nullable Unknown ??? 45

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use 50

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Tracking relations _ 52

Slide 53

Slide 53 text

Tracking relations _ 53

Slide 54

Slide 54 text

Tracking relations _ 54

Slide 55

Slide 55 text

Tracking relations _ 55

Slide 56

Slide 56 text

Tracking relations _ 56

Slide 57

Slide 57 text

Fields _ 57

Slide 58

Slide 58 text

Fields _ 58

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Pure methods _ 60

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Purity inference _ 62

Slide 63

Slide 63 text

External annotations _ 63

Slide 64

Slide 64 text

Slide 65

Slide 65 text

External annotations _ 65

Slide 66

Slide 66 text

Locality tracking _ 66

Slide 67

Slide 67 text

Array elements _ 67

Slide 68

Slide 68 text

Array elements _ void test(int[] array, int i) { if (array[0] == array[1]) { array[i] = 10; int diff = array[0] - array[1]; System.out.println(diff); } } 68

Slide 69

Slide 69 text

Array elements _ void test(int[] array, int i) { if (array[0] == array[1] && i >= 2) { array[i] = 10; int diff = array[0] - array[1]; System.out.println(diff); } } 69

Slide 70

Slide 70 text

Array elements _ void test(int[] array, int i, int j) { if (array[i] == array[j]) { int diff = array[i] - array[j]; System.out.println(diff); } } 70

Slide 71

Slide 71 text

public final class Point { private final int x, y; Point(int x, int y) { this.x = x; this.y = y; } public int getX() { return x; } public int getY() { return y; } } Getters _ 71

Slide 72

Slide 72 text

✓ Array length ✓ String length ✓ Collection size (mutable) ✓ Map size (mutable) ✓ Optional value ✓ Boxed value Special fields _ 72

Slide 73

Slide 73 text

Hardcoded contracts _ String.isEmpty(): this.length == 0 -> true; else -> false 73

Slide 74

Slide 74 text

Hardcoded contracts _ String.charAt(int idx): idx >= this.length -> fail; idx < 0 -> fail 74

Slide 75

Slide 75 text

Hardcoded contracts _ String.startsWith(String str): this.length < str.length -> false 75

Slide 76

Slide 76 text

Hardcoded contracts _ String.startsWith(String str): this.length < str.length -> false 76

Slide 77

Slide 77 text

Method handlers _ 77

Slide 78

Slide 78 text

Method handlers _ java.lang.String: contains, indexOf, startsWith, endsWith, lastIndexOf, length, trim, substring, equals, equalsIgnoreCase, charAt, codePointAt, compareTo, replace, toCharArray, valueOf java.lang.Math: abs, sqrt, min, max java.lang.Integer/java.lang.Long: toString, toBinaryString, toHexString, toOctalString, toUnsignedString, compare, compareUnsigned java.lang.Collections: singleton, singletonList, singletonMap, emptyList, emptySet, emptyMap java.util.Arrays: asList, copyOf 78

Slide 79

Slide 79 text

Inliners _ 79

Slide 80

Slide 80 text

Inliners _ 80

Slide 81

Slide 81 text

Method ranges _ 81

Slide 82

Slide 82 text

Method ranges _ 82

Slide 83

Slide 83 text

Mutability _ 83

Slide 84

Slide 84 text

Mutability _ java.util.Collections: java.util.List: 84

Slide 85

Slide 85 text

Common dataflow _ 85

Slide 86

Slide 86 text

Common dataflow _ 86

Slide 87

Slide 87 text

Common dataflow _ 87

Slide 88

Slide 88 text

Common dataflow _ 88

Slide 89

Slide 89 text

Dataflow in quick-fixes _ 89

Slide 90

Slide 90 text

Dataflow in quick-fixes _ 90

Slide 91

Slide 91 text

Dataflow in completion _ 91

Slide 92

Slide 92 text

Dataflow in completion _ 92

Slide 93

Slide 93 text

Dataflow in debugger _ 93

Slide 94

Slide 94 text

Dataflow in debugger _ 94

Slide 95

Slide 95 text

Conclusion _ ✓ Data flow analysis performs abstract interpretation of your code ✓ It tracks constant values, ranges, oddity, types, nullability, mutability, variable equality and so on ✓ It knows about behavior of many library methods ✓ See how data flow analysis works using advanced expression type feature (Ctrl+Shift+P several times) ✓ Check inferred annotations to understand the interprocedural analyser reasoning ✓ Control its behavior by explicit or external annotations such as @Contract, @Nullable, @Range, @UnmodifiableView, etc. ✓ Data flow analysis is everywhere: inspections, quick-fixes, completion, debugger, advanced expression type, refactorings, “dataflow to here”. 95

Slide 96

Slide 96 text

Conclusion _ ✓ Data flow analysis is your friend! ✓ Submit issues at https://youtrack.jetbrains.com/ 96

Slide 97

Slide 97 text

Thanks! Tagir Valeev tagir.valeev@jetbrains.com @tagir_valeev