Tweet
Tweet

Slide 1

Slide 1 text

Jabel – retrofitting Java Compiler by instrumenting it! Sergei Egorov @bsideup

Slide 2

Slide 2 text

About me • Staff Engineer at Pivotal’s Spring R&D, working on Project Reactor ⚛ • Java Champion • Testcontainers co-maintainer • Berlin Spring User Group co-organizer • Developer tools geek @bsideup

Slide 3

Slide 3 text

How compilers work?

Slide 4

Slide 4 text

Very easy! @bsideup

Slide 5

Slide 5 text

Source code @bsideup

Slide 6

Slide 6 text

Source code Scanner @bsideup

Slide 7

Slide 7 text

Source code Scanner Parser @bsideup

Slide 8

Slide 8 text

Source code Scanner Parser Tokens @bsideup

Slide 9

Slide 9 text

Source code Scanner Parser Semantic analyzer Tokens @bsideup

Slide 10

Slide 10 text

Source code Scanner Parser Semantic analyzer Tokens CST
 (concrete syntax tree) @bsideup

Slide 11

Slide 11 text

Front End Source code Scanner Parser Semantic analyzer Tokens CST
 (concrete syntax tree) @bsideup

Slide 12

Slide 12 text

Front End Source code Scanner Parser Semantic analyzer Optimizer Tokens CST
 (concrete syntax tree) @bsideup

Slide 13

Slide 13 text

Front End Source code Scanner Parser Semantic analyzer Optimizer Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 14

Slide 14 text

Middle End Front End Source code Scanner Parser Semantic analyzer Optimizer Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 15

Slide 15 text

Middle End Front End Source code Scanner Parser Semantic analyzer Optimizer code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 16

Slide 16 text

Middle End Back End Front End Source code Scanner Parser Semantic analyzer Optimizer code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 17

Slide 17 text

Middle End Back End Front End Source code Scanner Parser Semantic analyzer Optimizer Intermediate language code code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 18

Slide 18 text

@bsideup

Slide 19

Slide 19 text

@bsideup

Slide 20

Slide 20 text

Middle End Back End Front End Source code Scanner Parser Semantic analyzer Optimizer Intermediate language code code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup ?

Slide 21

Slide 21 text

@bsideup

Slide 22

Slide 22 text

Transpiller @bsideup

Slide 23

Slide 23 text

“Transpilers, or source-to-source compilers, are tools that read source code written in one programming language, and produce the equivalent code in another language that has a similar level of abstraction.” @bsideup

Slide 24

Slide 24 text

What about Lombok?

Slide 25

Slide 25 text

Project Lombok @Value public class ValueExample { String name; @Wither(AccessLevel.PACKAGE) @NonFinal int age; double score; protected String[] tags; @ToString(includeFieldNames = true) @Value(staticConstructor = “of") public static class Exercise { String name; T value; } } @bsideup

Slide 26

Slide 26 text

Project Lombok @Value public class ValueExample { String name; @Wither(AccessLevel.PACKAGE) @NonFinal int age; double score; protected String[] tags; @ToString(includeFieldNames = true) @Value(staticConstructor = “of") public static class Exercise { String name; T value; } } public final class ValueExample { private final String name; private int age; private final double score; protected final String[] tags; @java.beans.ConstructorProperties({"name", "age", "score", public ValueExample(String name, int age, double score, Str this.name = name; this.age = age; this.score = score; this.tags = tags; } public String getName() { return this.name; } public int getAge() { return this.age; } @bsideup

Slide 27

Slide 27 text

} public double getScore() { return this.score; } public String[] getTags() { return this.tags; } @java.lang.Override public boolean equals(Object o) { if (o == this) return true; if (!(o instanceof ValueExample)) return false; final ValueExample other = (ValueExample)o; final Object this$name = this.getName(); final Object other$name = other.getName(); if (this$name == null ? other$name != null : !this$name if (this.getAge() != other.getAge()) return false; if (Double.compare(this.getScore(), other.getScore()) ! if (!Arrays.deepEquals(this.getTags(), other.getTags()) return true; } @java.lang.Override public int hashCode() { final int PRIME = 59; int result = 1; final Object $name = this.getName(); result = result * PRIME + ($name == null ? 43 : $name.h result = result * PRIME + this.getAge(); final long $score = Double.doubleToLongBits(this.getSco result = result * PRIME + (int)($score >>> 32 ^ $score) result = result * PRIME + Arrays.deepHashCode(this.getT return result; } Project Lombok @bsideup @Value public class ValueExample { String name; @Wither(AccessLevel.PACKAGE) @NonFinal int age; double score; protected String[] tags; @ToString(includeFieldNames = true) @Value(staticConstructor = “of") public static class Exercise { String name; T value; } } o_o

Slide 28

Slide 28 text

int result = 1; final Object $name = this.getName(); result = result * PRIME + ($name == null ? 43 : $name.h result = result * PRIME + this.getAge(); final long $score = Double.doubleToLongBits(this.getSco result = result * PRIME + (int)($score >>> 32 ^ $score) result = result * PRIME + Arrays.deepHashCode(this.getT return result; } @java.lang.Override public String toString() { return "ValueExample(name=" + getName() + ", age=" + ge Arrays.deepToString(getTags()) + ")"; } ValueExample withAge(int age) { return this.age == age ? this : new ValueExample(name, } public static final class Exercise { private final String name; private final T value; private Exercise(String name, T value) { this.name = name; this.value = value; } public static Exercise of(String name, T value) return new Exercise(name, value); } public String getName() { return this.name; } public T getValue() { Project Lombok @bsideup @Value public class ValueExample { String name; @Wither(AccessLevel.PACKAGE) @NonFinal int age; double score; protected String[] tags; @ToString(includeFieldNames = true) @Value(staticConstructor = “of") public static class Exercise { String name; T value; } } o_O

Slide 29

Slide 29 text

} public T getValue() { return this.value; } @java.lang.Override public boolean equals(Object o) { if (o == this) return true; if (!(o instanceof ValueExample.Exercise)) return f final Exercise> other = (Exercise>)o; final Object this$name = this.getName(); final Object other$name = other.getName(); if (this$name == null ? other$name != null : !this$ final Object this$value = this.getValue(); final Object other$value = other.getValue(); if (this$value == null ? other$value != null : !thi return true; } @java.lang.Override public int hashCode() { final int PRIME = 59; int result = 1; final Object $name = this.getName(); result = result * PRIME + ($name == null ? 43 : $na final Object $value = this.getValue(); result = result * PRIME + ($value == null ? 43 : $v return result; } @java.lang.Override public String toString() { return "ValueExample.Exercise(name=" + getName() + } } } Project Lombok @bsideup @Value public class ValueExample { String name; @Wither(AccessLevel.PACKAGE) @NonFinal int age; double score; protected String[] tags; @ToString(includeFieldNames = true) @Value(staticConstructor = “of") public static class Exercise { String name; T value; } } O_O

Slide 30

Slide 30 text

Is it a transpiler?

Slide 31

Slide 31 text

It could be, but…

Slide 32

Slide 32 text

Middle End Back End Front End Source code Scanner Parser Semantic analyzer Optimizer Intermediate language code code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 33

Slide 33 text

Middle End Back End Front End Source code Scanner Parser Semantic analyzer Optimizer Intermediate language code code generator Tokens CST
 (concrete syntax tree) AST
 (abstract syntax tree) @bsideup

Slide 34

Slide 34 text

The Hitchhiker's Guide to javac

Slide 35

Slide 35 text

@bsideup https://openjdk.java.net/groups/compiler/doc/hhgtjavac/index.html

Slide 36

Slide 36 text

@bsideup

Slide 37

Slide 37 text

@bsideup

Slide 38

Slide 38 text

@bsideup

Slide 39

Slide 39 text

@bsideup

Slide 40

Slide 40 text

Don’t confuse with…

Slide 41

Slide 41 text

@bsideup

Slide 42

Slide 42 text

Middle End Optimizer @bsideup

Slide 43

Slide 43 text

Why?

Slide 44

Slide 44 text

@bsideup

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

Lower example var result = switch (args.length) { case 1 -> { yield "one"; } case 2, 3 -> "two or three"; default -> "idk"; }; @bsideup

Slide 48

Slide 48 text

Lower example var result = switch (args.length) { case 1 -> { yield "one"; } case 2, 3 -> "two or three"; default -> "idk"; }; String innerPublic = null; switch (args.length) { case 1: { innerPublic = "one"; break; } case 2: case 3: { innerPublic = "two or three"; break; } default: { innerPublic = "idk"; break; } } final String result = innerPublic; @bsideup

Slide 49

Slide 49 text

What can it “lower”? • Switch expressions • “var” • Try-with-resources • Pattern matching in “instanceof” • Even records!* @bsideup

Slide 50

Slide 50 text

@bsideup

Slide 51

Slide 51 text

How it decides when to lower?

Slide 52

Slide 52 text

It doesn’t!

Slide 53

Slide 53 text

But…

Slide 54

Slide 54 text

/** * Models a feature of the Java programming language. Each feature can be associated with a * minimum source level, a maximum source level and a diagnostic fragment describing the feature, * which is used to generate error messages of the kind {@code feature XYZ not supported in source N}. */ public enum Feature { DIAMOND(JDK7, Fragments.FeatureDiamond, DiagKind.NORMAL), MODULES(JDK9, Fragments.FeatureModules, DiagKind.PLURAL), EFFECTIVELY_FINAL_VARIABLES_IN_TRY_WITH_RESOURCES(JDK9, Fragments.FeatureVarInTryWithResources, DiagKind.PLURAL), DEPRECATION_ON_IMPORT(MIN, JDK8), POLY(JDK8), LAMBDA(JDK8, Fragments.FeatureLambda, DiagKind.PLURAL), METHOD_REFERENCES(JDK8, Fragments.FeatureMethodReferences, DiagKind.PLURAL), DEFAULT_METHODS(JDK8, Fragments.FeatureDefaultMethods, DiagKind.PLURAL), STATIC_INTERFACE_METHODS(JDK8, Fragments.FeatureStaticIntfMethods, DiagKind.PLURAL), STATIC_INTERFACE_METHODS_INVOKE(JDK8, Fragments.FeatureStaticIntfMethodInvoke, DiagKind.PLURAL), STRICT_METHOD_CLASH_CHECK(JDK8), EFFECTIVELY_FINAL_IN_INNER_CLASSES(JDK8), TYPE_ANNOTATIONS(JDK8, Fragments.FeatureTypeAnnotations, DiagKind.PLURAL), ANNOTATIONS_AFTER_TYPE_PARAMS(JDK8, Fragments.FeatureAnnotationsAfterTypeParams, DiagKind.PLURAL), REPEATED_ANNOTATIONS(JDK8, Fragments.FeatureRepeatableAnnotations, DiagKind.PLURAL), INTERSECTION_TYPES_IN_CAST(JDK8, Fragments.FeatureIntersectionTypesInCast, DiagKind.PLURAL), GRAPH_INFERENCE(JDK8), FUNCTIONAL_INTERFACE_MOST_SPECIFIC(JDK8), POST_APPLICABILITY_VARARGS_ACCESS_CHECK(JDK8), MAP_CAPTURES_TO_BOUNDS(MIN, JDK7), PRIVATE_SAFE_VARARGS(JDK9), DIAMOND_WITH_ANONYMOUS_CLASS_CREATION(JDK9, Fragments.FeatureDiamondAndAnonClass, DiagKind.NORMAL), UNDERSCORE_IDENTIFIER(MIN, JDK8), PRIVATE_INTERFACE_METHODS(JDK9, Fragments.FeaturePrivateIntfMethods, DiagKind.PLURAL), LOCAL_VARIABLE_TYPE_INFERENCE(JDK10), VAR_SYNTAX_IMPLICIT_LAMBDAS(JDK11, Fragments.FeatureVarSyntaxInImplicitLambda, DiagKind.PLURAL), IMPORT_ON_DEMAND_OBSERVABLE_PACKAGES(JDK1_2, JDK8), SWITCH_MULTIPLE_CASE_LABELS(JDK14, Fragments.FeatureMultipleCaseLabels, DiagKind.PLURAL), SWITCH_RULE(JDK14, Fragments.FeatureSwitchRules, DiagKind.PLURAL), SWITCH_EXPRESSION(JDK14, Fragments.FeatureSwitchExpressions, DiagKind.PLURAL), TEXT_BLOCKS(JDK14, Fragments.FeatureTextBlocks, DiagKind.PLURAL), PATTERN_MATCHING_IN_INSTANCEOF(JDK14, Fragments.FeaturePatternMatchingInstanceof, DiagKind.NORMAL), REIFIABLE_TYPES_INSTANCEOF(JDK14, Fragments.FeatureReifiableTypesInstanceof, DiagKind.PLURAL), RECORDS(JDK14, Fragments.FeatureRecords, DiagKind.PLURAL), ;

Slide 55

Slide 55 text

PRIVATE_SAFE_VARARGS(JDK9), DIAMOND_WITH_ANONYMOUS_CLASS_CREATION(JDK9, Fragments.FeatureDiamondAndAnonClass, DiagKind.NORMAL), UNDERSCORE_IDENTIFIER(MIN, JDK8), PRIVATE_INTERFACE_METHODS(JDK9, Fragments.FeaturePrivateIntfMethods, DiagKind.PLURAL), LOCAL_VARIABLE_TYPE_INFERENCE(JDK10), VAR_SYNTAX_IMPLICIT_LAMBDAS(JDK11, Fragments.FeatureVarSyntaxInImplicitLambda, DiagKind.PLURAL), IMPORT_ON_DEMAND_OBSERVABLE_PACKAGES(JDK1_2, JDK8), SWITCH_MULTIPLE_CASE_LABELS(JDK14, Fragments.FeatureMultipleCaseLabels, DiagKind.PLURAL), SWITCH_RULE(JDK14, Fragments.FeatureSwitchRules, DiagKind.PLURAL), SWITCH_EXPRESSION(JDK14, Fragments.FeatureSwitchExpressions, DiagKind.PLURAL), TEXT_BLOCKS(JDK14, Fragments.FeatureTextBlocks, DiagKind.PLURAL), PATTERN_MATCHING_IN_INSTANCEOF(JDK14, Fragments.FeaturePatternMatchingInstanceof, DiagKind.NORMAL), REIFIABLE_TYPES_INSTANCEOF(JDK14, Fragments.FeatureReifiableTypesInstanceof, DiagKind.PLURAL), RECORDS(JDK14, Fragments.FeatureRecords, DiagKind.PLURAL),

Slide 56

Slide 56 text

PRIVATE_SAFE_VARARGS(JDK9), DIAMOND_WITH_ANONYMOUS_CLASS_CREATION(JDK9, Fragments.FeatureDiamondAndAnonClass, DiagKind.NORMAL), UNDERSCORE_IDENTIFIER(MIN, JDK8), PRIVATE_INTERFACE_METHODS(JDK9, Fragments.FeaturePrivateIntfMethods, DiagKind.PLURAL), LOCAL_VARIABLE_TYPE_INFERENCE(JDK10), VAR_SYNTAX_IMPLICIT_LAMBDAS(JDK11, Fragments.FeatureVarSyntaxInImplicitLambda, DiagKind.PLURAL), IMPORT_ON_DEMAND_OBSERVABLE_PACKAGES(JDK1_2, JDK8), SWITCH_MULTIPLE_CASE_LABELS(JDK14, Fragments.FeatureMultipleCaseLabels, DiagKind.PLURAL), SWITCH_RULE(JDK14, Fragments.FeatureSwitchRules, DiagKind.PLURAL), SWITCH_EXPRESSION(JDK14, Fragments.FeatureSwitchExpressions, DiagKind.PLURAL), TEXT_BLOCKS(JDK14, Fragments.FeatureTextBlocks, DiagKind.PLURAL), PATTERN_MATCHING_IN_INSTANCEOF(JDK14, Fragments.FeaturePatternMatchingInstanceof, DiagKind.NORMAL), REIFIABLE_TYPES_INSTANCEOF(JDK14, Fragments.FeatureReifiableTypesInstanceof, DiagKind.PLURAL), RECORDS(JDK14, Fragments.FeatureRecords, DiagKind.PLURAL),

Slide 57

Slide 57 text

PRIVATE_SAFE_VARARGS(JDK9), DIAMOND_WITH_ANONYMOUS_CLASS_CREATION(JDK9, Fragments.FeatureDiamondAndAnonClass, DiagKind.NORMAL), UNDERSCORE_IDENTIFIER(MIN, JDK8), PRIVATE_INTERFACE_METHODS(JDK9, Fragments.FeaturePrivateIntfMethods, DiagKind.PLURAL), LOCAL_VARIABLE_TYPE_INFERENCE(JDK10), VAR_SYNTAX_IMPLICIT_LAMBDAS(JDK11, Fragments.FeatureVarSyntaxInImplicitLambda, DiagKind.PLURAL), IMPORT_ON_DEMAND_OBSERVABLE_PACKAGES(JDK1_2, JDK8), SWITCH_MULTIPLE_CASE_LABELS(JDK14, Fragments.FeatureMultipleCaseLabels, DiagKind.PLURAL), SWITCH_RULE(JDK14, Fragments.FeatureSwitchRules, DiagKind.PLURAL), SWITCH_EXPRESSION(JDK14, Fragments.FeatureSwitchExpressions, DiagKind.PLURAL), TEXT_BLOCKS(JDK14, Fragments.FeatureTextBlocks, DiagKind.PLURAL), PATTERN_MATCHING_IN_INSTANCEOF(JDK14, Fragments.FeaturePatternMatchingInstanceof, DiagKind.NORMAL), REIFIABLE_TYPES_INSTANCEOF(JDK14, Fragments.FeatureReifiableTypesInstanceof, DiagKind.PLURAL), RECORDS(JDK14, Fragments.FeatureRecords, DiagKind.PLURAL), What if we…

Slide 58

Slide 58 text

PRIVATE_SAFE_VARARGS(JDK8), DIAMOND_WITH_ANONYMOUS_CLASS_CREATION(JDK8, Fragments.FeatureDiamondAndAnonClass, DiagKind.NORMAL), UNDERSCORE_IDENTIFIER(MIN, JDK8), PRIVATE_INTERFACE_METHODS(JDK9, Fragments.FeaturePrivateIntfMethods, DiagKind.PLURAL), LOCAL_VARIABLE_TYPE_INFERENCE(JDK8), VAR_SYNTAX_IMPLICIT_LAMBDAS(JDK8, Fragments.FeatureVarSyntaxInImplicitLambda, DiagKind.PLURAL), IMPORT_ON_DEMAND_OBSERVABLE_PACKAGES(JDK1_2, JDK8), SWITCH_MULTIPLE_CASE_LABELS(JDK8, Fragments.FeatureMultipleCaseLabels, DiagKind.PLURAL), SWITCH_RULE(JDK8, Fragments.FeatureSwitchRules, DiagKind.PLURAL), SWITCH_EXPRESSION(JDK8, Fragments.FeatureSwitchExpressions, DiagKind.PLURAL), TEXT_BLOCKS(JDK8, Fragments.FeatureTextBlocks, DiagKind.PLURAL), PATTERN_MATCHING_IN_INSTANCEOF(JDK8, Fragments.FeaturePatternMatchingInstanceof, DiagKind.NORMAL), REIFIABLE_TYPES_INSTANCEOF(JDK8, Fragments.FeatureReifiableTypesInstanceof, DiagKind.PLURAL), RECORDS(JDK8, Fragments.FeatureRecords, DiagKind.PLURAL),

Slide 59

Slide 59 text

How?

Slide 60

Slide 60 text

let’s instrument it!

Slide 61

Slide 61 text

void checkSourceLevel(Feature feature) { checkSourceLevel(token.pos, feature); } @bsideup

Slide 62

Slide 62 text

void checkSourceLevel(Feature feature) { if (feature.allowedInSource(Source.JDK8)) { feature = Source.Feature.LAMBDA; } checkSourceLevel(token.pos, feature); } @bsideup

Slide 63

Slide 63 text

Field field = Source.Feature.class.getDeclaredField("minLevel"); field.setAccessible(true); for (Source.Feature feature : ENABLED_FEATURES) { field.set(feature, Source.JDK8); if (!feature.allowedInSource(Source.JDK8)) { throw new IllegalStateException(feature.name() + " minLevel instrumentation failed!"); } } @bsideup

Slide 64

Slide 64 text

Field field = Source.Feature.class.getDeclaredField("minLevel"); field.setAccessible(true); for (Source.Feature feature : ENABLED_FEATURES) { field.set(feature, Source.JDK8); if (!feature.allowedInSource(Source.JDK8)) { throw new IllegalStateException(feature.name() + " minLevel instrumentation failed!"); } } @bsideup

Slide 65

Slide 65 text

Set ENABLED_FEATURES = Stream .of( "PRIVATE_SAFE_VARARGS", "SWITCH_EXPRESSION", "SWITCH_RULE", "SWITCH_MULTIPLE_CASE_LABELS", "LOCAL_VARIABLE_TYPE_INFERENCE", "VAR_SYNTAX_IMPLICIT_LAMBDAS", "DIAMOND_WITH_ANONYMOUS_CLASS_CREATION", "EFFECTIVELY_FINAL_VARIABLES_IN_TRY_WITH_RESOURCES", "TEXT_BLOCKS", "PATTERN_MATCHING_IN_INSTANCEOF", "REIFIABLE_TYPES_INSTANCEOF", "RECORDS" ) .map(name -> { try { return Source.Feature.valueOf(name); } catch (IllegalArgumentException e) { return null; } }) .filter(Objects::nonNull) .collect(Collectors.toSet()); @bsideup

Slide 66

Slide 66 text

Demo

Slide 67

Slide 67 text

Takeaways • http://github.com/bsideup/jabel • Easy to integrate • Reuses compiler’s code (except for records) • Pretty safe to use
 @bsideup

Slide 68

Slide 68 text

@bsideup bsideup