$30 off During Our Annual Pro Sale. View Details »

Experimenting with the Kotlin Compiler

Experimenting with the Kotlin Compiler

Slides for my talk about the Kotlin Compiler, given at Android Makers 2020.

Jossi Wolf

April 21, 2020
Tweet

More Decks by Jossi Wolf

Other Decks in Programming

Transcript

  1. Jossi Wolf
    @jossiwolf
    Experimenting with
    the Kotlin Compiler

    View Slide

  2. @jossiwolf

    View Slide

  3. @jossiwolf
    # SDKs should be simple

    View Slide

  4. @jossiwolf
    class ChatSdk

    View Slide

  5. @jossiwolf
    interface MessageFeature {
    fun poll(): List
    fun push(message: Message)
    }

    View Slide

  6. @jossiwolf
    class ChatSdk: MessageFeature {
    private val messages = mutableListOf()
    override fun poll(): List = messages
    override fun push(message: Message) {
    messages.add(message)
    }
    }

    View Slide

  7. @jossiwolf
    class SimpleMessenger: MessageFeature {
    private val messages = mutableListOf()
    override fun poll(): List = messages
    override fun push(message: Message) {
    messages.add(message)
    }
    }

    View Slide

  8. @jossiwolf
    class ChatSdk: MessageFeature by SimpleMessenger()

    View Slide

  9. @jossiwolf
    public final class ChatSdk implements MessageFeature

    View Slide

  10. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  11. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  12. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  13. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  14. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  15. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 = new SimpleMessenger();
    @NotNull
    public List poll() {
    return this.$$delegate_0.poll();
    }
    public void push(@NotNull Message message) {
    Intrinsics.checkParameterIsNotNull(message, "message");
    this.$$delegate_0.push(message);
    }
    }

    View Slide

  16. @jossiwolf
    class ChatSdk: MessageFeature by SimpleMessenger()

    View Slide

  17. @jossiwolf
    # Kotlin is open source!

    View Slide

  18. @jossiwolf
    github.com/jetbrains/kotlin

    View Slide

  19. @jossiwolf
    $ git clone
    github.com/JetBrains/kotlin.git

    View Slide

  20. @jossiwolf
    To build Kotlin, you need
    different JDK versions!
    - github.com/jetbrains/kotlin
    #build-environment-requirements

    View Slide

  21. @jossiwolf

    View Slide

  22. @jossiwolf
    # Running Kotlin

    View Slide

  23. @jossiwolf

    View Slide

  24. @jossiwolf

    View Slide

  25. @jossiwolf
    # The Kotlin Compiler

    View Slide

  26. @jossiwolf
    Compiler
    .kt Bytecode

    View Slide

  27. @jossiwolf
    Compiler
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  28. @jossiwolf
    Compiler
    Lexer
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  29. @jossiwolf
    Compiler
    Parser
    Lexer
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  30. @jossiwolf
    Compiler
    Parser
    Lexer Codegen
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  31. @jossiwolf
    Compiler
    Parser
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  32. @jossiwolf
    Compiler
    Parser
    Syntax
    Analyzer
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  33. @jossiwolf
    Compiler
    Parser
    Syntax
    Analyzer
    Creates AST
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  34. @jossiwolf
    Compiler
    Parser
    Syntax
    Analyzer
    Semantic
    Analyzer
    Creates AST
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  35. @jossiwolf
    Abstract Syntax Tree

    View Slide

  36. @jossiwolf
    Abstract Syntax Tree
    @Annotation1("abc")
    // line comment between annotations
    @Annotation2("\${123}")
    fun parse() {}
    https://github.com/kotlinx/ast

    View Slide

  37. @jossiwolf
    Abstract Syntax Tree
    PackageHeader()
    importList
    KlassDeclaration(fun parse)
    KlassAnnotation(Annotation1)
    KlassArgument()
    KlassString
    "abc"
    KlassAnnotation(Annotation2)
    KlassArgument()
    KlassString
    Escape("\$")
    "{123}"
    https://github.com/kotlinx/ast

    View Slide

  38. @jossiwolf
    Compiler
    Parser
    Lexer Codegen
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course
    Creates PSI

    View Slide

  39. @jossiwolf

    View Slide

  40. @jossiwolf
    Compiler
    Parser
    Lexer
    Codegen
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  41. @jossiwolf
    Compiler
    Parser
    Lexer
    Codegen
    Intermediate
    Code
    Generator
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  42. @jossiwolf
    Compiler
    Parser
    Lexer
    Codegen
    Intermediate
    Code
    Generator
    Non-optimised
    IR
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  43. @jossiwolf
    Compiler
    Parser
    Lexer
    Codegen
    Intermediate
    Code
    Generator
    Intermediate
    Code
    Optimizer
    Non-optimised
    IR
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  44. @jossiwolf
    Compiler
    Frontend
    Parser
    Lexer
    Backend
    Codegen
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  45. @jossiwolf

    View Slide

  46. @jossiwolf
    class ChatSdk: MessageFeature by SimpleMessenger()

    View Slide

  47. @jossiwolf

    View Slide

  48. @jossiwolf
    KEEP
    =
    Kotlin Evolution and Enhancement Process

    View Slide

  49. @jossiwolf
    slack.kotlinlang.org
    #language-design

    View Slide

  50. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  51. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    fun swapMessenger() {
    messenger = when (messenger) {
    is SimpleMessenger -> ComplexMessenger()
    is ComplexMessenger -> SimpleMessenger()
    }
    }
    }

    View Slide

  52. @jossiwolf
    lateinit

    View Slide

  53. @jossiwolf
    lateinit

    View Slide

  54. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  55. @jossiwolf
    CMD + Shift + F

    View Slide

  56. @jossiwolf

    View Slide

  57. @jossiwolf
    KtModifierKeywordToken LATEINIT_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("lateinit");
    KtModifierKeywordToken DATA_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("data");
    KtModifierKeywordToken INLINE_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("inline");
    KtModifierKeywordToken NOINLINE_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("noinline");
    KtTokens.java

    View Slide

  58. @jossiwolf
    KtModifierKeywordToken CLASSDELEGATE_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("classdelegate");
    KtModifierKeywordToken LATEINIT_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("lateinit");
    KtModifierKeywordToken DATA_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("data");
    KtModifierKeywordToken INLINE_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("inline");
    KtModifierKeywordToken NOINLINE_KEYWORD =
    KtModifierKeywordToken.softKeywordModifier("noinline");
    KtTokens.java

    View Slide

  59. @jossiwolf
    classdelegate var messenger: MessageFeature

    View Slide

  60. @jossiwolf
    # Codegen

    View Slide

  61. @jossiwolf
    Compiler
    Parser
    Lexer Codegen
    https://github.com/ahinchman1/Kotlin-Compiler-Crash-Course

    View Slide

  62. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  63. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  64. @jossiwolf
    ConstructorCodegen.java

    View Slide

  65. @jossiwolf
    ConstructorCodegen
    #generatePrimaryConstructorImpl

    View Slide

  66. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    // $FF: synthetic field
    private final SimpleMessenger $$delegate_0 =
    new SimpleMessenger();

    View Slide

  67. @jossiwolf
    1. Generate Delegate Expressions

    View Slide

  68. @jossiwolf
    for (specifier: myClass.getSuperTypeListEntries()) {
    if (specifier instanceof KtDelegatedSuperTypeEntry) {
    genCallToDelegatorByExpressionSpecifier(
    iv,
    codegen,
    (KtDelegatedSuperTypeEntry) specifier,
    fieldsInfo
    );
    }
    }

    View Slide

  69. @jossiwolf
    for (specifier: myClass.getSuperTypeListEntries()) {
    if (specifier instanceof KtDelegatedSuperTypeEntry) {
    genCallToDelegatorByExpressionSpecifier(
    iv,
    codegen,
    (KtDelegatedSuperTypeEntry) specifier,
    fieldsInfo
    );
    }
    }

    View Slide

  70. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  71. @jossiwolf
    for (specifier: myClass.getSuperTypeListEntries()) {
    if (specifier instanceof KtDelegatedSuperTypeEntry) {
    genCallToDelegatorByExpressionSpecifier(
    iv,
    codegen,
    (KtDelegatedSuperTypeEntry) specifier,
    fieldsInfo
    );
    }
    }

    View Slide

  72. @jossiwolf
    for (specifier: myClass.getSuperTypeListEntries()) {
    if (specifier instanceof KtDelegatedSuperTypeEntry) {
    genCallToDelegatorByExpressionSpecifier(
    iv,
    codegen,
    (KtDelegatedSuperTypeEntry) specifier,
    fieldsInfo
    );
    }
    }

    View Slide

  73. @jossiwolf
    2. Generate Fields

    View Slide

  74. @jossiwolf
    List parameters =
    constructorDescriptor.getValueParameters();
    for (KtParameter parameter : classBodyCodegen.getPrimaryConstructorParameters()) {
    if (parameter.hasValOrVar()) {
    VariableDescriptor descriptor = parameters.get(curParam);
    Type type = typeMapper.mapType(descriptor);
    iv.load(0, classAsmType);
    iv.load(codegen.myFrameMap.getIndex(descriptor), type);
    PropertyDescriptor propertyDescriptor =
    bindingContext.get(BindingContext.PRIMARY_CONSTRUCTOR_PARAMETER, parameter);
    iv.putfield(classAsmType.getInternalName(),
    context.getFieldName(propertyDescriptor, false), type.getDescriptor());
    }
    curParam++;
    }

    View Slide

  75. @jossiwolf
    3. Generate Member Initializers

    View Slide

  76. @jossiwolf
    if (!isObject(descriptor)) {
    memberCodegen.generateInitializers(
    () -> codegen
    );
    }

    View Slide

  77. @jossiwolf
    1. Generate Delegate Expressions
    2. Generate Fields
    3. Generate Initializers

    View Slide

  78. @jossiwolf
    1. Generate Fields
    2. Generate Delegate Expressions
    3. Generate Initializers

    View Slide

  79. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    classdelegate var messenger: MessageFeature
    }

    View Slide

  80. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    public MessageFeature messenger;
    public final MessageFeature getMessenger() {
    return this.messenger;
    }
    public final void setMessenger(MessageFeature var1) {
    this.messenger = var1;
    }
    public List poll() {
    return this.messenger.poll();
    }
    public void push(Message message) {
    this.messenger.push(message);
    }
    }

    View Slide

  81. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    public MessageFeature messenger;
    public final MessageFeature getMessenger() {
    return this.messenger;
    }
    public final void setMessenger(MessageFeature var1) {
    this.messenger = var1;
    }
    public List poll() {
    return this.messenger.poll();
    }
    public void push(Message message) {
    this.messenger.push(message);
    }
    }

    View Slide

  82. @jossiwolf
    public final class ChatSdk implements MessageFeature {
    public MessageFeature messenger;
    public final MessageFeature getMessenger() {
    return this.messenger;
    }
    public final void setMessenger(MessageFeature var1) {
    this.messenger = var1;
    }
    public List poll() {
    return this.messenger.poll();
    }
    public void push(Message message) {
    this.messenger.push(message);
    }
    }

    View Slide

  83. @jossiwolf
    class ChatSdk: MessageFeature by messenger {
    lateinit classdelegate var messenger: MessageFeature
    }

    View Slide

  84. @jossiwolf
    PSI
    - KotlinStubVersions.kt
    - Lexer
    - KtTokens.java
    Resolution
    - FakeCallableDescriptorForObject.kt
    - Scopes
    - Scopes.kt
    - MemberScope.kt
    - ClassResolutionScopesSupport.kt
    - DelegationValueArgument.java
    - TypeResolver.kt for modifying the scope
    - DescriptorResolver.java
    - BodyResolver.java
    Codegen
    - DelegationFieldsInfo.java
    - ImplementationBodyCodegen.java
    - ConstructorCodegen.java
    - CodegenUtil.kt

    View Slide

  85. @jossiwolf
    IR
    - IrProperty.kt
    Backend
    - MoveCompanionObjectFieldsLowering.kt
    - SharedVariablesManager.kt
    Descriptors
    - JavaPropertyDescriptor.java
    - ValueParameterDescriptor.java
    - VariableDescriptor.java
    - PropertyDescriptorImpl.java
    - ErrorUtils.java
    Serialization Adjustments
    Debugger Tinkering
    Scripting Adjustments

    View Slide

  86. @jossiwolf

    View Slide

  87. @jossiwolf
    github.com/jossiwolf/kotlin/pull/1/files

    View Slide

  88. @jossiwolf
    # Awesome Resources

    View Slide

  89. @jossiwolf
    - github.com/ahinchman1/Kotlin-Compiler-Crash-Course
    - kotlin.github.io/kotlin-spec/
    - github.com/jossiwolf/kotlin/pull/1/files
    - KotlinLang Slack
    - #kontributors
    - #arrow-meta
    - #compiler
    - #language-proposals

    View Slide

  90. Jossi Wolf
    @jossiwolf
    Experimenting with
    the Kotlin Compiler

    View Slide