Slide 1
Slide 1 text
Yan Zhulanow, JetBrains
June 2024
K2 meets the IDE.
So let’s peek inside!
Slide 2
Slide 2 text
Jet
Slide 3
Slide 3 text
Jet
Slide 4
Slide 4 text
Compiler IDE Plugin
Slide 5
Slide 5 text
Compiler IDE Plugin
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
Jetpack Compose
Let’s write UI in Kotlin!
Code tr
a
nsform
a
tion needed
Implement in the new BE
Slide 8
Slide 8 text
Jetpack Compose
Let’s write UI in Kotlin!
Code tr
a
nsform
a
tion needed
Implement in the new BE
Slide 9
Slide 9 text
Gradle Kotlin DSL
Let’s write build scripts in Kotlin!
Not enough script m
a
gic
Design extensible script de
f
initions
Slide 10
Slide 10 text
Gradle Kotlin DSL
Let’s write build scripts in Kotlin!
Not enough script m
a
gic
Design extensible script de
f
initions
Slide 11
Slide 11 text
Sometimes, it goes the hard way
expect/act
u
a
l
binding
Mixed KMP compil
a
tion
Slide 12
Slide 12 text
Sometimes, it goes the hard way
expect/act
u
a
l
binding
Mixed KMP compil
a
tion
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
K2
Slide 15
Slide 15 text
Parser
fu
n main(a
r
gs: A
rr
a
y
) {
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
p
r
int
l
n("He
ll
o, $name!")
}
Slide 16
Slide 16 text
Parser
fu
n main(a
r
gs: A
rr
a
y
) {
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
p
r
int
l
n("He
ll
o, $name!")
}
main a
r
gs: A
rr
a
y
A
rr
a
y
St
r
ing
A
rr
a
y
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s" p
r
int
l
n("He
l l
o, $name!")
name a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
a
r
gs.sing
l
eO
r
N
ull
() "Anon
y
mo
u
s"
a
r
gs sing
l
eO
r
N
ull
p
r
int
l
n "He
ll
o, $name!"
a
r
gs
“He
ll
o, " name
Slide 17
Slide 17 text
Parser
fu
n main(a
r
gs: A
rr
a
y
) {
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
p
r
int
l
n("He
ll
o, $name!")
}
main a
r
gs: A
rr
a
y
A
rr
a
y
St
r
ing
A
rr
a
y
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s" p
r
int
l
n("He
l l
o, $name!")
name a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
a
r
gs.sing
l
eO
r
N
ull
() "Anon
y
mo
u
s"
a
r
gs sing
l
eO
r
N
ull
p
r
int
l
n "He
ll
o, $name!"
a
r
gs
“He
ll
o, " name
fun println(text: String) {
throw RuntimeException(text) // 😈
}
Slide 18
Slide 18 text
Semantic Analyzer
Type resolution
C
a
ll resolution
Type inference
V
a
rious checkers
P
a
rser
Code gener
a
tion
Addition
a
l st
a
tic checks
☕.c
l
ass
Slide 19
Slide 19 text
Compiler IDE Plugin
Slide 20
Slide 20 text
Compiler IDE Plugin
Use the
s
a
me API!
BindingContext
Dec
l
a
r
ationDesc
r
ipto
r
K
ot
l
inT
y
pe
Reso
l
vedCa
ll
ana
l y
ze()
Slide 21
Slide 21 text
Compiler IDE Plugin
BindingContext
Dec
l
a
r
ationDesc
r
ipto
r
K
ot
l
inT
y
pe
Reso
l
vedCa
ll
ana
l y
ze()
Debugger
Ref
a
ctorings
Completion
Inspections
Type Hier
a
rchy
Highlighting
Slide 22
Slide 22 text
Compiler IDE Plugin
BindingContext
Dec
l
a
r
ationDesc
r
ipto
r
K
ot
l
inT
y
pe
Reso
l
vedCa
ll
ana
l y
ze()
Debugger
Ref
a
ctorings
Completion
Inspections
Type Hier
a
rchy
Highlighting
K2
Slide 23
Slide 23 text
Compiler IDE Plugin
K2
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
How compilers work
Lexer P
a
rser
Sem
a
ntic
a
n
a
lyzer
Code
Gener
a
tor
Slide 26
Slide 26 text
How IDEs work
Slide 27
Slide 27 text
Multi-module Compilation
B
a
se
App
Kotlin Stdlib Coroutines
Compose
module
module
libr
a
ry libr
a
ry
libr
a
ry
Compil
a
tion of B
a
se
Kotlin Stdlib Coroutines B
a
se
Compil
a
tion of App
Kotlin Stdlib Coroutines
B
a
se.j
a
r
Compose
App
Slide 28
Slide 28 text
Inside the IDE
B
a
se
App
Kotlin Stdlib Coroutines
Compose
module
module
libr
a
ry libr
a
ry
libr
a
ry
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
Wh
a
t w
a
s wrong with the old IDE?
Slide 31
Slide 31 text
Lazy Compiler
class LazyClassDescriptor(val psi: KtClassOrObject, storageManager: StorageManager) {
val companionObject: ClassDescriptor
by storageManager.createLazyValue(::computeCompanionObject)
private fun computeCompanionObject(): ClassDescriptor { ... }
}
Compiler represent
a
tion of
a
cl
a
ss (simpli
f
ied)
interface StorageManager {
fun createMemoizedFunction(compute: (K) -> V): MemoizedFunctionToNotNull
fun createLazyValue(computable: () -> T): NotNullLazyValue
fun createNullableLazyValue(computable: () -> T?): NullableLazyValue
fun compute(computable: () -> T): T
}
L
a
zy v
a
lue provider (simpli
f
ied)
Slide 32
Slide 32 text
One lock to rule them all
class LockBasedStorageManager(val lock: SimpleLock) : StorageManager {
override fun createLazyValue(computable: () -> T): NotNullLazyValue {
return LockBasedNotNullLazyValue(computable)
}
private inner class LockBasedNotNullLazyValue(val computable: () -> T) : NullableLazyValue {
override fun isComputed(): Boolean { ... }
override fun isComputing(): Boolean { ... }
override fun invoke(): T? {
lock.lock()
try {
// Computation and caching
} finally {
lock.unlock()
}
}
}
}
The only l
a
zy v
a
lue provider implement
a
tion (simpli
f
ied)
Slide 33
Slide 33 text
Here we go again
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
No implicit
l
a
ziness!
Slide 34
Slide 34 text
SUPER_TYPES Phase
sealed class ConeKotlinType : ConeKotlinTypeProjection(), KotlinTypeMarker, TypeArgumentListMarker {
final override val kind: ProjectionKind
get() = ProjectionKind.INVARIANT
abstract val typeArguments: Array
final override val type: ConeKotlinType
get() = this
abstract val nullability: ConeNullability
abstract val attributes: ConeAttributes
final override fun toString(): String {
return renderForDebugging()
}
abstract override fun equals(other: Any?): Boolean
abstract override fun hashCode(): Int
}
Slide 35
Slide 35 text
TYPES Phase
sealed class ConeKotlinType : ConeKotlinTypeProjection(), KotlinTypeMarker, TypeArgumentListMarker {
final override val kind: ProjectionKind
get() = ProjectionKind.INVARIANT
abstract val typeArguments: Array
final override val type: ConeKotlinType
get() = this
abstract val nullability: ConeNullability
abstract val attributes: ConeAttributes
final override fun toString(): String {
return renderForDebugging()
}
abstract override fun equals(other: Any?): Boolean
abstract override fun hashCode(): Int
}
Slide 36
Slide 36 text
BODY_RESOLVE Phase
sealed class ConeKotlinType : ConeKotlinTypeProjection(), KotlinTypeMarker, TypeArgumentListMarker {
final override val kind: ProjectionKind
get() = ProjectionKind.INVARIANT
abstract val typeArguments: Array
final override val type: ConeKotlinType
get() = this
abstract val nullability: ConeNullability
abstract val attributes: ConeAttributes
final override fun toString(): String {
return renderForDebugging()
}
abstract override fun equals(other: Any?): Boolean
abstract override fun hashCode(): Int
}
Slide 37
Slide 37 text
IMPORTS
COMPILER_REQUIRED_ANNOTATIONS
COMPANION_GENERATION
SUPER_TYPES
SEALED_CLASS_INHERITORS
TYPES
STATUS
EXPECT_ACTUAL_MATCHING
CONTRACTS
IMPLICIT_TYPES_BODY_RESOLVE
CONSTANT_EVALUATION
ANNOTATION_ARGUMENTS
BODY_RESOLVE
17.5% 35% 52.5% 70%
Slide 38
Slide 38 text
fun isSubClassOf(subClass: FirClass, superClass: FirClass, useSiteSession: FirSession): Boolean {
subClass.lazyResolveToPhase(FirResolvePhase.SUPER_TYPES)
if (subClass.superConeTypes.any { it.toRegularClassSymbol(useSiteSession) == superClass.symbol }) {
return true
}
return subClass.superConeTypes
.asSequence()
.mapNotNull { it.toRegularClassSymbol(useSiteSession) }
.any { isSubClassOf(it.fir, superClass, useSiteSession) }
}
No implicit laziness
Now super types
a
re
resolved for s
u
bC
l
ass
Slide 39
Slide 39 text
Why
a
re you telling me
a
ll of this?
🤔
🤔
🤔
🤔
🤔
🤔
🤔
Slide 40
Slide 40 text
Debugger
Ref
a
ctorings
Completion
Intentions
Type Hier
a
rchy
Highlighting
C
a
ll Hier
a
rchy
Post
f
ix templ
a
tes
N
a
vig
a
tion
J
a
v
a
to Kotlin Converter
Find Us
a
ges
Quick Fixes
Line M
a
rkers
Slide 41
Slide 41 text
Debugger
Ref
a
ctorings
Completion
Intentions
Type Hier
a
rchy
Highlighting
C
a
ll Hier
a
rchy
Post
f
ix templ
a
tes
N
a
vig
a
tion
J
a
v
a
to Kotlin Converter
Find Us
a
ges
Quick Fixes
Line M
a
rkers
Slide 42
Slide 42 text
Local phases
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
Slide 43
Slide 43 text
Local phases
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
Slide 44
Slide 44 text
Jumping phases
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
Slide 45
Slide 45 text
interface Foo
interface Bar : Foo
SUPER_TYPES Phase
Slide 46
Slide 46 text
interface Foo : Bar
interface Bar : Foo
SUPER_TYPES Phase
Slide 47
Slide 47 text
Jumping phases
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
Slide 48
Slide 48 text
Jumping phases
enum class FirResolvePhase {
RAW_FIR,
IMPORTS,
COMPILER_REQUIRED_ANNOTATIONS,
COMPANION_GENERATION,
SUPER_TYPES,
SEALED_CLASS_INHERITORS,
TYPES,
STATUS,
EXPECT_ACTUAL_MATCHING,
CONTRACTS,
IMPLICIT_TYPES_BODY_RESOLVE,
CONSTANT_EVALUATION,
ANNOTATION_ARGUMENTS,
BODY_RESOLVE
}
100% p
a
r
a
llel!
Slide 49
Slide 49 text
No sem
a
ntic
highlighting
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
No content
Slide 56
Slide 56 text
No content
Slide 57
Slide 57 text
It’s not only about sources
App
Kotlin stdlib
kotlinx.coroutines
Compose
JDK
Ktor
koin
okio
Slide 58
Slide 58 text
//
c
l
ass ve
r
sion 65.0 (65)
//
access
fl
ags 0x21
p
u
b
l
ic c
l
ass test/MainJava {
/ /
compi
l
ed
fr
om: MainJava.java
/ /
access
fl
ags 0x1
p
u
b
l
ic ()V
/ /
access
fl
ags 0x9
p
u
b
l
ic static main([Ljava/
l
ang/St
r
ing;)V
}
Java metadata
Slide 59
Slide 59 text
//
c
l
ass ve
r
sion 52.0 (52)
//
access
fl
ags 0x31
p
u
b
l
ic
f
ina
l
c
l
ass test/Main
K
ot
l
in
K
t {
/ /
compi
l
ed
fr
om: Main
K
ot
l
in.
k
t
@L
k
ot
l
in/Metadata;(
mv={1, 9, 0},
k
=2, xi=48, d1={“\
u
0000\
u
0014\n\
u
0000
..
.
\
u
0006\
u
0006”},
d2={"main", "", "a
r
gs", "", "", "([Ljava/
l
ang/St
r
ing;)V", “Test"}
)
/ /
access
fl
ags 0x19
p
u
b
l
ic
f
ina
l
static main([Ljava/
l
ang/St
r
ing;)V
//
annotab
l
e pa
r
amete
r
co
u
nt: 1 (invisib
l
e)
@Lo
r
g/jetb
r
ains/annotations/NotN
u
ll
;()
/
/
invisib
l
e, pa
r
amete
r
0
}
Kotlin Metadata
Slide 60
Slide 60 text
message T
y
peA
l
ias {
optiona
l
int32
f
l
ags = 1 [de
f
a
ul
t = 6
/*
p
u
b
l
ic, no annotations
*/
];
r
eq
u
i
r
ed int32 name = 2 [(name_id_in_tab
l
e) = t
ru
e];
r
epeated T
y
pePa
r
amete
r
t
y
pe_pa
r
amete
r
= 3;
optiona
l
T
y
pe
u
nde
rly
ing_t
y
pe = 4;
optiona
l
int32
u
nde
rl
y
ing_t
y
pe_id = 5 [(t
y
pe_id_in_tab
l
e) = t
ru
e];
optiona
l
T
y
pe expanded_t
y
pe = 6;
optiona
l
int32 expanded_t
y
pe_id = 7 [(t
y
pe_id_in_tab
l
e) = t
ru
e];
r
epeated Annotation annotation = 8;
/
/
Index into the Ve
r
sionReq
u
i
r
ementTab
l
e
r
epeated int32 ve
r
sion_
r
eq
u
i
r
ement = 31;
Extensions 100 to 199;
}
Protobuf inside
Slide 61
Slide 61 text
Lazy libraries
class DeserializedClassDescriptor(
outerContext: DeserializationContext,
val classProto: ProtoBuf.Class,
nameResolver: NameResolver
) : AbstractClassDescriptor(...), DeserializedDescriptor {
private val primaryConstructor = c.storageManager.createNullableLazyValue { computePrimaryConstructor() }
private fun computePrimaryConstructor(): ClassConstructorDescriptor? {
if (kind.isSingleton) {
return DescriptorFactory.createPrimaryConstructorForObject(this, SourceElement.NO_SOURCE).apply {
returnType = getDefaultType()
}
}
return classProto.constructorList.firstOrNull { !Flags.IS_SECONDARY.get(it.flags) }?.let { constructorProto ->
c.memberDeserializer.loadConstructor(constructorProto, true)
}
}
...
}
Compiler represent
a
tion of
a
libr
a
ry cl
a
ss (simpli
f
ied)
Slide 62
Slide 62 text
From the IntelliJ IDEA document
a
tion
(https://plugins.jetbr
a
ins.com/docs/intellij/stub-indexes.html)
Use the stubs, Luke
Slide 63
Slide 63 text
pac
k
age test
fu
n main(a
r
gs: A
rr
a
y
) {
va
l
name = a
r
gs.sing
l
eO
r
N
ull
()
?:
"Anon
y
mo
u
s"
p
r
int
l
n("He
ll
o, $name!")
}
PAC
K
AGE_DIRECTIVE
REFERENCE_EXPRESSION(name = test)
IMPORT_LIST
FUN(
f
qName = test.main, isExtension =
f
a
l
se, hasB
l
oc
k
Bod
y
=
f
a
l
se)
VALUE_PARAMETER_LIST
VALUE_PARAMETER(name = a
r
gs, hasDe
f
a
u
l
tVa
l
u
e =
f
a
l
se)
TYPE_REFERENCE
USER_TYPE
REFERENCE_EXPRESSION(name = A
rr
a
y
)
TYPE_ARGUMENT_LIST
TYPE_PROJECTION(
k
ind = NONE)
TYPE_REFERENCE
USER_TYPE
REFERENCE_EXPRESSION(name = St
r
ing)
Gener
a
ted stub
Source
f
ile
Slide 64
Slide 64 text
P
P
PP
PPPP
Stubs
Slide 65
Slide 65 text
Stubs
Slide 66
Slide 66 text
Compiler plugins
Compiler plugin
Slide 67
Slide 67 text
Compiler plugins
Compiler plugin
Gr
a
dle plugin
M
a
ven plugin
Slide 68
Slide 68 text
Compiler plugins
Compiler plugin
IDE Plugin
Gr
a
dle plugin
M
a
ven plugin
Slide 69
Slide 69 text
Compiler plugins
Compiler plugin
Slide 70
Slide 70 text
Much more than code analysis
Auto-completion
Ref
a
ctorings
Scripting support
Ev
a
lu
a
te expression
Slide 71
Slide 71 text
How to try?
Slide 72
Slide 72 text
there is one more thing
Slide 73
Slide 73 text
The missing piece
Slide 74
Slide 74 text
Kotlin Analysis API
Provides bin
a
ry comp
a
tibility
Documented
Migr
a
tion guides
Slide 75
Slide 75 text
Kotlin Analysis API
Provides bin
a
ry comp
a
tibility
Documented
Migr
a
tion guides
Coming this summer
Slide 76
Slide 76 text
Beyond the IDE
An
a
lysis API
IDE Plugin
Slide 77
Slide 77 text
Beyond the IDE
An
a
lysis API
IDE Plugin
CLI Tools
Slide 78
Slide 78 text
No content