Tweet
Tweet

Slide 1

Slide 1 text

Making archive IL2C #6-55 2018.11.24 DOTNET600 2018 - KOUJI MATSUI (@KOZY_KEKYO, @KEKYO2)

Slide 2

Slide 2 text

Kouji Matsui – kozy, kekyo • NAGOYA city, AICHI pref., JP • Twitter – @kozy_kekyo, @kekyo2 / Facebook • Self employed (I’m looking for a job) • Microsoft Most Valuable Professional VS and DevTech 2015- • Certified Scrum master / Scrum product owner • Center CLR organizer. • .NET/C#/F#/IL/metaprogramming or like… • Bike rider

Slide 3

Slide 3 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 4

Slide 4 text

Abstract WARNING: ◦ This session contains very complex technology topics reached for LEVEL 600 or above. ◦ Are you ready? ;) How works and aiming for tiny resource requirements by the IL2C? How works AOT (ahead of time compilation) by the IL2C? What’s done, doing and will do the IL2C project?

Slide 5

Slide 5 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 6

Slide 6 text

What’s the IL2C A translator for ECMA-335 CIL/MSIL to C language. F# Code IL2C Target native binary C# Code

Slide 7

Slide 7 text

What’s the IL2C https://github.com/kekyo/IL2C

Slide 8

Slide 8 text

What’s the IL2C IL2C's implementation priorities, we're aiming for: ◦ Better predictability for runtime costs, better human readability for the IL2C translated C source code. ◦ Very tiny footprint requirements, we are thinking about how fit between tiny embedded system and large system with many resources. ◦ Better code/runtime portability, minimum requirements are only C99 compiler. ◦ Better interoperabilities for exist C libraries, we can use standard .NET interop technics (likely P/Invoke.) ◦ Contains seamless building system for major C toolkits, for example: CMake system, Arduino IDE, VC++ ...

Slide 9

Slide 9 text

What’s the IL2C IL2C Assembly (*.dll) C Language (*.c, *.h) C Language (*.c, *.h) Target dev C language compiler (VS, Arduino, mbed and etc…) Target native binary C# Code C# Compiler (Roslyn) F# Compiler (FCS) F# Code Another compiler

Slide 10

Slide 10 text

C language source code CIL / MSIL ECMA-335 specific binaries What’s the IL2C IL2C Assembly (*.dll) C Language (*.c, *.h) C Language (*.c, *.h) Target dev C language compiler (VS, Arduino, mbed and etc…) Target native binary C# Code C# Compiler (Roslyn) F# Compiler (FCS) F# Code

Slide 11

Slide 11 text

Building schemes (aiming for) IL2C Assembly (*.dll) C Language (*.c, *.h) C Language (*.c, *.h) Target dev C language compiler (VS, Arduino, mbed and etc…) Target native binary C# Code C# Compiler (Roslyn) NuGet (*.dll) C Language (*.c, *.h) IL2C Runtime (*.c, *.h) Prebuilt Libraries 3rd party Libraries

Slide 12

Slide 12 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 13

Slide 13 text

The runtime types – primitive and string typedef aliases ◦ stdint.h, stdbool.h, wchar_t, float.h byte short int long sbyte ushort uint ulong uint8_t int16_t int32_t int64_t int8_t uint16_t uint32_t uint64_t float double bool char IntPtr UIntPtr float double bool wchar_t intptr_t uintptr_t

Slide 14

Slide 14 text

The runtime types – primitive and string System.String – variable strage space System_String_VTable System_String (heap) vptr0__ string_body__ “ABCDEFGHIJ\0”

Slide 15

Slide 15 text

The runtime types – primitive and string Constant literal string System_String_VTABLE System_String (.rdata) vptr0__ string_body__ const wchar_t[] (.rdata) “ABCDEFGHIJ\0”

Slide 16

Slide 16 text

IL2C_REF_HEADER (.rdata) The runtime types – primitive and string Constant literal string System_String vptr0__ string_body__ const wchar_t[] (.rdata) “ABCDEFGHIJ\0” pNext type gcMark VTABLE

Slide 17

Slide 17 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 18

Slide 18 text

How works the garbage collector Basic strategy – mark and sweep algorithm (NOT include compaction) Root Root Root 1. Clear mark 2. Set mark 3. Free unmarked

Slide 19

Slide 19 text

How works the garbage collector: Phase 1 IL2C_REF_HEADER (heap) pNext type gcMark IL2C_REF_HEADER (heap) pNext g_pBeginHeader GCMARK_NOMARK GCMARK_NOMARK

Slide 20

Slide 20 text

How works the garbage collector: Phase 2 IL2C_REF_HEADER (heap) pNext type gcMark g_pBeginFrame GCMARK_LIVE GCMARK_LIVE GCMARK_LIVE System_String vptr0__ string_body__ “ABCDEFGHIJ\0”

Slide 21

Slide 21 text

How works the garbage collector: Phase 2 Function1() Function2() Function3() local1 local2 local1 local2 local3 g_pBeginFrame local2 GCMARK_LIVE null null null Step2 details local1 null 2

Slide 22

Slide 22 text

How works the garbage collector: Phase 3 IL2C_REF_HEADER (heap) pNext type gcMark IL2C_REF_HEADER (heap) pNext g_pBeginHeader GCMARK_LIVE GCMARK_NOMARK GCMARK_NOMARK GCMARK_NOMARK GCMARK_LIVE

Slide 23

Slide 23 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 24

Slide 24 text

IL2C_REF_HEADER (heap) System_ValueType The value type / boxing pNext type gcMark vptr0__ System_Int32 System_Int32 Boxed 32bit (4bytes) Bit exactly sizeof(IL2C_REF_HEADER) + sizeof(System_ValueType) + sizeof(System_Int32)

Slide 25

Slide 25 text

IL2C_REF_HEADER (heap) System_ValueType The value type / boxing pNext type gcMark vptr0__ System_Int32 Unboxed (System_Int32*) System_Int32 local 1. Got pointer 2. Dereference (Copy)

Slide 26

Slide 26 text

The value type / boxing Unboxed and NOT copy… Copy-free

Slide 27

Slide 27 text

The value type / boxing Valid? Q1:

Slide 28

Slide 28 text

The value type / boxing Valid? The value type is restricted for marking “virtual” by Roslyn. Therefore we can force virtualize by assigning the interface. Will boxed for call Q2:

Slide 29

Slide 29 text

The value type / boxing Valid? Mutable usage by the virtual method …Boxed? Q3:

Slide 30

Slide 30 text

The value type / boxing : Q1 System_Int32 (System_Int32*) System_Int32_ToString( System_Int32* this) 1. Got pointer 2. Arg0 (this) Copy-free invoking

Slide 31

Slide 31 text

IL2C_REF_HEADER (heap) System_ValueType The value type / boxing : Q3 pNext type gcMark vptr0__ System_Int32 System_Int32_VTABLE (.rdata) offset__ System_Int32_Equals(…) System_Object_Finalize(…) System_Int32_GetHashCode(…) System_Int32_ToString(…) System_Int32_ToString( System_Int32* this) Where’s unboxing?

Slide 32

Slide 32 text

The value type / boxing : Q3 System_Int32_VTABLE (.rdata) offset__ System_Int32_Equals_VFunc(…) System_Object_Finalize(…) System_Int32_GetHashCode_VFunc(…) System_Int32_ToString_VFunc(…) System_Int32_ToString_VFunc( System_ValueType* this) System_Int32_ToString( System_Int32* this) Unboxing (And not copy) We can (have to) manipulate the instance fields

Slide 33

Slide 33 text

The value type / boxing : Q2 System_Int32_IFoo_VTABLE (.rdata) offset__ System_Int32_ToString_VFunc(…) System_Int32_ToString_VFunc( System_ValueType* this) System_Int32_ToString( System_Int32* this) Unboxing (And not copy) We can manipulate the instance fields, but gonna discard

Slide 34

Slide 34 text

The value type / boxing My strategy was: ◦ Q1: The value type methods can access their fields by using unboxed-raw- pointer. (Copy-free access) → ◦ Q2: The interface implementation way can same procedure → We have to copy the instance… ◦ Q3: The value type virtual methods same procedure… → Will fix this problem…

Slide 35

Slide 35 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 36

Slide 36 text

The enum types Q: How implement the enum types will you use the C language? translate C# (IL) C

Slide 37

Slide 37 text

The enum types Q: How implement the enum types will you use the C language? translate C# (IL) C It has 2 problems

Slide 38

Slide 38 text

The enum types A1: The enum symbol names are globally. We have to able to apply different names each enum types… C Combined namespace Combined namespace and each value symbol

Slide 39

Slide 39 text

The enum types A2: In the C language, enum types can’t annotate the storage space. C consoleapplication1.c(3): error C2059: syntax error: ':' In the C++, we can use this syntax

Slide 40

Slide 40 text

The enum types Final results for IL2C way: C The IL2C has to calculate real value at each symbols…

Slide 41

Slide 41 text

The enum types Final results for IL2C way: ◦ Each enum types NOT derived from System.Enumtype. ◦ We can implicitly convert by bidirection both enum types and integer types. Convert from Int32 to Int32EnumType implicitly. C# IL

Slide 42

Slide 42 text

The enum types Convert from Int32 to Int32EnumType implicitly. C

Slide 43

Slide 43 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 44

Slide 44 text

The delegate types The delegate types almost always inherited from System.MulticastDelegate. ◦ System.Delegate type has two member for “Method” and “Target.” ◦ The derived delegate type has the type-safe method named “Invoke.” System.Object System.Delegate System.MulticastDelegate System.EventHandler void Invoke( object sender, EventArgs e) MethodInfo Method; object Target;

Slide 45

Slide 45 text

The delegate types “Method” member is the information for callee. ◦ If callee method is static (not instance method), the “Target” is null. private static void Form_Clicked( object sender, EventArgs e) static method callee method System.Delegate Target Method Target (instance) is nothing null

Slide 46

Slide 46 text

class Bar.Form The delegate types ◦ If callee method is instance (not static method), the “Target” refer to instance. System.Delegate private void Form_Clicked( object sender, EventArgs e) Target Method instance method Target (instance) callee method

Slide 47

Slide 47 text

The delegate types The IL2C way puts the code with instance detection expression. System_EventHandler_Invoke(…) C C# Detect the “Target” is not NULL

Slide 48

Slide 48 text

The delegate types The instance method into value type can assign to delegate and valid. Refer to: public override string System.Int32.ToString() C#

Slide 49

Slide 49 text

The delegate types The instance method into value type can assign to delegate and valid. boxing Resolve for virtual method pointer IL

Slide 50

Slide 50 text

IL2C_REF_HEADER (heap) The delegate types The boxing opcode is very important in this case. Because the “Target” member type is System.Object, so can’t store the native value or managed pointer. value type System.Int32 System.Delegate public override string ToString() Target (System.Object) Method instance method Target (boxed)

Slide 51

Slide 51 text

The delegate types Illustrates, the System.MulticastDelegate can hold multiple delegates into the delegate. The implementation is the list of delegates. It contains delegate[] C#

Slide 52

Slide 52 text

The delegate types Illustrates, the System.MulticastDelegate can contain multiple delegates into one delegate. The implementation is the list of delegates. MulticastDelegate invocationList … delegate[] [0] [1] [2] … System.Delegate … System.Delegate … System.Delegate …

Slide 53

Slide 53 text

IL2C_REF_HEADER (heap) IL2C_REF_HEADER (heap) IL2C_REF_HEADER (heap) IL2C_REF_HEADER IL2C_REF_HEADER (heap) The delegate types We know, these instances came from the heap… MulticastDelegate invocationList … delegate[] [0] [1] [2] … System.Delegate … System.Delegate … System.Delegate … Too many allocation!!

Slide 54

Slide 54 text

The delegate types The IL2C way, all delegate types aggregate into the one implement at “System_Delegate.” It contains list of delegate and it’s NON-array. Most important thing is the delegate storage size is VARIABLE. System_Delegate … … [0] Target [0] Method [1] Target [1] Method private static void Form_Clicked( object sender, EventArgs e) private static void Form_Clicked( object sender, EventArgs e) Variable Invocation table

Slide 55

Slide 55 text

IL2C_REF_HEADER (heap) The delegate types It will reduce heap management cost System_Delegate … … [0] Target [0] Method [1] Target [1] Method private static void Form_Clicked( object sender, EventArgs e) private static void Form_Clicked( object sender, EventArgs e)

Slide 56

Slide 56 text

The delegate types The IL2C’s System_Delegate real declaration: Target instance (or null) Function pointer Invocation list

Slide 57

Slide 57 text

The delegate types Delegate.Combine() and Delegate.Remove() calculate and store invocation table only once allocated delegate instance. C Allocate once Delegate.Combine(…) Combine the invocation tables

Slide 58

Slide 58 text

The delegate types Overall the IL2C translate to the specialized delegate invoker: C Invocation table Invoke by function pointer For all method

Slide 59

Slide 59 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 60

Slide 60 text

How works exceptions What’s the “local unwind?” C#

Slide 61

Slide 61 text

How works exceptions What’s the “local unwind?” C# Only inside for the method

Slide 62

Slide 62 text

How works exceptions What’s the “global unwind?” C#

Slide 63

Slide 63 text

How works exceptions What’s the “global unwind?” C# It’s transition by straddling

Slide 64

Slide 64 text

How works exceptions Important global unwind things: ◦ We have to find the matched exception filter (“catch” type clause.) ◦ If not found, try to find more filters crawling from stack frame top to bottom. Illustrated: Function1() Function2() Function3() Function4() C Function5() null main() Stack frame top Stack frame bottom g_pBeginFrame

Slide 65

Slide 65 text

How works exceptions Does the method contains multiple exception blocks? C#

Slide 66

Slide 66 text

How works exceptions Does the method contains multiple exception blocks? C# Ignored the exception filter

Slide 67

Slide 67 text

How works exceptions The unwind feature is likely stack crawling on the stack frame, but we have to complete exception handling with bit different: ◦ We have to use the “Exception frame.” instead the stack frame. ◦ It’s only instantiate for each exception blocks. Exception block 2 RaiseNestedBlockLocal() null main() Exception block 1 Exception frame 1 Exception frame 2 g_pTopUnwindTarget

Slide 68

Slide 68 text

How works exceptions How works the exception frames on the global unwind? RaiseNestedBlockGlobal() null main() Exception frame 1 g_pTopUnwindTarget RaiseNestedBlockCallee() Exception frame 2 Exception block 1 Exception block 2

Slide 69

Slide 69 text

How works exceptions This is the exception frame and overall usage. C Function3() Exception frame 3 g_pTopUnwindTarget Function1() null main() Exception frame 1 Exception frame 2 Function2() This function doesn’t contain any exception block

Slide 70

Slide 70 text

How works exceptions The IL2C handles exception by the way: 1. Raise exception with strongly-typed instance. 2. The runtime crawl the exception frames from “Top exception frame” to bottom. The “g_pTopUnwindFrame” pointer refers top frame. 3. Call the exception filter function from each exception frames. 4. If found the exception type from exception filter function, IL2C found catchable block. Send the execution context into it (unwind done). 5. If can’t find any cachable block at the bottom frame, the IL2C raise “Unhandled exception” state.

Slide 71

Slide 71 text

How works exceptions What’s the exception filter? C int16_t Function1_ExceptionFilter_0( System_Exception* ex)

Slide 72

Slide 72 text

How works exceptions What’s the exception filter? C Index number for local exception frame Method name “catch (Exception)” Result is filter number (unique in each filter)

Slide 73

Slide 73 text

How works exceptions If a exception block contain multiple caught blocks? C# Contains multiple caught into one exception block

Slide 74

Slide 74 text

How works exceptions If a exception block contain multiple caught blocks? C Result is filter number Series checks exception type

Slide 75

Slide 75 text

How works exceptions The “Filter number” is used for branching identity at the method body. Illustrated bone of the exception block: C Use this filter function The filter number

Slide 76

Slide 76 text

How works exceptions “il2c_try” “il2c_catch” “il2c_end_try” and another exception handling helpers: Declared by C language macro. How works these macros details??

Slide 77

Slide 77 text

How works exceptions C Exception frame declared each try block branch using setjmp() result to try and catch block Saved current execution context

Slide 78

Slide 78 text

How works exceptions C Use longjmp() function. The execution context will unwind to “saved.” How the exception raising side?

Slide 79

Slide 79 text

How works exceptions A lot… a lot of topics for the exception handling, this session uncovererd: ◦ How works “rethrow” feature? ◦ How works nested rethrow feature? ◦ How works nested local exception block? ◦ How works finally block? ◦ How works filter-fault blocks? ◦ The setjmp and longjmp way (named sjlj) is slower. Can we improve by another way? ◦ How works asynchronous exception? (NullReferenceException, ArithmeticExceptionand etc…)

Slide 80

Slide 80 text

Agenda Abstract What’s the IL2C ◦ Building scheme Translation details ◦ The runtime types – primitive and string ◦ How works the garbage collector ◦ The value type / boxing ◦ The enum types ◦ The delegate types ◦ How works exceptions ◦ How works virtual methods (virtual, override and interface implementations)

Slide 81

Slide 81 text

How works virtual methods The runtime type information: IL2C_REF_HEADER (heap) pNext type gcMark System_String vptr0__ string_body__ “ABCDEFGHIJ\0” IL2C_RUNTIME_TYPE_DECL (.rdata) pTypeName flags bodySize baseType vptr0 markTarget interfaceCount … “System.String” (UTF8 string) [System.Object] IL2C_RUNTIME_TYPE_DECL Copy … System_String_VTABLE__ …

Slide 82

Slide 82 text

How works virtual methods C IL2C_TYPE_REFERENCE (objref) IL2C_TYPE_VALUE (value type) IL2C_TYPE_VARIABLE (variable storage size) … The C language not valid using for sizeof() expression via empty storage space type. So this field coverered size is 0.

Slide 83

Slide 83 text

How works virtual methods The virtual method table (VTABLE) works: IL2C_RUNTIME_TYPE_DECL … vptr0 … System_String vptr0__ string_body__ System_String_VTABLE (.rdata) offset__ System_String_Equals(…) System_Object_Finalize(…) System_String_GetHashCode(…) System_String_ToString(…) System_String_GetHashCode( System_String* this) 0

Slide 84

Slide 84 text

How works virtual methods How works invoking virtual method? C#

Slide 85

Slide 85 text

How works virtual methods How works invoking virtual method? C VirtualBaseType_VTABLE__ offset__ GetStringFromInt32(…) … VirtualBaseType_GetStringFromInt32( VirtualBaseType* this, int32_t value) 0 vptr0__

Slide 86

Slide 86 text

How works virtual methods How works invoking virtual method? C VirtualBaseType_VTABLE__ offset__ GetStringFromInt32(…) … 0 VirtualBaseType_GetStringFromInt32( VirtualBaseType* this, int32_t value) Subtract offset from this pointer. But the “offset__” field always 0… arg0 = this - offset__

Slide 87

Slide 87 text

How works virtual methods The interface virtual method table works: C# Interface implemented

Slide 88

Slide 88 text

How works virtual methods The interface virtual method table works: TYPE [VirtualNewAndImplementType] … VirtualNewAndImplementType VirtualNewAndImplementType_VTABLE VirtualNewAndImplementType_GetStringFromInt32( VirtualNewAndImplementType* this) vptr0__ vptr0__ vptr_IInterfaceType1 … … VirtualNewAndImplementType_IInterfaceType1_VTABLE offset__ TYPE [IInterfaceType1] … interfaceType vptrInterface void GetStringFromInt32 () offset__ 0 4 +0 +4 arg0 = this - offset__ The interface VTABLE contains this pointer’s offset. Subtract offset to adjust pointer.

Slide 89

Slide 89 text

How works virtual methods How works invoking interface virtual method? C VirtualNewImplements_IInterfaceType1_VTABLE offset__ GetStringFromInt32(…) … 4 True same calculation both the class VTABLE and interface VTABLE. arg0 = this - offset__ VirtualNewAndImplementType_GetStringFromInt32( VirtualNewAndImplementType* this)

Slide 90

Slide 90 text

How works virtual methods C# Implicit implement

Slide 91

Slide 91 text

How works virtual methods C#

Slide 92

Slide 92 text

How works virtual methods C# 1. Find the interface from mostly derived 2. Listup methods from found to same signature 3. Find the method from mostly overrided and NOT newslot

Slide 93

Slide 93 text

Any question? Thank you for join!! https://github.com/kekyo/IL2C