Virtual Machines

Language virtual machines were designed and implemented with different concepts.

In this talk, we will peek into the implementation of some language VMs (especially CRuby and ZendEngine) to understand the complex internal parts.

#rubyconftw #ruby #cruby

Transcript

1. 7JSUVBM.BDIJOFT
   UIFJSDPNNPOQBSUTBOEXIBUNBLFTUIFNTQFDJBM
   :PBO-JO DT
   
   

   View Slide

2. !DT
   

   View Slide

3. 'FXUIJOHTBCPVUNF
   w0QFO4PVSDF%FWFMPQFSGPSNPSFUIBOZFBST
   w1FSM$1"/.PEVMFNBJOUBJOFS dNPEVMFT
   
   w'PVOEFEd1)1QSPKFDUTPO(JU)VC
   w4UBSUFE(PQSPHSBNNJOHTJODF
   w$POUSJCVUFEQBUDIFTUP(P ;FOE&OHJOF ))7.ʜFUD
   

   View Slide

4. *XBTB1FSMEFWFMPQFS
   

   View Slide

5. *MFBSOFE3VCZ
   JO
   

   View Slide

6. View Slide

7. #VU*EJEO`UHFUBDIBODF
   UPVTFJU
   

   View Slide

8. *pOBMMZXSPUFNZpSTU
   3VCZQSPHSBNEBZTBHP
   

   View Slide

9. View Slide

10. View Slide

11. 5IFZ"SF"MM7JSUVBM
    .BDIJOFT❤
    

    View Slide

12. w8IBU`T7JSUVBM.BDIJOF
    w8IZ%P8F/FFE7JSUVBM.BDIJOF
    w7JSUVBM.BDIJOFTJO3FBM8PSME
    

    View Slide

13. 8IBU`T7JSUVBM.BDIJOF
    

    View Slide

14. In computing, a virtual machine (VM) is an emulation of a computer system.
    Virtual machines are based on computer architectures and provide functionality of a
    physical computer. Their implementations may involve specialized hardware,
    software, or a combination.
    From Wikipedia, the free encyclopedia
    

    View Slide

15. 7JSUVBM.BDIJOF
    WT
    3FBM.BDIJOF
    

    View Slide

16. View Slide

17. $16FYFDVUFTUIFJOTUSVDUJPOT
    

    View Slide

18. CPU Memory
    I/O Controller
    External Storage
    Address & Data Bus
    #BTJD%JHJUBM$PNQVUFS"SDIJUFDUVSF
    

    View Slide

19. Execute
    Instruction
    Fetch
    Instruction
    Decode
    Instruction
    'FUDI%FDPEF&YFDVUF$ZDMF
    

    View Slide

20. View Slide

21. __text:
    100000f08: 55 pushq %rbp
    100000f09: 48 89 e5 movq %rsp, %rbp
    100000f0c: 48 83 c7 68 addq $104, %rdi
    100000f10: 48 83 c6 68 addq $104, %rsi
    100000f14: 5d popq %rbp
    100000f15: e9 62 36 00 00 jmp 13922
    100000f1a: 55 pushq %rbp
    100000f1b: 48 89 e5 movq %rsp, %rbp
    100000f1e: 48 8d 46 68 leaq 104(%rsi), %rax
    100000f22: 48 8d 77 68 leaq 104(%rdi), %rsi
    100000f26: 48 89 c7 movq %rax, %rdi
    100000f29: 5d popq %rbp
    100000f2a: e9 4d 36 00 00 jmp 13901
    100000f2f: 55 ushq %rbp
    100000f30: 48 89 e5 movq %rsp, %rbp
    100000f33: 4c 8b 46 60 movq 96(%rsi), %r8
    100000f37: 48 8b 57 60 movq 96(%rdi), %rdx
    100000f3b: 48 8b 4a 30 movq 48(%rdx), %rcx
    100000f3f: b8 01 00 00 00 movl $1, %eax
    100000f44: 49 39 48 30 cmpq %rcx, 48(%r8)
    100000f48: 7f 1a jg 26
    100000f4a: 7d 07 jge 7
    100000f4c: b8 ff ff ff ff movl $4294967295, %eax
    100000f51: eb 11 jmp 17
    100000f53: 48 8b 4a 38 movq 56(%rdx), %rcx
    100000f57: 49 39 48 38 cmpq %rcx, 56(%r8)
    100000f5b: 7f 07 jg 7
    100000f5d: b8 ff ff ff ff movl $4294967295, %eax
    100000f62: 7d 02 jge 2
    100000f64: 5d popq %rbp
    100000f65: c3 retq
    100000f66: 48 83 c7 68 addq $104, %rdi
    100000f6a: 48 83 c6 68 addq $104, %rsi
    100000f6e: 5d popq %rbp
    100000f6f: e9 08 36 00 00 jmp 13832
    BTTFNCMZDPEF
    NBDIJOFDPEF
    

    View Slide

22. View Slide

23. 5ZQFTPG7JSUVBM.BDIJOF
    

    View Slide

24. 4ZTUFN7JSUVBM.BDIJOF
    

    View Slide

25. 4ZTUFN7JSUVBM.BDIJOFT
    w'VMM7JSUVBMJ[BUJPO7.T
    w)BSEXBSF7JSUVBMJ[BUJPOŠ$16 .FNPSZʜFUD
    w7JSUVBMJ[FE%FWJDFTŠ%JTL /FUXPSL 64#
    ,FZCPBSEʜFUD
    w1IZTJDBM.BDIJOF4JNVMBUJPO
    w5IFpSTUTZTUFNUPBMMPXGVMMWJSUVBMJ[BUJPOŠ
    *#.`T$1$.4
    

    View Slide

26. 1SPDFTT7JSUVBM.BDIJOF
    

    View Slide

27. 1SPDFTT7JSUVBM.BDIJOFT
    w%FTJHOFEUPSVOZPVSQSPHSBNTJOBQMBUGPSN
    JOEFQFOEFOUFOWJSPONFOU
    w"QQMJDBUJPO7JSUVBMJ[BUJPO
    w+7. /&5$-3 1BSSPU7.ʜFUD
    

    View Slide

28. 0DPEF.BSUJO3JDIBSET
    wTŠ.BSUJO
    3JDIBSETJOWFOUFE0
    DPEFFNJUUFECZ#$1-
    w0DPEF 0CKFDUDPEF
    
    w0DPEFWJSUVBMNBDIJOF
    w#$1- #BTJD$PNCJOFE
    1SPHSBNNJOH-BOHVBHF
    
    Martin Richards (born 21 July 1940) is a
    British computer scientist known for his
    development of the BCPL programming
    

    View Slide

29. #$1-BOE#
    w#QSPHSBNNJOHMBOHVBHF
    XBTEFSJWFEGSPN#$1-BOE
    EFWFMPQFEBU#FMM-BCT
    wCZ,FO5IPNQTPOBOE
    %FOOJT3JUDIJF
    

    View Slide

30. 1DPEF/JLMBVT8JSUI
    wTŠQDPEFGPS&VMFS
    QSPHSBNNJOHMBOHVBHF
    wŠ1BTDBM1DPNQJMFS
    HFOFSBUFT1DPEF
    wTŠ#04XBTUIFpSTU
    DSPTTQMBUGPSNPQFSBUJOHTZTUFN
    EFTJHOFEUPSVOQDPEF
    wTŠUSBOTMBUJPOJOUPQDPEF
    CFDBNFBQPQVMBSTUSBUFHZ
    TVDIBT.47# 1ZUIPOʜFUD
    

    View Slide

31. 4NBMM5BMLBOE"MBO,BZ
    wTŠUIFpSTUWFSTJPO
    QVCMJDMZBWBJMBCMF
    4NBMM5BML
    w%FVUTDI4DIJ⒎NBOO
    JNQMFNFOUBUJPOQVTIFEKVTU
    JOUJNF +*5
    DPNQJMBUJPO
    GPSXBSE
    

    View Slide

32. +7.
    w*OUSPEVDFEJO
    w%FWFMPQFECZ4VO
    .JDSPTZTUFNT
    w+BWBCZUFDPEF
    

    View Slide

33. 8IFSFJT1FSM 3VCZ
    BOE1)1
    

    View Slide

34. #FDBVTF5IFZXFSF
    *OUFSQSFUFST
    

    View Slide

35. *OUFSQSFUFS
    WT
    7JSUVBM.BDIJOF
    

    View Slide

36. %J⒎FSFODFTCFUXFFO
    *OUFSQSFUFSBOE7JSUVBM.BDIJOF
    w*OUFSQSFUFSTEPO`UHFOFSBUFVTFCZUFDPEF
    w*OUFSQSFUFSTUSBWFSTFUIF"45OPEFUPFYFDVUFUIF
    MPHJDT
    w3FMBUJWFMZFBTJFSUPCFJNQMFNFOUFEUIBO7JSUVBM
    .BDIJOF
    

    View Slide

37. *OUFSQSFUFST
    8PSLPO"45EJSFDUMZ
    

    View Slide

38. 1FSM
    wŠ1FSMXBTEFWFMPQFE
    BTUIFOFX"8,4&%CZ
    -BSSZ8BMM
    wŠ1FSMXBTSFMFBTFE
    XJUIBDPNQMFUFMZSFXSJUUFO
    WJSUVBMNBDIJOF JUIBTB
    PQUSFFEFTJHO
    
    

    View Slide

39. 1ZUIPO
    wTŠEFTJHOFEBOE
    EFWFMPQFECZ(VJEPWBO
    3PTTVN
    wTŠ$1ZUIPOXBT
    SFMFBTFEXJUIWJSUVBM
    NBDIJOFBOECZUFDPEF
    TVQQPSU
    

    View Slide

40. 3VCZ
    wNJETŠEFTJHOFEBOE
    EFWFMPQFECZ:VLJIJSP.BU[
    w.3*Š.BU[`T3VCZ
    *OUFSQSFUFSCFGPSF3VCZ
    wTŠ:"37 :FU"OPUIFS
    3VCZ7.
    XBTEFWFMPQFECZ
    ,PJDIJ4BTBEB
    

    View Slide

41. 1)1
    w0SJHJOBMMZDSFBUFECZ
    3BTNVT-FSEPSGJO
    TUBSUFEBTBO*OUFSQSFUFSGPS
    $(*
    w;FFWBOE"OEJJOUSPEVDFE
    ;FOE&OHJOF**BU
    wŠ;FOE&OHJOF***GPS
    1)1
    

    View Slide

42. 1BSSPU7.
    w0SJHJOBMMZBJNFEUPTVQQPSU1FSM BOEPUIFSTFWFSBM
    QSPHSBNNJOHMBOHVBHFTMJLF1)1 1ZUIPO 3VCZ +BWB
    -VBʜFUD
    w3BLVEP1FSMVTFE1BSSPU7.BUUIFCFHJOOJOH
    w4UBSUFEBSPVOEdT
    w1BSSPU7.XBTSFMFBTFEJO
    

    View Slide

43. )PX%PFT7JSUVBM
    .BDIJOF8PSL
    

    View Slide

44. TPVSDFGSPNWEFWWJBDD
    

    View Slide

45. )PX5P#VJME"O"45
    

    View Slide

46. 8FOFFEB1BSTFS
    

    View Slide

47. x > 100 ? 'foo' : 'bar'
    X > 0 ? ‘foo’ : ‘bar’
    Lexer
    

    View Slide

48. Parser
    X > 0 ? ‘foo’ : ‘bar’
    

    View Slide

49. "OE(SBNNBS3VMFT
    

    View Slide

50. View Slide

51. expression => expression + term
    | expression - term
    | term
    term => term * factor
    | term / factor
    | factor

    factor => ‘(‘ expression ‘)’

    | NUMBER
    

    View Slide

52. 5PQ%PXO1BSTFS

    PS

    #PUUPNVQ1BSTFS
    

    View Slide

53. 5PQEPXO1BSTFS
    

    View Slide

54. 5PQ%PXO1BSTFS
    TUBSUGSPNIFSF
    

    View Slide

55. 5PQ%PXO1BSTFS
    'BNJMZ5SFF
    w3FDVSTJWF%FTDFOU
    w#BDLUSBDLJOH
    w/PO#BDLUSBDLJOH
    w1SFEJDUJWF1BSTFS
    w--1BSTFS
    

    View Slide

56. #BDLUSBDLJOH1BSTFS
    &9:
    9BCcB
    :C
    lBCzBTJOUIFJOQVU
    The backtracking parsers, like classical parsers and functional parsers, use a
    recursive descent algorithm. But:
    
    If a stream pattern component does not match the current position of the input
    stream, the control is given to the next case of the stream pattern component
    before it. If it is the first stream pattern component, the rule (the stream pattern
    is left and the next rule is tested.
    
    For example, the following grammar:
    
    E -> X Y
    
    X -> a b | a
    
    Y -> b
    
    works, with the backtracking algorithm, for the input "a b".
    
    Parsing with the non-terminal "E", the non-terminal "X" first accepts the input
    "a b" with its first rule. Then when "Y" is called, the parsing fails since nothing
    

    View Slide

57. #BDLUSBDLJOH
    &9:
    9BCcB
    :C
    lBCzBTJOUIFJOQVU
    NBUDI
    

    View Slide

58. #BDLUSBDLJOH
    &9:
    9BCcB
    :C
    lBCzBTJOUIFJOQVU
    bC`OPUGPVOE GBMMCBDLUPUIFTFDPOEDBTFGSPNUIFQBSFOUSVMF
    

    View Slide

59. #BDLUSBDLJOH
    &9:
    9BCcB
    :C
    lBCzBTJOUIFJOQVU
    NBUDI
    

    View Slide

60. #BDLUSBDLJOH
    &9:
    9BCcB
    :C
    lBCzBTJOUIFJOQVU
    NBUDI
    

    View Slide

61. --1BSTFS
    w--NFBOT -FGUUPSJHIU -FGUNPTUEFSJWBUJPO
    
    w-- L
    NFBOTLTZNCPMMPPLBIFBEBUFBDITUFQ
    w--
    NFBOTTZNCPMMPPLBIFBEBUFBDITUFQ
    

    View Slide

62. --1BSTFS
    HSBNNBS
    4ˠ&
    &ˠ &&
    
    &ˠ
    4㱺&
    Generally, there are multiple possibilities when selecting a rule to expand given
    (leftmost) non-terminal. In the previous example of the leftmost derivation, in step
    2 the parser must choose whether to apply rule 2 or rule 3:
    
    

    To be efficient, the parser must be able to make this choice deterministically
    when possible, without backtracking. For some grammars, it can do this by
    peeking on the unread input (without reading). In our example, if the parser
    㱺 &&
    㱺 &&
    &
    㱺 &
    &
    㱺
    &
    㱺
    
    
    JOQVU
    
    
    
    

    View Slide

63. #PUUPNVQ1BSTFS
    

    View Slide

64. A = B + C * 2
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

65. A = B + C * 2
    WBS
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

66. A = B + C * 2
    WBS WBS
    UFSN
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

67. A = B + C * 2
    WBS WBS
    UFSN
    WBS
    GBDUPS
    UFSN
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

68. A = B + C * 2
    WBS WBS
    UFSN
    WBS
    GBDUPS
    DPOTU
    GBDUPS
    UFSN
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

69. A = B + C * 2
    WBS WBS
    UFSN
    WBS
    GBDUPS
    FYQS
    UFSN
    DPOTU
    GBDUPS
    UFSN
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

70. A = B + C * 2
    WBS WBS
    UFSN
    WBS
    GBDUPS
    FYQS
    UFSN
    DPOTU
    GBDUPS
    UFSN
    FYQS
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

71. A = B + C * 2
    WBS
    BTTJHO
    WBS
    UFSN
    WBS
    GBDUPS
    FYQS
    UFSN
    DPOTU
    GBDUPS
    UFSN
    FYQS
    BTTJHOˠWBSFYQS
    FYQSˠUFSNFYQS
    FYQSˠUFSN
    UFSNˠUFSN∗GBDUPS
    UFSNˠGBDUPS
    GBDUPSˠ FYQS
    
    GBDUPSˠDPOTU
    GBDUPSˠWBS
    DPOTUˠJOUFHFS
    

    View Slide

72. -3
    QBSTFUBCMF
    

    View Slide

73. #PUUPNVQQBSTJOHWT5PQEPXOQBSTJOH
    1SPTBOE$POT
    

    View Slide

74. --WT-3
    w--QBSTFSTJTFBTJFSUPVOEFSTUBOEBOEJNQMFNFOU:PVDBOIBOE
    XSJUFSFDVSTJWFEFTDFOUQBSTFSTXJUIDPEFUIBUDMPTFMZNBUDIFTUIF
    HSBNNBS
    w-3QBSTFSTBSFTUSJDUMZNPSFQPXFSGVMUIBO--QBSTFST BOEJO
    BEEJUJPO -"-3QBSTFSTDBOSVOJO0 O
    MJLF--QBSTFST4PZPV
    XPOUpOEBOZGVODUJPOBMBEWBOUBHFTPG--PWFS-3
    w5IVT UIFPOMZBEWBOUBHFPG--JTUIBU-3TUBUFNBDIJOFTBSFRVJUF
    BCJUNPSFDPNQMFYBOEEJ⒏DVMUUPVOEFSTUBOE BOE-3QBSTFST
    UIFNTFMWFTBSFOPUFTQFDJBMMZJOUVJUJWF0OUIFPUIFSIBOE --
    QBSTFSDPEFXIJDIJTBVUPNBUJDBMMZHFOFSBUFEDBOCFWFSZFBTZUP
    VOEFSTUBOEBOEEFCVH
    

    View Slide

75. View Slide

76. 1BSTFS(FOFSBUPS
    

    View Slide

77. expression : expression ‘+’ term { $$ = $1 + $3 }
    | expression ‘-‘ term { $$ = $1 - $3 }
    | term { $$ = $1 }
    ;
    term : term ‘*’ factor { $$ = $1 * $3 }
    | term ‘/‘ factor { $$ = $1 * $4 }
    | factor { $$ = $1 }

    ;

    factor : ‘(‘ expression ‘)’ { $$ = $2 }
    | NUMBER { $$ = $1 }

    | ID { $$ = valueof($1); }
    parser.y
    $POUFYU'SFF(SBNNBSEFpOFEJO#JTPO4ZOUBY
    

    View Slide

78. 1BSTFS(FOFSBUPS
    w#BTI (P ;FOE&OHJOF $3VCZVTFT#JTPO
    w+ZUIPO (SPPWZVTFT"/5-3
    

    View Slide

79. View Slide

80. "MNPTUIBMGPGUIFCPPLJTUBMLJOH
    BCPVUl1BSTFSTzBOEl1BSTFS
    (FOFSBUPSTz
    

    View Slide

81. :PV$BO+VTU
    4LJQ5IFN
    

    View Slide

82. 5IFUFDIOJRVFTBSFNBUVSF
    FOPVHIJOUIFTFEFDBEFT
    

    View Slide

83. *OUVJUJWFBOE
    TUSBJHIUGPSXBSE
    

    View Slide

84. $PEF(FOFSBUJPO
    

    View Slide

85. :PVQSPCBCMZIFBSE
    *3
    

    View Slide

86. *OUFSNFEJBUF
    3FQSFTFOUBUJPO
    

    View Slide

87. )PX$PNQJMFS8PSLT
    Parser
    Intermediate
    Code
    Generator
    Optimizer
    Target
    Code
    Generator
    IR
    AST
    x86
    ARM
    MIPS
    PowerPC
    IR
    /BUJWF$PEF
    

    View Slide

88. --7."SDIJUFDUVSF
    

    View Slide

89. --7.*3
    define i32 @square_unsigned(i32 %a) {
    %1 = mul i32 %a, %a
    ret i32 %1
    }
    

    View Slide

90. 7JSUVBM.BDIJOFTEPO`U
    (FOFSBUF/BUJWF$PEF
    

    View Slide

91. *U&YFDVUFT
    *OUFSNFEJBUF$PEF
    %JSFDUMZ
    

    View Slide

92. *OUFSNFEJBUF$PEF
    
    #ZUF$PEF
    

    View Slide

93. Parser Byte-Code
    Generator
    Byte-code
    Execution
    Byte
    Code
    AST
    

    View Slide

94. Parser Byte-Code
    Generator
    Optimizer
    Byte-code
    Execution
    Byte
    Code
    Byte
    Code
    AST
    XJUICZUFDPEFPQUJNJ[FS
    

    View Slide

95. View Slide

96. View Slide

97. 4P 8IBU`T#ZUFDPEF
    

    View Slide

98. #ZUFDPEFJTQSPHSBNDPEFUIBUIBTCFFODPNQJMFEGSPN
    TPVSDFDPEFJOUPMPXMFWFMDPEFEFTJHOFEGPSBTPGUXBSF
    JOUFSQSFUFS*UNBZCFFYFDVUFECZBWJSUVBMNBDIJOF TVDIBT
    B+7.
    PSGVSUIFSDPNQJMFEJOUPNBDIJOFDPEF XIJDIJT
    SFDPHOJ[FECZUIFQSPDFTTPS
    

    View Slide

99. /BUJWF$PEF
    movl $0x000F, %eax
    YBTNJOTUSVDUJPOJO("4 (/6BTTFNCMFS
    
    

    View Slide

100. +BWB#ZUFDPEF
     iconst_2
     istore_1
     iload_1
     sipush 1000
     if_icmpge 44
     iconst_2
     istore_2
     iload_2
     iload_1
     if_icmpge 31
     iload_1
     iload_2
     irem
     ifne 25
     goto 38
     iinc 2, 1
     

     View Slide

101. ;FOE&OHJOF#ZUFDPEF
     

     View Slide

102. $3VCZ#ZUFDPEF
     

     View Slide

103. %JEZPVpOEUIF
     EJ⒎FSFODFT
     

     View Slide

104. 3FHJTUFSCBTFECZUFDPEF
     WT
     4UBDLCBTFECZUFDPEF
     

     View Slide

105. 4UBDLCBTFE
     7JSUVBM.BDIJOF
     

     View Slide

106. 4UBDLCBTFE#ZUFDPEF
     putobject 2
     Operator Operand
     3VCZ:"37Š*4FR
     *OTUSVDUJPO4FRVFODF
     

     View Slide

107. 4UBDLCBTFE#ZUFDPEF
     Operator Operand
     Operator Operand
     Operator Operand
     Operator Operand
     Operator Operand
     Operator Operand
     1$ QSPHSBNDPVOUFS
     
     

     View Slide

108. 0QFSBOE4UBDL
     101
     101
     "%% SFTVMU
     164)SFTVMU
     4UBDLCBTFEIBTBOFYUSBTUBDLGPSFYFDVUJOHPQFSBUPST
     

     View Slide

109. 3VCZ:"37
     41
     1$
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     

     View Slide

110. 3VCZ:"37
     self
     41
     1$
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     

     View Slide

111. 3VCZ:"37
     2
     self
     41
     1$
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     

     View Slide

112. 3VCZ:"37
     2
     2
     self
     41
     1$
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     

     View Slide

113. 3VCZ:"37
     4
     self
     41
     1$
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     SFUVSOWBMVF
     

     View Slide

114. 3VCZ:"37
     putself
     putobject 2
     putobject 2
     opt_plus
     opt_send_simple leave
     nil
     41
     1$
     SFUVSOWBMVF
     

     View Slide

115. 3FHJTUFSCBTFE
     7JSUVBM.BDIJOF
     

     View Slide

116. 3FHJTUFSCBTFE#ZUFDPEF
     ADD R1, R2, R3
     Operator Operand Operand Result
     5ISFF"EESFTT$PEF
     

     View Slide

117. 3FHJTUFSCBTFE#ZUFDPEF
     1$ QSPHSBNDPVOUFS
     Operator Operand Operand Result
     Operator Operand Operand Result
     Operator Operand Operand Result
     Operator Operand Operand Result
     Operator Operand Operand Result
     Operator Operand Operand Result
     Operator Operand Operand Result
     

     View Slide

118. ;FOE&OHJOF
     R0:
     R1:
     TMP_VAR ~4:
     1$
     ASSIGN !0, 1
     ASSIGN !1, 2
     ADD ~4 !0, !1
     ECHO ~4
     RETURN
     

     View Slide

119. ;FOE&OHJOF
     R0: 1
     R1:
     TMP_VAR ~4:
     1$
     ASSIGN !0, 1
     ASSIGN !1, 2
     ADD ~4 !0, !1
     ECHO ~4
     RETURN
     

     View Slide

120. ;FOE&OHJOF
     R0: 1
     R1: 2
     TMP_VAR ~4:
     1$
     ASSIGN !0, 1
     ASSIGN !1, 2
     ADD ~4 !0, !1
     ECHO ~4
     RETURN
     

     View Slide

121. ;FOE&OHJOF
     R0: 1
     R1: 2
     TMP_VAR ~4: 3
     1$
     ASSIGN !0, 1
     ASSIGN !1, 2
     ADD ~4 !0, !1
     ECHO ~4
     RETURN
     

     View Slide

122. ;FOE&OHJOF
     R0: 1
     R1: 2
     TMP_VAR ~4: 3
     ASSIGN !0, 1
     ASSIGN !1, 2
     ADD ~4 !0, !1
     ECHO ~4
     RETURN
     1$
     

     View Slide

123. 4UBDLCBTFE7.
     WT3FHJTUFSCBTFE7.
     w4UBDLCBTFE7.
     wUIFCZUFDPEFTUSVDUVSFJTDPNQBDU
     wSFMBUJWFMZTNBMM
     wCVUJUOFFETNPSFTUFQTUPCFFYFDVUFE
     w3FHJTUFSCBTFE7.
     wUIFCZUFDPEFTUSVDUVSFJTTJNJMBSUPUIFOBUJWFNBDIJOFJOTUSVDUJPO
     wGFXFSJOTUSVDUJPOTBSFOFFEFEJOTUSVDUJPOTDBOCFFYFDVUFEGBTUFSJOUIFEJTQBUDIMPPQ
     wSFHJTUFSTBSFSFMBUJWFMZFBTJFSUPCFNBQQFEUPUIFOBUJWFNBDIJOFSFHJTUFS BOEXIJDI
     BMTPBMMPXTTPNFDPNQJMFSPQUJNJ[BUJPO
     wSFTVMUTJOUIFSFHJTUFSTDBOCFSFVTFEXIJMFOFFEFE
     

     View Slide

124. %BMWJL7.WT+7.
     w+BWB7.JTTUBDLCBTFE %BMWJLJTSFHJTUFSCBTFE
     w/PUVTJOHUIFTUBDL 3". WJB--DBDIFT
     NBLFT
     %BMWJLTPNFXIBUGBTUFS
     w+BWBCZUFDPEFJTBDUVBMMZNPSFDPNQBDUUIBO%BMWJL
     w+BWBJOTUSVDUJPOTUBLFCZUFT %BMWJLUBLF
     CZUFT JOCZUFNVMUJQMFT
     
     w%BMWJLCZUFDPEFJTNPSFTVJUFEUP"3.BSDIJUFDUVSFT
     

     View Slide

125. 3FHJTUFSCBTFEPS4UBDL
     CBTFE XIJDIPOFJTCFUUFS
     

     View Slide

126. IUUQTXXXVTFOJYPSHMFHBDZFWFOUTWFFGVMM@QBQFSTQZVOIFQEG
     

     View Slide

127. )PX5P(FOFSBUF
     #ZUFDPEF
     

     View Slide

128. $PNQJMFS$PEF(FOFSBUJPO
     w*OTUSVDUJPO4FMFDUJPO
     w*OTUSVDUJPO4DIFEVMJOH
     w3FHJTUFS"MMPDBUJPO
     

     View Slide

129. $3VCZ#ZUFDPEF
     

     View Slide

130. def iterator
     yield 'yield, '
     yield 'blocks,'
     yield 'Ruby'
     end
     iterator {|yeilded| print "use #{yeilded}"}
     iterator.c
     JTFRNFUIPE
     

     View Slide

131. def iterator
     yield 'yield, '
     yield 'blocks,'
     yield 'Ruby'
     end
     iterator {|yeilded| print "use #{yeilded}"}
     iterator.c
     JTFR UPQ
     
     

     View Slide

132. 0002 trace 1
     0004 putself
     0005 putstring "hello, world"
     0007 send :puts, 1, nil, 8,
     0013 trace 16
     rb_block_t
     iseq
     

     View Slide

133. JTFR
     iseq_trace_data
     iseq_link_element
     *prev *next
     custom fields
     iseq_insn_data
     iseq_link_element
     *prev *next
     custom fields
     iseq_insn_data
     iseq_link_element
     *prev *next
     custom fields
     iseq_link_anchor
     anchor *last
     

     View Slide

134. $3VCZ
     iseq_compile_each0(rb_iseq_t *iseq, LINK_ANCHOR *const ret, const NODE *node, int popped)
     switch (type) {
     case NODE_BLOCK:{
     while (node && nd_type(node) == NODE_BLOCK) {
     CHECK(COMPILE_(ret, "BLOCK body", node->nd_head,
     (node->nd_next ? 1 : popped)));
     node = node->nd_next;
     }
     if (node) {
     CHECK(COMPILE_(ret, "BLOCK next", node->nd_next, popped));
     }
     break;
     }
     case NODE_IF:
     case NODE_UNLESS:
     CHECK(compile_if(iseq, ret, node, popped, type));
     break;
     case NODE_CASE:
     CHECK(compile_case(iseq, ret, node, popped));
     break;
     case NODE_CASE2:
     CHECK(compile_case2(iseq, ret, node, popped));
     break;
     

     View Slide

135. $3VCZJTFR
     /* T_IMEMO/iseq */
     /* typedef rb_iseq_t is in method.h */
     struct rb_iseq_struct {
     VALUE flags; /* 1 */
     VALUE wrapper; /* 2 */
     struct rb_iseq_constant_body *body; /* 3 */
     union { /* 4, 5 words */
     struct iseq_compile_data *compile_data; /* used at compile time */
     struct {
     VALUE obj;
     int index;
     } loader;
     struct {
     struct rb_hook_list_struct *local_hooks;
     rb_event_flag_t global_trace_events;
     } exec;
     } aux;
     };
     

     View Slide

136. $3VCZ#ZUFDPEF
     wJTFR JOTUSVDUJPOTFRVFODF
     JTPSHBOJ[FECZTDPQFT
     /0%&@4$01&
     FH UPQ NFUIPE DMBTT CMPDL
     wJTFRJTBEPVCMFMJOLFEMJTU
     w5IFSC@JTFR@DPNQJMF@OPEFGVODUJPOJUFSBUFTUIF"45
     OPEFTBOEHFOFSBUFTUIFSFMBUFESC@JTFR@UTUSVDUVSF
     wJTFRBSF3VCZPCKFDUT UIFZDBOCFHBSCBHFDPMMFDUFE
     

     View Slide

137. ;FOE&OHJOF#ZUFDPEF
     

     View Slide

138. [FOE@PQ@BSSBZ
     zend_op_array
     zend_op *opcodes
     zval *literals
     zend_string **vars
     zend_op
     const void* handler
     zend_uchar opcode
     znode_op op2
     znode_op op1
     znode_op result
     zval
     223
     zval
     abc
     zend_op
     const void* handler
     zend_uchar opcode
     znode_op op2
     znode_op op1
     znode_op result
     zend_op
     const void* handler
     zend_uchar opcode
     znode_op op2
     znode_op op1
     znode_op result
     zval
     abc
     zval
     abc
     

     View Slide

139. [FOE@DPNQJMF@WBS
     zend_op *zend_compile_var(znode *result, zend_ast *ast, uint32_t type, int by_ref) /* {{{ */
     {
     CG(zend_lineno) = zend_ast_get_lineno(ast);
     switch (ast->kind) {
     case ZEND_AST_VAR:
     return zend_compile_simple_var(result, ast, type, 0);
     case ZEND_AST_DIM:
     return zend_compile_dim(result, ast, type);
     case ZEND_AST_PROP:
     return zend_compile_prop(result, ast, type, by_ref);
     case ZEND_AST_STATIC_PROP:
     return zend_compile_static_prop(result, ast, type, by_ref, 0);
     case ZEND_AST_CALL:
     zend_compile_call(result, ast, type);
     return NULL;
     case ZEND_AST_METHOD_CALL:
     zend_compile_method_call(result, ast, type);
     return NULL;
     case ZEND_AST_STATIC_CALL:
     zend_compile_static_call(result, ast, type);
     return NULL;
     case ZEND_AST_ZNODE:
     *result = *zend_ast_get_znode(ast);
     

     View Slide

140. [FOE@PQ
     struct _zend_op {
     const void *handler;
     znode_op op1;
     znode_op op2;
     znode_op result;
     uint32_t extended_value;
     uint32_t lineno;
     zend_uchar opcode;
     zend_uchar op1_type;
     zend_uchar op2_type;
     zend_uchar result_type;
     };
     

     View Slide

141. ;FOE&OHJOF#ZUFDPEF
     w;FOE&OHJOFCZUFDPEFJTBDPNQBDU BMJHOFEPQTUSVDU
     BSSBZ DBMMFE[FOE@PQ@BSSBZ
     w;FOE&OHJOFVTFTQPJOUFSUPJUFSBUFUIFJOTUSVDUJPOT
     w5IF[FOE@PQ@BSSBZTUSVDUBSFBMMPDBUFEEJSFDUMZGSPN
     UIFNFNPSZNBOBHFSUIFZBSF/05HBSCBHF
     DPMMFDUFE
     

     View Slide

142. %VDL5ZQJOH7BMVF
     BOE.FNPSZ.BOBHFNFOU
     

     View Slide

143. $3VCZVTFT37"-6&
     

     View Slide

144. typedef struct RVALUE {
     union {
     struct {
     VALUE flags; /* always 0 for freed obj */
     struct RVALUE *next;
     } free;
     struct RBasic basic;
     struct RObject object;
     struct RClass klass;
     struct RFloat flonum;
     struct RString string;
     struct RArray array;
     struct RRegexp regexp;
     struct RHash hash;
     struct RData data;
     struct RTypedData typeddata;
     struct RStruct rstruct;
     struct RBignum bignum;
     struct RFile file;
     struct RMatch match;
     struct RRational rational;
     struct RComplex complex;
     union { ... } imemo;
     struct {
     struct RBasic basic;
     VALUE v1;
     VALUE v2;
     VALUE v3;
     } values;
     } as;
     } RVALUE;
     
     RVALUE.as.basic
     RVALUE.as.object
     RVALUE.as.klass
     

     View Slide

145. VALUE struct RString
     7"-6&JTUIFQPJOUFSPG3VCZPCKFDUT
     

     View Slide

146. typedef unsigned long VALUE
     8):
     

     View Slide

147. typedef unsigned long VALUE
     32-bit VALUE space
     MSB ------------------------ LSB
     false 00000000000000000000000000000000
     true 00000000000000000000000000000010
     nil 00000000000000000000000000000100
     undef 00000000000000000000000000000110
     symbol ssssssssssssssssssssssss00001110
     object oooooooooooooooooooooooooooooo00
     fixnum fffffffffffffffffffffffffffffff1
     

     View Slide

148. static inline int rb_type(VALUE obj)
     {
     if (RB_IMMEDIATE_P(obj)) {
     if (RB_FIXNUM_P(obj)) return RUBY_T_FIXNUM;
     if (RB_FLONUM_P(obj)) return RUBY_T_FLOAT;
     if (obj == RUBY_Qtrue) return RUBY_T_TRUE;
     if (RB_STATIC_SYM_P(obj)) return RUBY_T_SYMBOL;
     if (obj == RUBY_Qundef) return RUBY_T_UNDEF;
     }
     else if (!RB_TEST(obj)) {
     if (obj == RUBY_Qnil) return RUBY_T_NIL;
     if (obj == RUBY_Qfalse) return RUBY_T_FALSE;
     }
     return RB_BUILTIN_TYPE(obj);
     }
     5:1& PCK
     NBDSPSFUVSOTUIFUZQFPGB7"-6&
     

     View Slide

149. struct RObject {
     struct RBasic basic;
     union {
     struct {
     uint32_t numiv;
     VALUE *ivptr;
     void *iv_index_tbl;
     } heap;
     VALUE ary[ROBJECT_EMBED_LEN_MAX];
     } as;
     };
     TIBSFTUIFTBNFDPNNPOTUSVDUGSPNUIFNFNPSZ
     

     View Slide

150. struct RString {
     struct RBasic basic;
     union {
     struct {
     long len;
     char *ptr;
     union {
     long capa;
     VALUE shared;
     } aux;
     } heap;
     char ary[RSTRING_EMBED_LEN_MAX + 1];
     } as;
     };
     

     View Slide

151. struct RArray {
     struct RBasic basic;
     union {
     struct {
     long len;
     union {
     long capa;
     VALUE shared;
     } aux;
     const VALUE *ptr;
     } heap;
     const VALUE ary[RARRAY_EMBED_LEN_MAX];
     } as;
     };
     

     View Slide

152. System Memory
     Malloc Heap
     Ruby Object Heap
     Allocated via jemalloc
     ObjectSpace
     

     View Slide

153. $3VCZ
     w$3VCZVTFT7"-6&CJUNBTLUPQSFTFOUUIFUZQF
     wJTFR PCKFDUʜBMPUPGUIJOHTBSFVTJOH7"-6&
     w7"-6&TBSFBMMHBSCBHFDPMMFDUFE
     

     View Slide

154. ;FOE&OHJOFVTFT;7"-
     

     View Slide

155. typedef union _zend_value {
     zend_long lval; /* long value */
     double dval; /* double value */
     zend_refcounted *counted;
     zend_string *str;
     zend_array *arr;
     zend_object *obj;
     zend_resource *res;
     zend_reference *ref;
     zend_ast_ref *ast;
     zval *zv;
     void *ptr;
     zend_class_entry *ce;
     zend_function *func;
     struct {
     uint32_t w1;
     uint32_t w2;
     } ww;
     } zend_value;
     

     View Slide

156. struct _zend_string {
     zend_refcounted_h gc;
     zend_ulong h; /* hash value */
     size_t len;
     char val[1];
     };
     TIBSFTUIFTBNFDPNNPOTUSVDUGSPNUIFNFNPSZ
     

     View Slide

157. struct _zend_array {
     zend_refcounted_h gc;
     union {
     struct {
     ZEND_ENDIAN_LOHI_4(
     zend_uchar flags,
     zend_uchar _unused,
     zend_uchar nIteratorsCount,
     zend_uchar _unused2)
     } v;
     uint32_t flags;
     } u;
     uint32_t nTableMask;
     Bucket *arData;
     uint32_t nNumUsed;
     uint32_t nNumOfElements;
     uint32_t nTableSize;
     uint32_t nInternalPointer;
     zend_long nNextFreeElement;
     dtor_func_t pDestructor;
     };
     

     View Slide

158. static zend_always_inline zend_uchar zval_get_type(const zval* pz) {
     return pz->u1.v.type;
     }
     UZQFWBMVFJTTUPSFEJOUIFTUSVDUpFME
     

     View Slide

159. $PMMFDUBCMFBOE3FGDPVOUFE
     refcounted collectable copyable
     simple types
     string x x
     array x x x
     object x x
     resource x
     reference x
     

     View Slide

160. ;7"-
     w;FOE&OHJOFVTFTVDIBSpFMEUPQSFTFOUUIFUZQFPG
     UIFWBMVF
     w/PUBMM[WBMTBSFHBSCBHFDPMMFDUFEFH PQDPEFTBSF
     OPUHBSCBHFDPMMFDUFE
     w5IFTJ[FPG;7"-JTTNBMMFOPVHIUPCFDBDIFEJOUIF
     $16%$BDIF CZUFTQFS[WBM
     
     

     View Slide

161. #ZUFDPEF&YFDVUJPO
     

     View Slide

162. #ZUFDPEF%JTQBUDI-PPQ
     zend_op_array
     zend_op {opcode, op1, op2, result}
     zend_user_opcode_handlers
     ZEND_NOP_SPEC
     ZEND_ADD_SPEC_CONST_CONST
     ZEND_SUB_SPEC_CONST_CONST
     ZEND_MUL_SPEC_CONST_CONST
     ZEND_DIV_SPEC_CONST_CONST
     …
     …
     zend_op {opcode, op1, op2, result}
     zend_op {opcode, op1, op2, result}
     zend_op {opcode, op1, op2, result}
     zend_op {opcode, op1, op2, result}
     Executor Loop
     lookup
     iterate
     

     View Slide

163. YTUBDLGSBNF
     

     View Slide

164. x86 Memory Model
     Stack
     Heap
     Uninitialized data
     Initialized data
     Text
     High address
     Low address
     command-line arguments
     function calls
     dynamic memory allocation (dynamic variable)
     declared variables (uninitialized)
     initialized variables (predefined data)
     program code
     

     View Slide

165. View Slide

166. [FOEFOHJOF
     DBMMGSBNF
     

     View Slide

167. struct _zend_execute_data {
     const zend_op *opline; /* executed opline */
     zend_execute_data *call; /* current call */
     zval *return_value;
     zend_function *func; /* executed function */
     zval This; /* this + call_info + num_args */
     zend_execute_data *prev_execute_data;
     zend_array *symbol_table;
     };
     

     View Slide

168. zend_execute_data
     zval zval …
     function args
     zval
     num
     args
     zval zval …
     compiled vars
     zval zval …
     extra args (optional)
     execution information
     zend_function
     this
     symbol
     table
     return
     value
     …
     16 bytes 16 bytes 16 bytes 16 bytes 16 bytes 16 bytes 16 bytes 16 bytes 16 bytes 16 bytes
     72 bytes
     Grow
     lower address
     0x10000000
     zend_execute_data vars …
     GVODUJPONFUIPEDBMMTBSFTJNVMBUFECZUIFEFpOFE$TUSVDU
     

     View Slide

169. stack call frame
     zend_arena
     page
     zend_vm_stack
     top
     end
     prev
     zend_vm_stack
     top
     end
     prev
     zend_execute_data
     symbol table
     current call
     executed zend function
     return value
     this
     runtime cache
     prev_execute_data
     vars (zval array)
     0
     1
     

     View Slide

170. [FOEFOHJOFDBMMGSBNF
     w5IFTUSVDUJT[FOE@FYFDVUF@EBUB
     w5IFDBMMGSBNFT [FOE@FYFDVUF@EBUB
     BSFBMMPDBUFE
     DPOUJOVPVTMZGSPNUIF7.TUBDL$BMMBSHVNFOUTBSF
     BQQFOEFEBGUFSFBDIDBMMGSBNF
     

     View Slide

171. SVCZDPOUSPMGSBNF
     

     View Slide

172. typedef struct rb_control_frame_struct {
     const VALUE *pc; /* program counter */
     VALUE *sp; /* stack pointer */
     const rb_iseq_t *iseq; /* instruction sequence */
     VALUE self; /* self */
     const VALUE *ep; /* environment pointer */
     const void *block_code; /* iseq or ifunc */
     const VALUE *bp; /* cfp[6] */
     } rb_control_frame_t;
     

     View Slide

173. SC@DPOUSPM@GSBNF
     w5IFSFBSFUXPLJOETPGTUBDLJOUIF$3VCZ7.DBMM
     TUBDLBOEPQFSBOETUBDL JOUFSOBMTUBDL
     
     w4JODF3VCZJTTUBDLCBTFE7. FBDIDPOUSPMGSBNFIBT
     POFPQFSBOETUBDL JOUFSOBMTUBDL
     
     w5IFFOWJSPONFOUQPJOUFSJTVTFEUPSFGFSFODJOHPUIFS
     DPOUSPMGSBNF FH$MPTVSFDBOSFGFSFODFUIFMPDBM
     WBSJBCMFTGSPNUIFQBSFOUTDPQF
     
     

     View Slide

174. #ZUFDPEF#JOBSZ
     

     View Slide

175. #ZUFDPEF$BO#F
     4FSJBMJ[FEBT#JOBSZ
     

     View Slide

176. KBWBD
     Parser Byte-Code
     Generator
     Optimizer
     Byte-code
     Generator
     Byte
     Code
     Byte
     Code
     AST
     Byte Code
     Binary
     

     View Slide

177. .BKPS#JOBSZ'PSNBUT
     VTFEGPS/BUJWF$PEF
     

     View Slide

178. -JOVY&-'
     

     View Slide

179. 8JOEPXT1&
     

     View Slide

180. #JOBSZ'PSNBUT
     w&-'1&BSFGPSNBUTVTFEGPSUIF04QSPHSBN
     MPBEFS5IFGPSNBUDPOUBJOTUIFOBUJWFDPEF UIF
     OBUJWFJOTUSVDUJPOT F H YJOTUSVDUJPOTPSBSN
     JOTUSVDUJPOT
     
     w#ZUFDPEFCJOBSZGPSNBUTBSFVTFEGPS7JSUVBM
     .BDIJOFTFH +7.VTFTDMBTT %BMWJL7.VTFTEY
     %&9GPSNBU
     1ZUIPOVTFTQZD 1&1
     ))7.
     VTFT))#$
     

     View Slide

181. View Slide

182. +7.WT%BMWJL7.
     

     View Slide

183. 4VNNBSZ
     w%J⒎FSFOUMBOHVBHFEFTJHOMFBETUPEJ⒎FSFOU
     BQQSPBDIFT
     w$PNQBDU BMJHOFE DPOUJOVPVTTUSVDUVSFTCSJOHCFUUFS
     F⒏DJFODZ
     w7JSUVBM.BDIJOFTBSFTUJMMFWPMWJOH5IFSFBSFTUJMM
     NBOZDIBMMFOHFTMJLF($ +*5FUD
     

     View Slide

184. 7JSUVBM.BDIJOFT
     "SF'VO
     

     View Slide

185. 5IF-BTU
     

     View Slide

186. View Slide

187. &⒏DJFODZ
     㱺&OFSHZ4BWJOH
     

     View Slide

188. &OFSHZ4BWJOH㱺
     &OWJSPONFOU'SJFOEMZ
     

     View Slide

189. -FU`TIBWFUIJTJONJOE
     BOE#VJMEB#FUUFS8PSME
     

     View Slide