ざっくり学ぶ言語のしくみ

Transcript

1. ͬ͘͟ΓֶͿݴޠͷ͘͠Έ Rails Developers Meetup 2017 2017/12/09

2. ࣗݾ঺հ • ൘૔ୡ໵ʢ@itkrt2yʣ • λέϢʔɾ΢Σϒגࣜձࣾ • Railsͷडୗձࣾ • ϑϧϦϞʔτɺϑϧϑϨοΫε •

   ઈࢍ࠾༻தʂʂڵຯ͋Δํ͸ޙͰ੠Λ͔͚͍ͯͩ͘͞

3. ࠓ೔ͷ࿩

4. ݴޠॲཧܥ

5. ͜͜Β΁Μͷ୯ޠ͕Θ͔Βͳ͍ਓ޲͚ ࣈ۟ղੳ, ߏจղੳ, ධՁ, ίϯύΠϧ, yacc, τʔΫϯ, ந৅ߏจ໦ʢASTʣ, Ripper

6. Α͋͘ΔݴޠॲཧܥͷྲྀΕ ʮͬ͘͟ΓʯͰ͢ͷͰɺࡉ͔͍࿩͸શྗͰল͖·͢

7. ίʔυ

8. ίʔυ ࣈ۟ղੳ Lex

9. ίʔυ τʔΫϯྻ ࣈ۟ղੳ Lex

10. ίʔυ τʔΫϯྻ ߏจղੳ Parse ࣈ۟ղੳ Lex

11. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ࣈ۟ղੳ Lex

12. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ධՁ Evaluate ࣈ۟ղੳ Lex

13. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ධՁ Evaluate ࣈ۟ղੳ Lex

14. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ࣈ۟ղੳ

    Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

15. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

16. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

17. ίʔυΛτʔΫϯʢίʔυͷ࠷খ୯Ґʣʹ෼ׂ͢Δ puts ( 2 ࣝผࢠ + ਺஋ ࠨׅހ puts (10

    + 2*3)/4 10 ਺஋ + * ۭന ਺஋ * 3 ) ӈׅހ ۭന / / ਺஋ 4 ۭന

18. # Structͷ഑ྻ΍࿈૝഑ྻͷ഑ྻͳͲɺ΍Γํ͸৭ʑ [ [:identifier, "puts"], [:space, " "], [:lparen, "("],

    [:int, "10"], [:space, " "], [:plus, "+"], [:space, " "], [:int, "2"], [:asterisk, "*"], [:int, "3"], [:rparen, ")"], [:slash, "/"], [:int, "4"] ] ίʔυ্Ͱ͸͜Μͳײ͡

19. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

20. τʔΫϯྻ͔Βந৅ߏจ໦Λ࡞Δ ident:puts lparen:( int:10 plus:+ space: space: space: int:2 asterisk:*

    int:3 rparen:) slash:/ int:4 int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 <token>:<literal>

21. ந৅ߏจ໦ͱ͸ʁ • ໦ߏ଄ͷσʔλߏ଄ • AST (Abstract Syntax Tree) • ந৅

    == ຊ࣭Λൈ͖ग़ͨ͠΋ͷ • ίϝϯτ΍ۭനͳͲɺ࣮ߦ্ෆཁͳ৘ใΛഉআ͠ඞཁ ͳ৘ใͷΈͰߏ੒͞ΕΔ

22. ident:puts lparen:( int:10 plus:+ space: space: space: int:2 asterisk:* int:3

    rparen:) slash:/ int:4 int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ࠓճͷྫͰ͸੺࿮ͷ΋ͷ͕ASTʹ͸ͳ͍

23. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ͪͳΈʹׅހ͕ͳ͔ͬͨΒ puts

    10 + 2 * 3 / 4 ߏจ໦ͷߏ଄Ͱ࣮ߦॱং͕ දݱ͞Ε͍ͯΔ

24. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

25. ίϯύΠϧ • ͋ΔݴޠΛଞͷݴޠʹ຋༁͢Δ͜ͱ • ྫɿ CιʔείʔυΛόΠφϦʹ຋༁͢Δ • େ఍͸ߴڃݴޠ͔Β௿ڃݴޠ΁ͷ຋༁Λࢦ͢ • ಉਫ४ؒͰ΋຋༁ͨ͠ͳΒίϯύΠϧ͕ͩɺίϯύΠ

    ϧͱ͍͏ݴ༿ʹ͸ʮߴڃݴޠ͔Β௿ڃݴޠ΁ʯͷ຋༁ ͱ͍͏ҙຯ߹ؚ͍͕·ΕΔ͜ͱ͕ଟ͍

26. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ͜ͷॲཧΛԿ͔ ผͷݴޠʹஔ͖׵͑Δ

27. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

28. ධՁ • ࣮ߦͯ͠஋ΛಘΔ͜ͱ • ୯ʹʮ࣮ߦʯͱݴΘͳ͍ͷ͸ɺࣜΛͲͷΑ͏ʹղऍ͠ ࣮ͯߦ͢Δ͔͸ͦͷݴޠʹΑΔͨΊ • ΠϯλϓϦλݴޠͳΒ͹ɺCͳͲͰॻ͔ΕͨΠϯλϓϦλ ͕ASTΛஞ࣮࣍ߦ͢Δͷ͕Ұൠత •

    ΋ͪΖΜɺํ๏͸ͦͷଞ৭ʑ͋Δ

29. ந৅ߏจ໦ΛධՁ͢ΔྲྀΕͷΠϝʔδ ʢίʔυʹ͢Δͱ࠶ؼָ͕͍͘͢͝͠ʣ

30. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

31. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

32. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

33. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

34. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

35. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

36. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

37. int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

38. int:10 plus:+ int: 6 ident:puts slash:/ int:4

39. int:16 ident:puts slash:/ int:4

40. int:16 ident:puts slash:/ int:4

41. ident:puts int:4

42. 4 puts 4ͷ݁Ռ͕දࣔ͞ΕΔ

43. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ

    Evaluate ࣈ۟ղੳ Lex Ҏ্ɺͬ͘͟Γͨ͠ݴޠͷ͘͠ΈͰͨ͠ ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ

44. ͪΌΜͱษڧ͍ͨ͠ਓ͸ Writing An Interpreter In Go͕Φεεϝ • ӳޠͷຊͰ͕͢ɺςετίʔυ͕͔ͬ͠Γͯ͠ΔͷͰίʔ υΛಡΊ͹ͳΜͱ͔ͳΓ·͢ •

    A Tour of GoΛҰ௨Γཧղ͍ͯ͠Ε͹େৎ෉ • amazon.co.jpͰkindle൛ΛߪೖՄೳ • 2351ԁʢ2017/12/09࣌఺ʣ

45. ͬ͘͟ΓֶͿݴޠͷ͘͠Έ ׬

46. ୈ2෦ ͬ͘͟ΓֶͿRubyͷ͘͠Έ

47. ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ωΠςΟϒίʔυ
    όΠτίʔυ
    ͳͲ ࣮ߦɾධՁ

    ίϯύΠϧ Compile ධՁ Evaluate ࣈ۟ղੳ Lex Ұൠతͳϑϩʔ

48. Ruby 1.8·Ͱ ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ

49. ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ Ruby 1.9Ҏ߱ :"37
    

    ໋ྩྻ ίϯύΠϧ YARV

50. ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ Ruby 3? :"37
    

    ໋ྩྻ ίϯύΠϧ ωΠςΟϒ
    ίʔυ ίϯύΠϧ ࣮ߦ

51. ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ :"37
    ໋ྩྻ ίϯύΠϧ

52. parse.y ίʔυ ந৅ߏจ໦

53. parse.c parse.y bison ίʔυ ந৅ߏจ໦

54. parse.c parse.y bison def/keywords ίʔυ ந৅ߏจ໦

55. parse.c + lex.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦

56. parse.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦ ΑΓਖ਼֬ʹݴ͏ͱ parse.o y.tab.c

    mv lex.c cc Rubyιʔείʔυ׬શղઆ ୈ10ষ ύʔαΑΓ cpp

57. parse.c + lex.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦

58. parse.y • ߏจఆٛɾτʔΫϯఆٛϑΝΠϧ • ͜ͷϑΝΠϧΛݩʹɺbisonͰߏจղੳثΛ࡞Δ • io.cʹ͍࣍Ͱ2൪໨ʹߦ਺ͷଟ͍ϑΝΠϧʢͷ͸ͣʣ • 12000ߦӽ͑ʂʂʢ2017/12/09࣌఺ʣ •

    ७ਮͳߏจఆٛͷΈΛݟ͍ͨ৔߹͸sample/exyacc.rb͕ศར • `ruby sample/exyacc.rb parse.y`

59. parse.c + lex.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦

60. bison (yacc) • ύʔαδΣωϨʔλ • yaccͷ্Ґޓ׵ • yacc == yet

    another compiler compiler • compiler compiler == compiler generator • Ͱ΋yacc͸parser generator • Backus-Naur FormʢBNF: όοΧεφ΢Ξه๏ʣ • ͜͏͍͏ͷ !

61. parse.c + lex.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦

62. def/keywords • ༧໿ޠఆٛϑΝΠϧ • ͜ͷϑΝΠϧΛݩʹgperfΛ࢖ͬͯlex.cΛੜ੒͠ɺͦΕΛ parse.cͰinclude͢Δ • gperf == ׬શϋογϡؔ਺ੜ੒ث

63. ࣈ۟ղੳɾߏจղੳ݁ՌΛݟΔ

64. Ripper • Ruby૊ΈࠐΈͷύʔαϥΠϒϥϦ require 'ripper' code = 'puts "Hello World!"'

    # ࣈ۟ղੳ Ripper.lex(code) # ߏจղੳ Ripper.sexp(code) # sexp == symbolic expression (Sࣜ)

65. ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ :"37
    ໋ྩྻ ίϯύΠϧ

66. ந৅ߏจ໦ :"37
    ໋ྩྻ compile.c

67. code = 'puts "Hello World!"' RubyVM::InstructionSequence.compile(code).disasm ίϯύΠϧ݁ՌΛݟΔ 1. --dump=insns ΦϓγϣϯΛ͚࣮ͭͯߦ

    2. ίʔυ಺ͰίϯύΠϧͯ͠ٯΞηϯϒϧ͢Δ $ ruby --dump=insns sample.rb

68. puts (10 + 2*3)/4 == disasm: #<ISeq:<main>@sample.rb>===================================== 0000 trace 1

    ( 1) 0002 putself 0003 putobject 10 0005 putobject 2 0007 putobject 3 0009 opt_mult <callinfo!mid:*, argc:1, ARGS_SIMPLE>, <callcache> 0012 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0015 putobject 4 0017 opt_div <callinfo!mid:/, argc:1, ARGS_SIMPLE>, <callcache> 0020 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0023 leave

69. ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ :"37
    ໋ྩྻ ίϯύΠϧ

70. ݁Ռ :"37
    ໋ྩྻ eval.c, insns.defͳͲ

71. YARVͷධՁͷྲྀΕ

72. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave YARV໋ྩྻ YARVͷ ಺෦ελοΫ

73. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave YARV໋ྩྻ YARVͷ ಺෦ελοΫ

74. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

75. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave  TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

76. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave   TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

77. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave    TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

78. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave   TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

79. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave  TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

80. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave   TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

81. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave  TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

82. 0000 trace 0002 putself 0003 putobject 10 0005 putobject 2

    0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave OJM YARV໋ྩྻ YARVͷ ಺෦ελοΫ

83. ৄ͘͠͸ʰRubyͷ͘͠ΈʱΛ ಡΜͰ͍ͩ͘͞

84. ͦͷଞɺ΋ͬͱ஌Γ͍ͨਓʹΦεεϝͷຊɾαΠτ • Rubyιʔείʔυ׬શղઆ • https://docs.ruby-lang.org/en/2.4.0/ extension_ja_rdoc.html • https://github.com/ko1/rubyhackchallenge

85. ͬ͘͟ΓֶͿRubyͷ͘͠Έ ׬

86. Ҏ্

87. ݴޠॲཧܥָ͍͔͠Β ΈΜͳ΍Ζ͏ʂʂ