ざっくり学ぶ言語のしくみ

1.

ͬ͘͟ΓֶͿݴޠͷ͘͠Έ Rails Developers Meetup 2017 2017/12/09

2.

ࣗݾ঺հ • ൘૔ୡ໵ʢ@itkrt2yʣ • λέϢʔɾ΢Σϒגࣜձࣾ • Railsͷडୗձࣾ • ϑϧϦϞʔτɺϑϧϑϨοΫε •
ઈࢍ࠾༻தʂʂڵຯ͋Δํ͸ޙͰ੠Λ͔͚͍ͯͩ͘͞

3.

ࠓ೔ͷ࿩

4.

ݴޠॲཧܥ

5.

͜͜Β΁Μͷ୯ޠ͕Θ͔Βͳ͍ਓ޲͚ ࣈ۟ղੳ, ߏจղੳ, ධՁ, ίϯύΠϧ, yacc, τʔΫϯ, ந৅ߏจ໦ʢASTʣ, Ripper

6.

Α͋͘ΔݴޠॲཧܥͷྲྀΕ ʮͬ͘͟ΓʯͰ͢ͷͰɺࡉ͔͍࿩͸શྗͰল͖·͢

7.

ίʔυ

8.

ίʔυ ࣈ۟ղੳ Lex

9.

ίʔυ τʔΫϯྻ ࣈ۟ղੳ Lex

10.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ࣈ۟ղੳ Lex

11.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ࣈ۟ղੳ Lex

12.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ධՁ Evaluate ࣈ۟ղੳ Lex

13.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ධՁ Evaluate ࣈ۟ղੳ Lex

14.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ࣈ۟ղੳ
Lex ωΠςΟϒίʔυ όΠτίʔυ ͳͲ

15.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ࣮ߦɾධՁ ίϯύΠϧ Compile ධՁ
Evaluate ࣈ۟ղੳ Lex ωΠςΟϒίʔυ όΠτίʔυ ͳͲ

16.


17.

ίʔυΛτʔΫϯʢίʔυͷ࠷খ୯Ґʣʹ෼ׂ͢Δ puts ( 2 ࣝผࢠ + ਺஋ ࠨׅހ puts (10
+ 2*3)/4 10 ਺஋ + * ۭന ਺஋ * 3 ) ӈׅހ ۭന / / ਺஋ 4 ۭന

18.

# Structͷ഑ྻ΍࿈૝഑ྻͷ഑ྻͳͲɺ΍Γํ͸৭ʑ [ [:identifier, "puts"], [:space, " "], [:lparen, "("],
[:int, "10"], [:space, " "], [:plus, "+"], [:space, " "], [:int, "2"], [:asterisk, "*"], [:int, "3"], [:rparen, ")"], [:slash, "/"], [:int, "4"] ] ίʔυ্Ͱ͸͜Μͳײ͡

19.


20.

τʔΫϯྻ͔Βந৅ߏจ໦Λ࡞Δ ident:puts lparen:( int:10 plus:+ space: space: space: int:2 asterisk:*
int:3 rparen:) slash:/ int:4 int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 <token>:<literal>

21.

ந৅ߏจ໦ͱ͸ʁ • ໦ߏ଄ͷσʔλߏ଄ • AST (Abstract Syntax Tree) • ந৅
== ຊ࣭Λൈ͖ग़ͨ͠΋ͷ • ίϝϯτ΍ۭനͳͲɺ࣮ߦ্ෆཁͳ৘ใΛഉআ͠ඞཁ ͳ৘ใͷΈͰߏ੒͞ΕΔ

22.

ident:puts lparen:( int:10 plus:+ space: space: space: int:2 asterisk:* int:3
rparen:) slash:/ int:4 int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ࠓճͷྫͰ͸੺࿮ͷ΋ͷ͕ASTʹ͸ͳ͍

23.

int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ͪͳΈʹׅހ͕ͳ͔ͬͨΒ puts
10 + 2 * 3 / 4 ߏจ໦ͷߏ଄Ͱ࣮ߦॱং͕ දݱ͞Ε͍ͯΔ

24.


25.

ίϯύΠϧ • ͋ΔݴޠΛଞͷݴޠʹ຋༁͢Δ͜ͱ • ྫɿ CιʔείʔυΛόΠφϦʹ຋༁͢Δ • େ఍͸ߴڃݴޠ͔Β௿ڃݴޠ΁ͷ຋༁Λࢦ͢ • ಉਫ४ؒͰ΋຋༁ͨ͠ͳΒίϯύΠϧ͕ͩɺίϯύΠ
ϧͱ͍͏ݴ༿ʹ͸ʮߴڃݴޠ͔Β௿ڃݴޠ΁ʯͷ຋༁ ͱ͍͏ҙຯ߹ؚ͍͕·ΕΔ͜ͱ͕ଟ͍

26.

int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4 ͜ͷॲཧΛԿ͔ ผͷݴޠʹஔ͖׵͑Δ

27.


28.

ධՁ • ࣮ߦͯ͠஋ΛಘΔ͜ͱ • ୯ʹʮ࣮ߦʯͱݴΘͳ͍ͷ͸ɺࣜΛͲͷΑ͏ʹղऍ͠ ࣮ͯߦ͢Δ͔͸ͦͷݴޠʹΑΔͨΊ • ΠϯλϓϦλݴޠͳΒ͹ɺCͳͲͰॻ͔ΕͨΠϯλϓϦλ ͕ASTΛஞ࣮࣍ߦ͢Δͷ͕Ұൠత •
΋ͪΖΜɺํ๏͸ͦͷଞ৭ʑ͋Δ

29.

ந৅ߏจ໦ΛධՁ͢ΔྲྀΕͷΠϝʔδ ʢίʔυʹ͢Δͱ࠶ؼָ͕͍͘͢͝͠ʣ

30.

int:10 plus:+ int:2 asterisk:* int:3 ident:puts slash:/ int:4

31.


32.


33.


34.


35.


36.


37.


38.

int:10 plus:+ int: 6 ident:puts slash:/ int:4

39.

int:16 ident:puts slash:/ int:4

40.

int:16 ident:puts slash:/ int:4

41.

ident:puts int:4

42.

4 puts 4ͷ݁Ռ͕දࣔ͞ΕΔ

43.

Evaluate ࣈ۟ղੳ Lex Ҏ্ɺͬ͘͟Γͨ͠ݴޠͷ͘͠ΈͰͨ͠ ωΠςΟϒίʔυ όΠτίʔυ ͳͲ

44.

ͪΌΜͱษڧ͍ͨ͠ਓ͸ Writing An Interpreter In Go͕Φεεϝ • ӳޠͷຊͰ͕͢ɺςετίʔυ͕͔ͬ͠Γͯ͠ΔͷͰίʔ υΛಡΊ͹ͳΜͱ͔ͳΓ·͢ •
A Tour of GoΛҰ௨Γཧղ͍ͯ͠Ε͹େৎ෉ • amazon.co.jpͰkindle൛ΛߪೖՄೳ • 2351ԁʢ2017/12/09࣌఺ʣ

45.

ͬ͘͟ΓֶͿݴޠͷ͘͠Έ ׬

46.

ୈ2෦ ͬ͘͟ΓֶͿRubyͷ͘͠Έ

47.

ίʔυ τʔΫϯྻ ߏจղੳ Parse ந৅ߏจ໦ ݁Ռ ωΠςΟϒίʔυ όΠτίʔυ ͳͲ ࣮ߦɾධՁ
ίϯύΠϧ Compile ධՁ Evaluate ࣈ۟ղੳ Lex Ұൠతͳϑϩʔ

48.

Ruby 1.8·Ͱ ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ

49.

ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ Ruby 1.9Ҏ߱ :"37
໋ྩྻ ίϯύΠϧ YARV

50.

ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ Ruby 3? :"37
໋ྩྻ ίϯύΠϧ ωΠςΟϒ ίʔυ ίϯύΠϧ ࣮ߦ

51.

ίʔυ ධՁ ࣈ۟ղੳ + ߏจղੳ ந৅ߏจ໦ ݁Ռ :"37 ໋ྩྻ ίϯύΠϧ

52.

parse.y ίʔυ ந৅ߏจ໦

53.

parse.c parse.y bison ίʔυ ந৅ߏจ໦

54.

parse.c parse.y bison def/keywords ίʔυ ந৅ߏจ໦

55.

parse.c + lex.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦

56.

parse.c parse.y bison def/keywords gperf ίʔυ ந৅ߏจ໦ ΑΓਖ਼֬ʹݴ͏ͱ parse.o y.tab.c
mv lex.c cc Rubyιʔείʔυ׬શղઆ ୈ10ষ ύʔαΑΓ cpp

57.


58.

parse.y • ߏจఆٛɾτʔΫϯఆٛϑΝΠϧ • ͜ͷϑΝΠϧΛݩʹɺbisonͰߏจղੳثΛ࡞Δ • io.cʹ͍࣍Ͱ2൪໨ʹߦ਺ͷଟ͍ϑΝΠϧʢͷ͸ͣʣ • 12000ߦӽ͑ʂʂʢ2017/12/09࣌఺ʣ •
७ਮͳߏจఆٛͷΈΛݟ͍ͨ৔߹͸sample/exyacc.rb͕ศར • `ruby sample/exyacc.rb parse.y`

59.


60.

bison (yacc) • ύʔαδΣωϨʔλ • yaccͷ্Ґޓ׵ • yacc == yet
another compiler compiler • compiler compiler == compiler generator • Ͱ΋yacc͸parser generator • Backus-Naur FormʢBNF: όοΧεφ΢Ξه๏ʣ • ͜͏͍͏ͷ !

61.


62.

def/keywords • ༧໿ޠఆٛϑΝΠϧ • ͜ͷϑΝΠϧΛݩʹgperfΛ࢖ͬͯlex.cΛੜ੒͠ɺͦΕΛ parse.cͰinclude͢Δ • gperf == ׬શϋογϡؔ਺ੜ੒ث

63.

ࣈ۟ղੳɾߏจղੳ݁ՌΛݟΔ

64.

Ripper • Ruby૊ΈࠐΈͷύʔαϥΠϒϥϦ require 'ripper' code = 'puts "Hello World!"'
# ࣈ۟ղੳ Ripper.lex(code) # ߏจղੳ Ripper.sexp(code) # sexp == symbolic expression (Sࣜ)

65.


66.

ந৅ߏจ໦ :"37 ໋ྩྻ compile.c

67.

code = 'puts "Hello World!"' RubyVM::InstructionSequence.compile(code).disasm ίϯύΠϧ݁ՌΛݟΔ 1. --dump=insns ΦϓγϣϯΛ͚࣮ͭͯߦ
2. ίʔυ಺ͰίϯύΠϧͯ͠ٯΞηϯϒϧ͢Δ $ ruby --dump=insns sample.rb

68.

puts (10 + 2*3)/4 == disasm: #<ISeq:<main>@sample.rb>===================================== 0000 trace 1
( 1) 0002 putself 0003 putobject 10 0005 putobject 2 0007 putobject 3 0009 opt_mult <callinfo!mid:*, argc:1, ARGS_SIMPLE>, <callcache> 0012 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0015 putobject 4 0017 opt_div <callinfo!mid:/, argc:1, ARGS_SIMPLE>, <callcache> 0020 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0023 leave

69.


70.

݁Ռ :"37 ໋ྩྻ eval.c, insns.defͳͲ

71.

YARVͷධՁͷྲྀΕ

72.

0000 trace 0002 putself 0003 putobject 10 0005 putobject 2
0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave YARV໋ྩྻ YARVͷ ಺෦ελοΫ

73.

0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave YARV໋ྩྻ YARVͷ ಺෦ελοΫ

74.

0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave TFMG YARV໋ྩྻ YARVͷ ಺෦ελοΫ

75.


76.


77.


78.


79.


80.


81.


82.

0007 putobject 3 0009 opt_mult 0012 opt_plus 0015 putobject 4 0017 opt_div 0020 opt_send_without_block mid:puts 0023 leave OJM YARV໋ྩྻ YARVͷ ಺෦ελοΫ

83.

ৄ͘͠͸ʰRubyͷ͘͠ΈʱΛ ಡΜͰ͍ͩ͘͞

84.

ͦͷଞɺ΋ͬͱ஌Γ͍ͨਓʹΦεεϝͷຊɾαΠτ • Rubyιʔείʔυ׬શղઆ • https://docs.ruby-lang.org/en/2.4.0/ extension_ja_rdoc.html • https://github.com/ko1/rubyhackchallenge

85.

ͬ͘͟ΓֶͿRubyͷ͘͠Έ ׬

86.

Ҏ্

87.

ݴޠॲཧܥָ͍͔͠Β ΈΜͳ΍Ζ͏ʂʂ

ざっくり学ぶ言語のしくみ

ざっくり学ぶ言語のしくみ

Want more?

Transcript