Walking around the ruby forest more deeply

my presentation of RubyKaigi2014. for beginners, introducing about how to start to read Ruby implementation source code.

Transcript

1. YUKI TORII(SASADA) 2014.10.19 RUBYKAIGI WALK AROUND THE RUBY FOREST
 MORE DEEPLY. THE GUIDE TO KNOW RUBY IMPLEMENTATION FOR NON-C LANGUAGE PROGRAMERS ϧϏΟϊεεϝ
2. WHO AM I? • ௗҪʢ࡫ాʣઇ / YUKI TORI • Rails application developer in Japan 
 (company: ສ༿ʣ • RailsGirls in JP organiser • Learned Programming by Rails
3. ADVERTISEMENT • Title:ϓϩϑΣογϣφϧͷͨΊͷ࣮ફHerokuೖ໳ • Author:
 Ayumu Aizawa
 Arton
 Yuki Torii
 Keiko Oda
4. FIRST • This talk is for non C programmers/ beginners like me. • goal:
 After go back home tonight, you glance at Ruby’s source code. ͜ͷτʔΫ͸ॳ৺ऀ޲͚Ͱ͢
 ʢࢲΈ͍ͨͳʣ
5. FIRST • But for beginners, Ruby’s implementation is like a deep, dark forest… • You must get Maps and Compass.
6. THE PREPARATION TO WALK INTO THE RUBY INTERNAL FOREST ࢄาͷ४උ
7. PUT ON YOUR BOOTS: • Get a set of Ruby source
 
 • Put a first book in C-language beside of you
 (ask your friends or Google about good book.)
 > git clone https://github.com/ruby/ruby ϧϏʔͷιʔείʔυΛखʹೖΕΔ
8. MAP&COMPASS • RUBY UNDER A MICROSCOPE • RUBY HACKING GUIDE 5XP
   CPPLT
9. MAP1: RUBY UNDER A MICROSCOPE • author: Pat Shaughnessy • ISBN: 9781593275273 • an Illustrated Guide
 to Ruby Internals.
10. RUBY UNDER A MICROSCOPE 
 WILL GIVE YOU… • a large scale Map
 about 2.0 Ruby Implementation. 
 • Parsing, and Compilation of Ruby code • Structures of Ruby Basic objects • and etc…
11. MAP1: PARSE-COMPILE Class MyCode deff my … end ! … end Your Ruby Code Perse Compile AST ByteCode Code-> ந৅ߏจ໦ -> ByteCode
12. MAP2:STRUCTURE OF OBJECTS String info… Array info… VALUE RSring flags klass RBasic VALUE RArray flags klass RBasic integer value Flags RObject built-in Object FIXNUM_FLAG : 1 simple ruby value RObject numiv ivptr VALUE flags klass RBasic and RRegexp etc…
 built-in objects and Symbol ect…
 Simple values
13. JAPANESE VER. IS COMING SOON! • ʰRubyͷ͘͠Έ 
 Ruby Under a Microscopeʱ • translators:
 Koji Shimada 
 Shintaro Kakutani • ൃߦɿΦʔϜࣾ։ൃ෦ • ץߦ༧ఆɿ2014೥ळ • I joins as one of reviewers. * ΋Ζ΋Ζௐ੔தͷͨΊ·ͩΦʔϜࣾʹ͸͓໰͍߹Θͤ͠ͳ͍Ͱ͍ͩ͘͞
14. MAP2: 
 RUBY HACKING GUIDE(RHG) • author: Minero Aoki • Original is written in Japanese, and is translated into English. 
 All contents are available at
 http://ruby-hacking- guide.github.io/ • Talking about very details on Ruby source code and implementation. • about Ruby ver. 1.7.3…
 old, but Very Very Useful!!
15. WHY RHG IS USEFUL? 
 BECAUSE IT’S VERY UNIVERSAL. • “How to read source codes” • making a method simple by cutting not essential code off. • understanding object structure before. • and many other wisdoms • It was written for C-programmers.
 But non C-programmer can learn
 Basic C essence for reading Ruby Implementation. essence
16. WHY RHG IS USEFUL? 
 BECAUSE IT’S SOMEHOW OLD. • Ruby is improving. • While reading Ruby source code with RHG, you will find a difference between the description and the current code.
 => This is a chance to know Why the source has been changed. It’s a trace of improvement of ruby.
 You can trace it by ChangeLog and <git log> with its commit message.
17. MAP3: SOURCE FILE MAP BSDL COPYING COPYING.ja ChangeLog GPL KNOWNBUGS.rb LEGAL Makefile.in NEWS README.EXT README.EXT.ja README.ja.md README.md addr2line.c addr2line.h array.c benchmark bignum.c bin boots traptest class.c common.mk compar.c compile.c complex.c configure.in constant.h cont.c cygwin debug.c defs dir.c dln.c dln.h dln_find.c dmydln.c dmyext.c doc enc encoding.c enum.c en umerator.c error.c eval.c eval_error.c eval_intern.h eval_jump.c ext file.c gc.c gc.h gem_prelude.rb golf_prelude.rb goruby.c hash.c ia64.s include inits.c insns.def internal.h io.c iseq.c iseq.h lex.c.blt lib load.c loadpath.c localeinit.c main.c man marshal.c math.c method.h miniinit.c misc missing nacl node.c node.h numeric.c object.c pack.c parse.y prelude.rb probes.d probes_helper.h proc.c process.c random.c range.c rational.c re.c regcomp.c regenc.c regenc.h regerror.c regexec.c regint.h regparse.c regparse.h regsyntax.c ruby.c ruby_atomic.h safe.c sample signal.c siphash.c siphash.h sparc.c spec sprintf.c st.c strftime.c string.c struct.c symbian template test thread.c thread_native.h thread_pthread.c thread_pth read.h thread_win32.c thread_win32.h time.c timev.h tool transcode.c transcode_data.h util.c variable.c version.c version.h vm.c vm_backtrace.c vm_core.h vm_debug.h vm_dump.c vm_eval .c vm_exec.c vm_exec.h vm_insnhelper.c vm_insnhelper.h vm_method.c vm_opts.h vm_trace.c vsnprintf.c win32 > cd ./ruby; ls ઈ๬….(desperate..)
18. MAP3: SOURCE FILE MAP BSDL COPYING COPYING.ja ChangeLog GPL KNOWNBUGS.rb LEGAL Makefile.in NEWS README.EXT README.EXT.ja README.ja.md README.md addr2line.c addr2line.h array.c benchmark bignum.c bin boots traptest class.c common.mk compar.c compile.c complex.c configure.in constant.h cont.c cygwin debug.c defs dir.c dln.c dln.h dln_find.c dmydln.c dmyext.c doc enc encoding.c enum.c en umerator.c error.c eval.c eval_error.c eval_intern.h eval_jump.c ext file.c gc.c gc.h gem_prelude.rb golf_prelude.rb goruby.c hash.c ia64.s include inits.c insns.def internal.h io.c iseq.c iseq.h lex.c.blt lib load.c loadpath.c localeinit.c main.c man marshal.c math.c method.h miniinit.c misc missing nacl node.c node.h numeric.c object.c pack.c parse.y prelude.rb probes.d probes_helper.h proc.c process.c random.c range.c rational.c re.c regcomp.c regenc.c regenc.h regerror.c regexec.c regint.h regparse.c regparse.h regsyntax.c ruby.c ruby_atomic.h safe.c sample signal.c siphash.c siphash.h sparc.c spec sprintf.c st.c strftime.c string.c struct.c symbian template test thread.c thread_native.h thread_pthread.c thread_pth read.h thread_win32.c thread_win32.h time.c timev.h tool transcode.c transcode_data.h util.c variable.c version.c version.h vm.c vm_backtrace.c vm_core.h vm_debug.h vm_dump.c vm_eval .c vm_exec.c vm_exec.h vm_insnhelper.c vm_insnhelper.h vm_method.c vm_opts.h vm_trace.c vsnprintf.c win32 Don't give up your hope. File names are easy to understand. and README.EXT has
 “Appendix A. Ruby Source Files Overview”
19. MAP3: SOURCE FILE MAP ”Ruby Source Files Overview” in README.EXT groups source files by their contents. Ruby Language Core class.c :: classes and modules error.c :: exception classes and exception mechanism gc.c :: memory management load.c :: library loading object.c :: objects variable.c :: variables and constants Class Library array.c :: Array bignum.c :: Bignum compar.c :: Comparable complex.c :: Complex cont.c :: Fiber, Continuation dir.c :: Dir enum.c :: Enumerable enumerator.c :: Enumerator file.c :: File hash.c :: Hash io.c :: IO marshal.c :: Marshal math.c :: Math numeric.c :: Numeric, Integer, Fixnum, Float pack.c :: Array#pack, String#unpack proc.c :: Binding, Proc process.c :: Process random.c :: random number range.c :: Range rational.c :: Rational re.c :: Regexp, MatchData signal.c :: Signal sprintf.c :: String#sprintf string.c :: String struct.c :: Struct time.c :: Time Ruby Syntax Parser parse.y :: grammar definition parse.c :: automatically generated from parse.y keywords :: reserved keywords lex.c :: automatically generated from keywords Ruby Evaluator (a.k.a. YARV), Regular Expression Engine (Oniguruma), Utility Functions, Ruby Interpreter Implementation, Multilingualization, goruby Interpreter Implementation and other groups
20. MAP3: SOURCE FILE MAP File names are easy to understand. see the class libraries. Ruby Language Core class.c error.c gc.c load.c object.c variable.c Ruby Syntax Parser parse.y :: grammar definition parse.c :: automatically generated from parse.y keywords :: reserved keywords lex.c :: automatically generated from keywords Ruby Evaluator (a.k.a. YARV) Regular Expression Engine (Oniguruma) Utility Functions Ruby Interpreter Implementation Multilingualization goruby Interpreter Implementation and other groups Class Library array.c :: Array bignum.c :: Bignum compar.c :: Comparable complex.c :: Complex cont.c :: Fiber, Continuation dir.c :: Dir enum.c :: Enumerable enumerator.c :: Enumerator file.c :: File hash.c :: Hash io.c :: IO
21. MAP3: SOURCE FILE MAP extension also give us hints. xx.c => Of cause c code xx.h => header file. Many definitions such as macros, data-structures. xx.y => only parse.y (grammar definition) xx.rb => Of cause ruby file. xx.def => miscellaneous definitions
22. MAP3: SOURCE FILE MAP /include/ … many ruby’s core header files. /defs/ … definition files such as keywords, errors… /ext/ … c extensions such as json, socket… /lib/ …ruby libraries such as csv, irb, yaml, rake… /missing/ …implements of C functions need for ruby,
 but not all Systems have. /test/ … test codes for ruby implementations. /spec/ … RubySpec will be downloaded here. Special directories.
23. COMPASS:METHOD DEFINITION Because some file names correspond to class names, it is easy to find where a method is defined. What Hash#to_a does…? 'PS
    FYBNQMF
    JG
    POF
    EBZ
    *
    TUBSU
    
    UP
    XPOEFSJOH BOE
    DBOOPU
    TMFFQ
    XFMM
24. COMPASS:METHOD DEFINITION Because some file names correspond to class name, it is easy to find where a method is defined. What Hash#to_a does…? 'PS
    FYBNQMF
    JG
    POF
    EBZ
    *
    TUBSU
    
    UP
    XPOEFSJOH BOE
    DBOOPU
    TMFFQ
    XFMM See “hash.c”, Of course. and GoodNight.
25. COMPASS:METHOD DEFINITION Easy to find the definition of Hash#to_a in hash.c Because… ! WPJE
    *OJU@)BTI WPJE
    \
    
    
    
    
    
    TOJQ
    !
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI JOJUJBMJ[F
    SC@IBTI@JOJUJBMJ[F
     
    
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI JOJUJBMJ[F@DPQZ
    SC@IBTI@JOJUJBMJ[F@DPQZ
     
    
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI SFIBTI
    SC@IBTI@SFIBTI
     
    !
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI UP@IBTI
    SC@IBTI@UP@IBTI
     
    
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI UP@I
    SC@IBTI@UP@I
     
    
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI UP@B
    SC@IBTI@UP@B
     
    
    
    
    
    SC@EFpOF@NFUIPE SC@D)BTI JOTQFDU
    SC@IBTI@JOTQFDU
     
    
    
    
    
    SC@EFpOF@BMJBT SC@D)BTI
    UP@T
    lJOTQFDU 
    !
    
    
    
    TOJQ
    NBOZ
    PUIFS
    NFUIPE
    EFpOJUJPOT
    built-in class has Init_xx function
 at bottom of its file. }In the Init_xx, almost all methods are defined by “rb_define_method”
26. COMPASS:METHOD DEFINITION rb_define_method takes understandable args. TUBUJD
    7"-6&
    SC@IBTI@UP@B 7"-6&
    IBTI
    \
    
    
    
    
    7"-6&
    BSZ
    !
    
    
    
    BSZ
    
    SC@BSZ@OFX@DBQB 3)"4)@4*;& IBTI 
    
    
    
    
    SC@IBTI@GPSFBDI IBTI
    UP@B@J
    BSZ 
    
    
    
    
    0#+@*/'&$5 BSZ
    IBTI 
    !
    
    
    
    SFUVSO
    BSZ
    ^
    SC@EFpOF@NFUIPE SC@D)BTI UP@B
    SC@IBTI@UP@B
      klass method_name function init arg function name is put on the head of line
 to search easily. make new array with hash size convert hash’s each pair to array Macro often defined in ***.h.
 but I can imagine how work from name…
27. Good Night.
28. NOW, YOU GOT A READY.
 WOULD YOU LIKE TO
 WALK IN RUBY INTERNAL A LITTLE?
29. AN EXAMPLE ROUTE:
 ABOUT “HASH-TABLE”.
30. HASH TABLE IS USED EVERYWHERE. • Hash Class • Method Table • Instance Variables • Constant • Symbol 
    HJU
    HSFQ
    TU@JOJU
    c
    HSFQ
    W
    EPD
    c
    HSFQ
    W
    UFTU
31. WHAT IS A HASH TABLE. The data-structure for keeping key/value pair. obj of Hash class Method Table Key Value :favorite_animal “rabbit" “scores” [10,3,4] :neighborhood #<Person> Key Value “name” def name … “age” def age… “love?” def love?(me)….
32. OPEN HASH TABLE STRUCTURE bins entries ɾ ! ! ɾ “entry” under corresponding with hash-value in bins.
33. RUBY HASH TABLE STRUCTURE bins entries st_table type num_bins bins (big) num_entries st_table_entry hash key record next fore back 1 2 3 4 next next fore back (hash is ordered
 from Ruby 1.9~) 5 6 7 entries_packed (0) back
34. RUBY HASH TABLE STRUCTURE entries st_packed_entry hash key value 1 2 3 4 5 st_table type num_bins packed num_entries entries_packed (1) real_entries
35. WHEN ENTRIES ARE TOO MANY. Packed -> bins 1 2 3 4 5 6 key hash val
36. WHEN ENTRIES ARE TOO MANY. rehash(allocate more bins) (ਤͷentryͷ਺͸ద౰Ͱ͢…)
37. YOU CAN FIND ALL OF THEM IN • st_table
 in “include/ruby/st.h” • st_table_entry /st_packed_entry • add_packed_direct • unpacked • add_direct • rehash in “st.c”
38. LAST:DRAW YOUR OWN MAY
 BY YOUR HAND • The map of two books are a part of the rich Ruby forest. • You can find your favorite path, 
 or nice views by your self. ࣗ෼ͷ஍ਤΛ࡞ͬͯΈΑ͏
39. ENJOY YOUR RUBY LIFE. THANK YOU!