ROM: the final frontier of mruby

presentation material of "ROM: the final frontier of mruby" in RubyConf 2018
http://www.rubyconf.org/

Transcript

1. ROM: the final frontier of mruby 2018-11-14 RubyConf 2018 Los Angeles Yurie Yamane, Masayoshi Takahashi
2. Notice ‣ This session is a technical talk ‣ But you don’t need to know about mruby in detail ‣ This session shows many C code, not Ruby code ‣ We explain the implementation of mruby using C ✨
3. About Us ‣ Team Yamanekko ‣ Yurie Yamane ‣ Masayoshi Takahashi ‣ https://github.com/yamanekko/
4. Yurie Yamane ‣ Toppers Project member ‣ project of RTOS and Embedded Systems TOPAME
5. Masayoshi Takahashi ‣ a committer of Re:VIEW (a publishing tool) ‣ https://rubygems.org/ gems/review ‣ RubyKaigi Organizer Team http://rubykaigi.org/
6. Today's Topic
7. Today's Topic ‣ mruby, and ‣ ROM
8. Today's Topic ‣mruby, and ‣ ROM
9. What's mruby ‣ Yet Another (lightweight) Ruby Implementation ‣ Complying to (part of) ISO Ruby (ISO/IEC 30170) ‣ Used in embedded devices and in server applications
10. Embedded Devices ‣ micro controller ‣ Nucleo F401RE + Adafruit 802 TFT LCD ‣ M5Stack (ESP32)
11. Micro controller MPU ROM RAM SPI I2C GPIO PWM ADC Ethernet WiFi peripheral Bluetooth RNG SHA microcontroller contains processor core(s), ROM, RAM, peripherals and other modules on single chip
12. compiler and mruby VM DPNQJMFS
    NSCD NSVCZ
    7. DPNQJMFS
    
    NSVCZ
    7. &YFDVUF SC
    3VCZ
    $PEF NSC
    CZUFDPEF
13. mruby Update in 2018 ‣mruby 2.0 will be released soon ‣in next month? ‣The first major update ‣bytecode incompatible against 1.x ‣support keyword arguments ‣Let’s start with brand-new mruby!
14. Today's Topic ‣ mruby, and ‣ROM
15. ROM ‣ Read Only Memory ‣ aka Non Volatile Memory https://commons.wikimedia.org/wiki/File:ANT_Nachrichtentechnik_DBT-03_- _Texas_Instruments_TBP18SA030N-0019.jpg#file
16. EEPROM ‣ Electrically Erasable Programmable Read-Only Memory ‣ store without battery ‣ erase/write with battery ‣ see https://en.wikipedia.org/ wiki/EEPROM https://commons.wikimedia.org/wiki/ File:AT24C02_EEPROM_1480355_6_7_HDR_Enhancer .jpg
17. Flash ROM ‣ is a type of EEPROM ‣ erase blocks (typically 512 bytes) ‣ major storage in microcontroller ‣ see https://en.wikipedia.org/ wiki/EEPROM https://commons.wikimedia.org/wiki/ File:IPhone_3G_teardown_- _Intel_3050M0Y0CE_-3303.jpg
18. ROM / EEPROM / Flash 30.
    
    
    
    
    &&130.
    
    
    
    QFS
    CZUF
    'MBTI
    NFNPSZ
    QFS
    CMPDL &MFDUSJDBMMZ
    &SBTBCMF
19. Why we want to use ROM? ‣ larger than RAM in microcontroller
20. Microcontroller Boards List #PBSE 1SJDF 3". 'MBTI
    30. NSVCZ -&(0
    .JOE4UPSNT  .# .# ✔ (31&"$)  .# .# ✔ .4UBDL  ,# .# ✔ OVDMFP
    ';(  ,# ,# ✔ OVDMFP
    '3&  ,# ,# OVDMFP
    '3&  ,# ,# "SEVJOP
    
    VOP  ,# ,# &YQFOTJWFʜ XFMMLOPXO &YQFOTJWFʜ QPQVMBS
21. Why so little RAM? ‣ Cost (price) ‣ Power consumption ‣ No need for microcontrollers ‣ cf. https://electronics.stackexchange.com/questions/ 134496/why-do-microcontrollers-have-so-little-ram
22. Why we want to use ROM? ‣ larger than RAM in microcontroller ‣ Ruby consumes RAM before execution
23. “Before execution” puts “Hello, world!”
24. “Before execution” puts “Hello, world!” Initialize GC and heap memory Initialize Standard Classes
25. “Before execution” puts “Hello, world!” Initialize GC and heap memory Initialize Standard Classes in RAM
26. Ruby consumes RAM when initializing ‣ making standard (pre-defined) Classes and Objects ‣ Object, Class, Kernel, Numeric, String, ...
27. Why Ruby’s classes in RAM? ‣ Because Ruby is a dynamic language
28. Ruby is a dynamic language ‣ method added/removed/redefined in runtime ‣ class is an object (instance of Class class)
29. Summary the points ‣ Embedded Devices have small amount of RAM ‣ We want to use ROM ‣ But Ruby is dynamic, so put every Objects on RAM ‣ We should solve this issue
30. See Objects in detail
31. mruby VM has class objects Proc String Float Fixnum Symbol Kernel Object Class Array Hash TrueClass FalseClass Module Range NilClass mrb_state (mruby VM)
32. class has ivars and methods iv mt internal data methods class object (RClass) iv: instance variable mt: method table
33. Ruby objects are struct in C struct RClass { MRB_OBJECT_HEADER; struct iv_tbl *iv; struct kh_mt *mt; struct RClass *super; };
34. Which’s in ROM, Which’s in RAM ‣ ROM: Immutable things ‣ RAM: Mutable things ‣ a member of Immutable structure can be Mutable ‣ and vice versa
35. Which’s in ROM, Which’s in RAM class mrb_state iv_tbl internal data methods RAM ROM mt
36. add methods in runtime class mrb_state iv_tbl internal data methods RAM ROM mt methods
37. Symbol table ‣ symbol table: pairs of id (integer) and name (string) ‣ Every class names and method names are in symbol table ‣ So if some classes and methods are in ROM, symbol table is defined and in ROM id name 1 BasicObject 2 Object 3 Module 4 Class 5 __classname__ 6 Proc 7 initialize … …
38. Implementation
39. Hello World in mruby #include <mruby.h> #include <mruby/compile.h> int main(void) { mrb_state *mrb = mrb_open(); if (!mrb) { /* handle error */ } mrb_load_string(mrb, "puts 'hello world'"); mrb_close(mrb); return 0; } initialize load & run
40. mrb_open() ‣ Initialization ≒ mrb_open();
41. mrb_open() { mrb_gc_init(mrb, &mrb->gc); /* initializing HEAP */ mrb_init_symtbl(mrb); /* making symbol table */ mrb_init_class(mrb); /* Class */ mrb_init_object(mrb); /* Object */ mrb_init_kernel(mrb); /* Kernel */ mrb_init_XXXX(mrb); /* class libraries in C */ mrb_init_mrblib(mrb); /* class libraries in Ruby */ mrb_init_mrbgems(mrb); /* gems for mruby */ }
42. Symbol table in ROM ‣ symbol id -> name: use Array of C ‣ name -> symbol id: use gperf ‣ perfect hash function generator ‣ used in MRI (CRuby) ‣ see https://www.gnu.org/software/gperf/
43. detect symbols ‣ scan mruby code in C and Ruby ‣ generate JSON ‣ make *.key file for gperf from JSON ‣ use gperf to generate *.c JSON *.key *.c script gperf script
44. detect methods ‣ scan mruby code in C and Ruby ‣ generate *.h files ‣ include them in mruby code *.h script
45. Memory Map segment RAM/ROM description .text ROM actual code .rodata ROM read only data, constant .data RAM variables initialized explicitly by data .bss RAM variables initialized by 0
46. put symbols on ROM static const char *presym2name[] = { "__attached__", "BasicObject", "Object", “Module", /* … */ } static unsigned int presym_hash (register const char *str, register unsigned int len) { static const unsigned short asso_values[] = { 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, 1410, /* … */ }
47. result 0 20,000 40,000 60,000 80,000 100,000 malloc 66,919 82,783
48. Future Works
49. Future Works ‣ make IREP into ROM ‣ support mruby 2.0 ‣ feedback to mruby core
50. IREP into ROM ‣ IREP: a structure for compiled ruby scripts ‣ standard libraries and mrbgems written in Ruby are compiled into IREPs /* Program data array struct */ typedef struct mrb_irep { uint16_t nlocals; /* Number of local variables */ uint16_t nregs; /* Number of register variables */ uint8_t flags; mrb_code *iseq; mrb_value *pool; mrb_sym *syms; struct mrb_irep **reps; struct mrb_locals *lv; /* debug info */ mrb_bool own_filename; const char *filename; uint16_t *lines; struct mrb_irep_debug_info* debug_info; int ilen, plen, slen, rlen, refcnt; } mrb_irep;
51. mrb_open() and dump ‣ execute mrb_open() ‣ dump all IREPs, Objects, … as C code ‣ generate JSON of symbols ‣ build them all
52. result (WIP) 0 20,000 40,000 60,000 80,000 100,000 120,000 malloc 21,940 109,440
53. support mruby 2.0 /* Instance variable table structure */ typedef struct iv_tbl { segment *rootseg; size_t size; size_t last_len; } iv_tbl; /* Instance variable table structure */ typedef struct iv_tbl { khash_t(iv) h; } iv_tbl; mruby 1.4.1 mruby 2.0
54. feedback to mruby core ‣ The body of some mruby methods in C are static functions, so we cannot refer in generated C files. ‣ We need to fix them. static mrb_value mrb_ary_plus(mrb_state *mrb, mrb_value self) { struct RArray *a1 = mrb_ary_ptr(self); /* ... */
55. Future Work ‣ Our project is selected by the Ruby Association grant committee! ‣ https://www.ruby.or.jp/ en/news/20181106
56. Summary
57. Summary ‣ Embedded Devices have small RAM, so use ROM more ‣ Ruby is dynamic, every class objects can be mutable ‣ Even if they are mutable, decompose and put parts of them on ROM ‣ You can use mruby on various boards!
58. Special Thanks to: ‣ Matz, Prof. Tanaka and SWEST participants ‣ @kishima ‣ Hiroaki Nagashima ‣ @TAKASEhideki ‣ @tenderlove
59. Thanks!