What's happening when initializing mruby?

mruby and its initialization.
A problem of initialization of mruby and how to improve it.
How to use mruby on M5Stick-C.

Transcript

1. What's happening when initializing mruby? 2019-11-20 RubyConf 2019 Team Yamanekko Yurie Yamane, Masayoshi Takahashi
2. Outline of this session • About us • What’s mruby • mruby demo (m5stickc) • What’s initialization • A problem of initialization • How to improve initialization • Introducing mruby-bin-nvgen • nvgen: problem and solution • Result
3. About Us
4. Yurie Yamane • Toppers Project member • a project of RTOS and Embedded Systems • “Talking about mruby with Matz: Today and Tomorrow” @ET2019 in this night TOPAME
5. Masayoshi Takahashi • +20 Rubyist • RubyKaigi organizer team • RubyKaigi 2020 in next April, at matsumoto city
6. “Introduction to mruby” • using mruby with C API • self-publishing zine in Japanese • 108 pages We are going to translate this book into English.
7. What's happening when initializing mruby?
8. mruby
9. What’s mruby? • Lightweight implementation of Ruby • comply to ISO 30170:2012 • Used in embedded devices and in server applications • 2.1.0 will be released in this week was
10. M5StickC ESP32-based 4MByte Flash, 520 KB SRAM 6-Axis IMU 0.96 inch LCD WiFi + Bluetooth 80 mAh Lipo Battery Grove Port
11. M5StickC ESP32-based 4MByte Flash, 520 KB SRAM 6-Axis IMU 0.96 inch LCD WiFi + Bluetooth 80 mAh Lipo Battery Grove Port
12. demo
13. What's happening when initializing mruby?
14. Quiz How many objects created in initialization? (1) 68 (2) 105 (3) 212 Hint: class is object
15. What’s initialization “Initialization” means in this talk: == everything before executing your Ruby scripts
16. What’s initialization “Initialization” means in this talk: == everything before executing your Ruby scripts •initializing mruby VM •initializing GC system •initializing all Classes and Modules •creating Object, String, Integer, Array, … •defining methods for all classes / modules
17. • components of VM • context • callinfo • stack • create new context • create 1 callinfo array and 1 stak on the context initializing mruby VM mruby VM (mrb_state) context … callinfo stack
18. • initialize memories for Objects • fixed-size memory spaces initializing GC system RVALUE (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE)
19. • create RClass (RVALUE) • create related data • methods of the class • classname • constants • create symbols initializing Class/Module Object String RClass RClass Exception RClass methods …… other data methods other data methods other data
20. Problem of Initialization
21. Problem of initialization • memory consumption • initialize classes, methods… • but almost all classes/modules are not changed • micro controller has small memory • mruby is (relatively) not enough to small
22. How to improve it
23. How to improve it • In general, ROM is wider than RAM in micro controller • RAM ROM LEGO MindStorms 64MB 16MB M5StickC 520KB 4MB SparkFun Pro nRF52840 Mini 256KB 1MB nucleo F412ZG 256KB 1024KB nucleo F411RE 128KB 512KB
24. How to put data on ROM • We can’t put data on Flash ROM dynamically • because it’s ROM! • We can put them as source code in C • use keyword const • pre-execution and code generation const struct RClass nv_object_212 = { .tt=MRB_TT_CLASS, .color=7, .flags=212, .c=(struct RClass *)&nv_object_211, .gcnext= (struct RBasic*)&nv_object_210, .iv=(struct iv_tbl*)&nv_iv_90, .mt=(struct kh_mt*)&nv_mt_86_empty, .super =NULL, /* name=BasicObject */ };
25. pre-execution A = Array.new A << 1 if A.size >= 1 A += [2, 3] end A = [1, 2, 3] pre- execution original generated
26. e code generation generator and executable • generator generates *.c files • symbols • objects • … • put on ROM and RAM • executable is build with *.c, *.rb and mruby core mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) generator mruby core app.rb executable
27. nvgen
28. nvgen • Non-Volatile mruby Generator (mruby-bin-nvgen) • a mrbgem of (modified) mruby • generate Symbol tables (presym.c), objects and related data structures (generated.c) • C structures generated via mrb_open() mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) nvgen
29. nvgen • new app consists of mruby core and generated files (presym.c and generated.c) mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) new app (executable)
30. memory map • ROM • RODATA • RAM • DATA • global variables in C • HEAP • allocated dynamically • STACK • call stack in C • sharing area with HEAP RODATA DATA HEAP STACK
31. memory usage of nvgen 30%"5"
    30. %"5"
    3". )&"1
    3". 0CKFDU
    37"-6& NFUIPE
    UBCMF
    BEEFE NSC@TUBUF 4ZNCPM
    *%
    OBNF *7
    UBCMF TUBDL NFUIPE
    UBCMF
    EFpOFE DPOUFYU 4ZNCPM
    BEEFE *3&1 qBHT<> 0CKFDU
    BEEFE TUBDL
    PSJH NFUIPET
    BEEFE NSC@TUBUF
    PSJH JTFR<>
32. without nvgen class mrb_state iv_tbl internal data mt methods HEAP
33. with nvgen class mrb_state iv_tbl internal data methods ROM mt HEAP DATA flags
34. with nvgen: add methods class mrb_state iv_tbl internal data methods ROM mt methods HEAP DATA flags
35. with nvgen: add class class mrb_state iv_tbl internal data methods ROM mt methods HEAP DATA class flags
36. with nvgen: new objects class mrb_state iv_tbl internal data methods ROM mt methods HEAP DATA class object flags
37. algorythm of nvgen collect objects scan information to dump dump symbols dump objects and methods dump mruby VM (mrb_state, callinfos, stack)
38. mruby’s objects in C mrb_value mrb_value Integer, Symbol Object, Class RVALUE String mrb_value RVALUE char[] Array mrb_value RVALUE mrb_value[]
39. mrb_value • TT + mrb_int • Integer • TT + mrb_sym • Symbol • TT + pointer • Other Classes • TT: Type Tag; type of value (0-24) TT void *p (32bit pointer) mrb_int (32bit int) mrb_sym (32bit uint) mrb_value TT TT Symbol Fixnum Other
40. RVALUE • common items: MRB_OBJECT_HEADER • class-specific items ex) RClass • common object header • pointer to instance variables • pointer to method table • pointer to super class MRB_OBJECT_HEADER *iv (instance variables) *mt (method table) *super (super class) RClass
41. RVALUE class Foo def initialize(name) @name = name end def say puts “#{@name}!!” end end foo = Foo.new MRB_OBJECT_HEADER class Foo •initialize •say class Object mt super iv
42. Method structure • struct RProc (RVALUE) • method defined in C • pointer to C function • method defined in Ruby • compiled as bytecode • IREP (mrb_irep) structure • IREP can be nested RProc irep RProc c function in C in Ruby irep irep
43. def hello(name) puts name end hello("Hello!") IREP data structure of code (method / block)
44. def hello(name) puts name end hello("Hello!") IREP irep irep data structure of code (method / block)
45. def hello(name) puts name end hello("Hello!") IREP • nlocals (num. of local variables) • nregs (num. of registers) • ISEQ (bytecode) • pools • symbols • reps (pointer to IREPs) • rlen (num. of sub IREPs) • … data structure of code (method / block) irep irep
46. IREP (mrbc) irep 0x7fe595c0cb20 nregs=4 nlocals=1 pools=1 syms=1 reps=1 iseq=20 file: hello.rb 1 000 OP_TCLASS R1 1 002 OP_METHOD R2 I(0:0x7fe595c0cdd0) 1 005 OP_DEF R1 :hello 5 008 OP_LOADSELF R1 5 010 OP_STRING R2 L(0) ; "Hello!" 5 013 OP_SEND R1 :hello 1 5 017 OP_RETURN R1 5 019 OP_STOP irep 0x7fe595c0cdd0 nregs=6 nlocals=3 pools=0 syms=1 reps=0 iseq=15 local variable names: R1:name R2:& file: hello.rb 1 000 OP_ENTER 1:0:0:0:0:0:0 2 004 OP_LOADSELF R3 2 006 OP_MOVE R4 R1 ; R1:name 2 009 OP_SEND R3 :puts 1 2 013 OP_RETURN R3 def hello(name) puts name end hello("Hello!")
47. problem and solution of nvgen
48. flags of objects • obj->flags is used as status of objects • flags should be mutable • So an array of flags for ROM’s objects are located in RAM • add macro to read and write them • obj->flags in ROM are indexes of the array flags flags flags flags flags flags … ROM RAM
49. GC with RAM and ROM • mruby uses tri-color GC (Black, White, Gray) • We use quad-color GC, add Red • Red are never GCed; objects in ROM are Red • At first of GC cycle, all children of Red objects are marked • VM has a list of Red objects ROM RAM
50. methods with RAM and ROM • method tables are in both ROM and RAM • tables for pre-defined methods are in ROM • using iv_tbl to store pointer of pre-defined method tables • key is __mt2__, which is not used as instance variables new methods pre-defined methods __mt2__ *mt *iv_tbl
51. Demo
52. IoT Demo • Sensor • 6-axis motion sensor • Network • WiFi • Protocol • MQTT publish sensor data to server Server sensor Device MQTT
53. Motion Sensor • X • Y • Z • Roll • Pitch • Yaw MPU-6886 6 axis motion sensor (IMU) Device Z Y X Pitch Roll Yaw sensor
54. aws_iot.rb accx, accy, accz = mpu.accel_data pitch, roll, yaw = mpu.gyro_data temp = mpu.temp_data t = M5StickC::Time.now LCD.printf("ah pitch:%2.2f roll:%2.2f, yaw:%2.2f\n", pitch, roll, yaw) message = sprintf(‘{"timestamp":"%s","gyroX": %2.2f,"gyroY": %2.2f,"gyroZ": %2.2f,"accX": %. 2f,"accY": %.2f,"accZ": %.2f,"temp": %.2f}', t.xmlschema, pitch, roll, yaw, accx * 1000, accy * 1000, accz * 1000, temp) aws.publish(pub_topic, message)
55. IoT Demo • AWS IoT • MQTT Broker • Elasticsearch • collect and analyze data from AWS IoT • Kibana • visualize on dashboard publish sensor data to server AWS IoT Elasticseaerch Device Kibana
56. Let's try!
57. Benchmark
58.      EFGBVMU  /7    
59. Future Works
60. Future Work • support 2.1.0 • feedback to mruby/mruby • make available without modification of mruby
61. Special Thanks to: • The Ruby Association • This work is supported Ruby Association grant program in 2018 • The mruby forum • Aaron Patterson @tenderlove • Kishimaworks
62. more information http://yamanekko.github.io/m5stickc.html
63. Thank you!