What's happening when initializing mruby?

644ee43c84fe676bbbdb658691b373ac?s=47 Yamanekko
November 20, 2019

What's happening when initializing mruby?

mruby and its initialization.
A problem of initialization of mruby and how to improve it.
How to use mruby on M5Stick-C.

644ee43c84fe676bbbdb658691b373ac?s=128

Yamanekko

November 20, 2019
Tweet

Transcript

  1. 2.

    Outline of this session • About us • What’s mruby

    • mruby demo (m5stickc) • What’s initialization • A problem of initialization • How to improve initialization • Introducing mruby-bin-nvgen • nvgen: problem and solution • Result
  2. 4.

    Yurie Yamane • Toppers Project member • a project of

    RTOS and Embedded Systems • “Talking about mruby with Matz: Today and Tomorrow” @ET2019 in this night TOPAME
  3. 5.

    Masayoshi Takahashi • +20 Rubyist • RubyKaigi organizer team •

    RubyKaigi 2020 in next April, at matsumoto city
  4. 6.

    “Introduction to mruby” • using mruby with C API •

    self-publishing zine in Japanese • 108 pages We are going to translate this book into English.
  5. 8.
  6. 9.

    What’s mruby? • Lightweight implementation of Ruby • comply to

    ISO 30170:2012 • Used in embedded devices and in server applications • 2.1.0 will be released in this week was
  7. 10.

    M5StickC ESP32-based 4MByte Flash, 520 KB SRAM 6-Axis IMU 0.96

    inch LCD WiFi + Bluetooth 80 mAh Lipo Battery Grove Port
  8. 11.

    M5StickC ESP32-based 4MByte Flash, 520 KB SRAM 6-Axis IMU 0.96

    inch LCD WiFi + Bluetooth 80 mAh Lipo Battery Grove Port
  9. 12.
  10. 14.
  11. 16.

    What’s initialization “Initialization” means in this talk: == everything before

    executing your Ruby scripts •initializing mruby VM •initializing GC system •initializing all Classes and Modules •creating Object, String, Integer, Array, … •defining methods for all classes / modules
  12. 17.

    • components of VM • context • callinfo • stack

    • create new context • create 1 callinfo array and 1 stak on the context initializing mruby VM mruby VM (mrb_state) context … callinfo stack
  13. 18.

    • initialize memories for Objects • fixed-size memory spaces initializing

    GC system RVALUE (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE) (RVALUE)
  14. 19.

    • create RClass (RVALUE) • create related data • methods

    of the class • classname • constants • create symbols initializing Class/Module Object String RClass RClass Exception RClass methods …… other data methods other data methods other data
  15. 21.

    Problem of initialization • memory consumption • initialize classes, methods…

    • but almost all classes/modules are not changed • micro controller has small memory • mruby is (relatively) not enough to small
  16. 23.

    How to improve it • In general, ROM is wider

    than RAM in micro controller • RAM ROM LEGO MindStorms 64MB 16MB M5StickC 520KB 4MB SparkFun Pro nRF52840 Mini 256KB 1MB nucleo F412ZG 256KB 1024KB nucleo F411RE 128KB 512KB
  17. 24.

    How to put data on ROM • We can’t put

    data on Flash ROM dynamically • because it’s ROM! • We can put them as source code in C • use keyword const • pre-execution and code generation const struct RClass nv_object_212 = { .tt=MRB_TT_CLASS, .color=7, .flags=212, .c=(struct RClass *)&nv_object_211, .gcnext= (struct RBasic*)&nv_object_210, .iv=(struct iv_tbl*)&nv_iv_90, .mt=(struct kh_mt*)&nv_mt_86_empty, .super =NULL, /* name=BasicObject */ };
  18. 25.

    pre-execution A = Array.new A << 1 if A.size >=

    1 A += [2, 3] end A = [1, 2, 3] pre- execution original generated
  19. 26.

    e code generation generator and executable • generator generates *.c

    files • symbols • objects • … • put on ROM and RAM • executable is build with *.c, *.rb and mruby core mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) generator mruby core app.rb executable
  20. 27.
  21. 28.

    nvgen • Non-Volatile mruby Generator (mruby-bin-nvgen) • a mrbgem of

    (modified) mruby • generate Symbol tables (presym.c), objects and related data structures (generated.c) • C structures generated via mrb_open() mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) nvgen
  22. 29.

    nvgen • new app consists of mruby core and generated

    files (presym.c and generated.c) mruby core symbol table (presym.c) objects, method tables, strings, … (generated.c) new app (executable)
  23. 30.

    memory map • ROM • RODATA • RAM • DATA

    • global variables in C • HEAP • allocated dynamically • STACK • call stack in C • sharing area with HEAP RODATA DATA HEAP STACK
  24. 31.

    memory usage of nvgen 30%"5" 30. %"5" 3". )&"1 3".

    0CKFDU 37"-6& NFUIPEUBCMF BEEFE NSC@TUBUF 4ZNCPM*% OBNF *7UBCMF TUBDL NFUIPEUBCMF EFpOFE DPOUFYU 4ZNCPM BEEFE *3&1 qBHT<> 0CKFDU BEEFE TUBDL PSJH NFUIPET BEEFE NSC@TUBUF PSJH JTFR<>
  25. 35.
  26. 36.

    with nvgen: new objects class mrb_state iv_tbl internal data methods

    ROM mt methods HEAP DATA class object flags
  27. 37.

    algorythm of nvgen collect objects scan information to dump dump

    symbols dump objects and methods dump mruby VM (mrb_state, callinfos, stack)
  28. 38.

    mruby’s objects in C mrb_value mrb_value Integer, Symbol Object, Class

    RVALUE String mrb_value RVALUE char[] Array mrb_value RVALUE mrb_value[]
  29. 39.

    mrb_value • TT + mrb_int • Integer • TT +

    mrb_sym • Symbol • TT + pointer • Other Classes • TT: Type Tag; type of value (0-24) TT void *p (32bit pointer) mrb_int (32bit int) mrb_sym (32bit uint) mrb_value TT TT Symbol Fixnum Other
  30. 40.

    RVALUE • common items: MRB_OBJECT_HEADER • class-specific items ex) RClass

    • common object header • pointer to instance variables • pointer to method table • pointer to super class MRB_OBJECT_HEADER *iv (instance variables) *mt (method table) *super (super class) RClass
  31. 41.

    RVALUE class Foo def initialize(name) @name = name end def

    say puts “#{@name}!!” end end foo = Foo.new MRB_OBJECT_HEADER class Foo •initialize •say class Object mt super iv
  32. 42.

    Method structure • struct RProc (RVALUE) • method defined in

    C • pointer to C function • method defined in Ruby • compiled as bytecode • IREP (mrb_irep) structure • IREP can be nested RProc irep RProc c function in C in Ruby irep irep
  33. 45.

    def hello(name) puts name end hello("Hello!") IREP • nlocals (num.

    of local variables) • nregs (num. of registers) • ISEQ (bytecode) • pools • symbols • reps (pointer to IREPs) • rlen (num. of sub IREPs) • … data structure of code (method / block) irep irep
  34. 46.

    IREP (mrbc) irep 0x7fe595c0cb20 nregs=4 nlocals=1 pools=1 syms=1 reps=1 iseq=20

    file: hello.rb 1 000 OP_TCLASS R1 1 002 OP_METHOD R2 I(0:0x7fe595c0cdd0) 1 005 OP_DEF R1 :hello 5 008 OP_LOADSELF R1 5 010 OP_STRING R2 L(0) ; "Hello!" 5 013 OP_SEND R1 :hello 1 5 017 OP_RETURN R1 5 019 OP_STOP irep 0x7fe595c0cdd0 nregs=6 nlocals=3 pools=0 syms=1 reps=0 iseq=15 local variable names: R1:name R2:& file: hello.rb 1 000 OP_ENTER 1:0:0:0:0:0:0 2 004 OP_LOADSELF R3 2 006 OP_MOVE R4 R1 ; R1:name 2 009 OP_SEND R3 :puts 1 2 013 OP_RETURN R3 def hello(name) puts name end hello("Hello!")
  35. 48.

    flags of objects • obj->flags is used as status of

    objects • flags should be mutable • So an array of flags for ROM’s objects are located in RAM • add macro to read and write them • obj->flags in ROM are indexes of the array flags flags flags flags flags flags … ROM RAM
  36. 49.

    GC with RAM and ROM • mruby uses tri-color GC

    (Black, White, Gray) • We use quad-color GC, add Red • Red are never GCed; objects in ROM are Red • At first of GC cycle, all children of Red objects are marked • VM has a list of Red objects ROM RAM
  37. 50.

    methods with RAM and ROM • method tables are in

    both ROM and RAM • tables for pre-defined methods are in ROM • using iv_tbl to store pointer of pre-defined method tables • key is __mt2__, which is not used as instance variables new methods pre-defined methods __mt2__ *mt *iv_tbl
  38. 51.
  39. 52.

    IoT Demo • Sensor • 6-axis motion sensor • Network

    • WiFi • Protocol • MQTT publish sensor data to server Server sensor Device MQTT
  40. 53.

    Motion Sensor • X • Y • Z • Roll

    • Pitch • Yaw MPU-6886 6 axis motion sensor (IMU) Device Z Y X Pitch Roll Yaw sensor
  41. 54.

    aws_iot.rb accx, accy, accz = mpu.accel_data pitch, roll, yaw =

    mpu.gyro_data temp = mpu.temp_data t = M5StickC::Time.now LCD.printf("ah pitch:%2.2f roll:%2.2f, yaw:%2.2f\n", pitch, roll, yaw) message = sprintf(‘{"timestamp":"%s","gyroX": %2.2f,"gyroY": %2.2f,"gyroZ": %2.2f,"accX": %. 2f,"accY": %.2f,"accZ": %.2f,"temp": %.2f}', t.xmlschema, pitch, roll, yaw, accx * 1000, accy * 1000, accz * 1000, temp) aws.publish(pub_topic, message)
  42. 55.

    IoT Demo • AWS IoT • MQTT Broker • Elasticsearch

    • collect and analyze data from AWS IoT • Kibana • visualize on dashboard publish sensor data to server AWS IoT Elasticseaerch Device Kibana
  43. 57.
  44. 60.

    Future Work • support 2.1.0 • feedback to mruby/mruby •

    make available without modification of mruby
  45. 61.

    Special Thanks to: • The Ruby Association • This work

    is supported Ruby Association grant program in 2018 • The mruby forum • Aaron Patterson @tenderlove • Kishimaworks