What's happening when initializing mruby?

mruby and its initialization.
A problem of initialization of mruby and how to improve it.
How to use mruby on M5Stick-C.

Transcript

1. What's happening
   when initializing mruby?
   2019-11-20 RubyConf 2019
   
   Team Yamanekko
   
   Yurie Yamane, Masayoshi Takahashi
   

   View Slide

2. Outline of this session
   • About us
   
   • What’s mruby
   
   • mruby demo (m5stickc)
   
   • What’s initialization
   
   • A problem of initialization
   
   • How to improve initialization
   
   • Introducing mruby-bin-nvgen
   
   • nvgen: problem and solution
   
   • Result
   

   View Slide

3. About Us
   

   View Slide

4. Yurie Yamane
   • Toppers Project member
   
   • a project of RTOS and
   Embedded Systems
   
   • “Talking about mruby with
   Matz: Today and
   Tomorrow” @ET2019 in
   this night TOPAME
   

   View Slide

5. Masayoshi Takahashi
   • +20 Rubyist
   
   • RubyKaigi organizer team
   
   • RubyKaigi 2020 in next April, at
   matsumoto city
   

   View Slide

6. “Introduction to mruby”
   • using mruby with C API
   
   • self-publishing zine in Japanese
   
   • 108 pages
   We are going to translate this
   book into English.
   

   View Slide

7. What's happening
   when initializing mruby?
   

   View Slide

8. mruby
   

   View Slide

9. What’s mruby?
   • Lightweight implementation of Ruby
   
   • comply to ISO 30170:2012
   
   • Used in embedded devices and in server applications
   
   • 2.1.0 will be released in this week
   was
   

   View Slide

10. M5StickC
    ESP32-based
    
    4MByte Flash, 520 KB SRAM
    
    6-Axis IMU
    
    0.96 inch LCD
    
    WiFi + Bluetooth
    
    80 mAh Lipo Battery
    
    Grove Port
    

    View Slide

11. M5StickC
    ESP32-based
    
    4MByte Flash, 520 KB SRAM
    
    6-Axis IMU
    
    0.96 inch LCD
    
    WiFi + Bluetooth
    
    80 mAh Lipo Battery
    
    Grove Port
    

    View Slide

12. demo
    

    View Slide

13. What's happening
    when initializing mruby?
    

    View Slide

14. Quiz
    How many objects
    
    created in initialization?
    (1) 68
    (2) 105
    (3) 212 Hint: class is object
    

    View Slide

15. What’s initialization
    “Initialization” means in this talk:
    == everything before executing your Ruby scripts
    

    View Slide

16. What’s initialization
    “Initialization” means in this talk:
    == everything before executing your Ruby scripts
    •initializing mruby VM
    •initializing GC system
    •initializing all Classes and Modules
    •creating Object, String, Integer, Array, …
    •defining methods for all classes / modules
    

    View Slide

17. • components of VM
    • context
    • callinfo
    • stack
    • create new context
    • create 1 callinfo array and
    1 stak on the context
    initializing mruby VM
    mruby VM (mrb_state)
    context …
    callinfo
    stack
    

    View Slide

18. • initialize memories for
    Objects
    • fixed-size memory spaces
    initializing GC system
    RVALUE (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE)
    

    View Slide

19. • create RClass (RVALUE)
    • create related data
    • methods of the class
    • classname
    • constants
    • create symbols
    initializing Class/Module
    Object
    String
    RClass
    RClass
    Exception
    RClass
    methods
    ……
    other data
    methods
    other data
    methods
    other data
    

    View Slide

20. Problem of Initialization
    

    View Slide

21. Problem of initialization
    • memory consumption
    • initialize classes, methods…
    • but almost all classes/modules are not changed
    • micro controller has small memory
    • mruby is (relatively) not enough to small
    

    View Slide

22. How to improve it
    

    View Slide

23. How to improve it
    • In general, ROM is wider
    than RAM in micro controller
    •
    RAM ROM
    LEGO MindStorms 64MB 16MB
    M5StickC 520KB 4MB
    SparkFun Pro
    nRF52840 Mini
    
    256KB 1MB
    nucleo F412ZG 256KB 1024KB
    nucleo F411RE 128KB 512KB
    

    View Slide

24. How to put data on ROM
    • We can’t put data on Flash ROM
    dynamically
    
    • because it’s ROM!
    
    • We can put them as source code
    in C
    
    • use keyword const
    
    • pre-execution and code
    generation
    const struct RClass nv_object_212 = {
    
    .tt=MRB_TT_CLASS,
    
    .color=7,
    
    .flags=212,
    
    .c=(struct RClass *)&nv_object_211,
    
    .gcnext=
    
    (struct RBasic*)&nv_object_210,
    
    .iv=(struct iv_tbl*)&nv_iv_90,
    
    .mt=(struct kh_mt*)&nv_mt_86_empty,
    
    .super =NULL,
    
    /* name=BasicObject */
    
    };
    

    View Slide

25. pre-execution
    A = Array.new
    
    A << 1
    
    if A.size >= 1
    
    A += [2, 3]
    
    end
    
    A = [1, 2, 3]
    pre-
    execution
    original generated
    

    View Slide

26. e
    code generation
    generator and executable
    
    • generator generates *.c files
    
    • symbols
    
    • objects
    
    • …
    
    • put on ROM and RAM
    
    • executable is build with *.c, *.rb
    and mruby core
    mruby core
    symbol table
    
    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    generator
    mruby core app.rb
    executable
    

    View Slide

27. nvgen
    

    View Slide

28. nvgen
    • Non-Volatile mruby Generator
    (mruby-bin-nvgen)
    
    • a mrbgem of (modified) mruby
    
    • generate Symbol tables
    (presym.c), objects and related
    data structures (generated.c)
    
    • C structures generated via
    mrb_open()
    mruby core
    symbol table
    
    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    nvgen
    

    View Slide

29. nvgen
    • new app consists of mruby core
    and generated files (presym.c and
    generated.c)
    mruby core
    symbol table
    
    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    new app (executable)
    

    View Slide

30. memory map
    • ROM
    
    • RODATA
    
    • RAM
    
    • DATA
    
    • global variables in C
    
    • HEAP
    
    • allocated dynamically
    
    • STACK
    
    • call stack in C
    
    • sharing area with HEAP
    RODATA
    DATA
    HEAP
    STACK
    

    View Slide

31. memory usage of nvgen
    30%"5" 30.
    %"5" 3".
    )&"1 3".
    
    0CKFDU 37"-6&
    NFUIPEUBCMF BEEFE
    NSC@TUBUF
    4ZNCPM*% OBNF *7UBCMF TUBDL
    NFUIPEUBCMF EFpOFE
    DPOUFYU 4ZNCPM BEEFE
    
    *3&1 qBHT<> 0CKFDU BEEFE
    
    TUBDL PSJH
    NFUIPET BEEFE
    
    NSC@TUBUF PSJH
    
    JTFR<>
    

    View Slide

32. without nvgen
    class
    mrb_state
    iv_tbl
    internal data
    mt
    methods
    HEAP
    

    View Slide

33. with nvgen
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    HEAP DATA
    flags
    

    View Slide

34. with nvgen: add methods
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    flags
    

    View Slide

35. with nvgen: add class
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    class
    flags
    

    View Slide

36. with nvgen: new objects
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    class
    object
    flags
    

    View Slide

37. algorythm of nvgen
    collect objects
    scan information to dump
    dump symbols
    dump objects and methods
    dump mruby VM (mrb_state, callinfos, stack)
    

    View Slide

38. mruby’s objects in C
    mrb_value
    mrb_value
    Integer, Symbol
    Object, Class RVALUE
    String mrb_value RVALUE char[]
    Array mrb_value RVALUE mrb_value[]
    

    View Slide

39. mrb_value
    • TT + mrb_int
    
    • Integer
    
    • TT + mrb_sym
    
    • Symbol
    
    • TT + pointer
    
    • Other Classes
    
    • TT: Type Tag; type of value (0-24)
    TT
    void *p
    (32bit pointer)
    mrb_int
    (32bit int)
    mrb_sym
    (32bit uint)
    mrb_value
    TT
    TT
    Symbol
    Fixnum
    Other
    

    View Slide

40. RVALUE
    • common items:
    MRB_OBJECT_HEADER
    
    • class-specific items
    
    ex) RClass
    
    • common object header
    
    • pointer to instance variables
    
    • pointer to method table
    
    • pointer to super class
    MRB_OBJECT_HEADER
    *iv (instance variables)
    *mt (method table)
    *super (super class)
    RClass
    

    View Slide

41. RVALUE
    class Foo
    def initialize(name)
    @name = name
    end
    def say
    puts “#{@name}!!”
    end
    end
    foo = Foo.new
    MRB_OBJECT_HEADER
    class Foo
    •initialize
    •say
    class Object
    mt
    super
    iv
    

    View Slide

42. Method structure
    • struct RProc (RVALUE)
    
    • method defined in C
    
    • pointer to C function
    
    • method defined in Ruby
    
    • compiled as bytecode
    
    • IREP (mrb_irep) structure
    
    • IREP can be nested
    RProc irep
    RProc c function
    in C
    in Ruby
    irep
    irep
    

    View Slide

43. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    data structure of code (method /
    block)
    

    View Slide

44. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    irep
    irep
    data structure of code (method /
    block)
    

    View Slide

45. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    • nlocals (num. of local variables)
    
    • nregs (num. of registers)
    
    • ISEQ (bytecode)
    
    • pools
    
    • symbols
    
    • reps (pointer to IREPs)
    
    • rlen (num. of sub IREPs)
    
    • …
    data structure of code (method /
    block)
    irep
    irep
    

    View Slide

46. IREP (mrbc)
    irep 0x7fe595c0cb20 nregs=4 nlocals=1 pools=1 syms=1 reps=1 iseq=20
    
    file: hello.rb
    
    1 000 OP_TCLASS R1
    
    1 002 OP_METHOD R2 I(0:0x7fe595c0cdd0)
    
    1 005 OP_DEF R1 :hello
    
    5 008 OP_LOADSELF R1
    
    5 010 OP_STRING R2 L(0) ; "Hello!"
    
    5 013 OP_SEND R1 :hello 1
    
    5 017 OP_RETURN R1
    
    5 019 OP_STOP
    
    irep 0x7fe595c0cdd0 nregs=6 nlocals=3 pools=0 syms=1 reps=0 iseq=15
    
    local variable names:
    
    R1:name
    
    R2:&
    
    file: hello.rb
    
    1 000 OP_ENTER 1:0:0:0:0:0:0
    
    2 004 OP_LOADSELF R3
    
    2 006 OP_MOVE R4 R1 ; R1:name
    
    2 009 OP_SEND R3 :puts 1
    
    2 013 OP_RETURN R3
    def hello(name)
    puts name
    end
    hello("Hello!")
    

    View Slide

47. problem and solution of
    nvgen
    

    View Slide

48. flags of objects
    • obj->flags is used as status
    of objects
    • flags should be mutable
    • So an array of flags for ROM’s
    objects are located in RAM
    • add macro to read and write
    them
    • obj->flags in ROM are
    indexes of the array
    flags
    flags
    flags flags
    flags flags …
    ROM
    RAM
    

    View Slide

49. GC with RAM and ROM
    • mruby uses tri-color GC
    (Black, White, Gray)
    • We use quad-color GC, add
    Red
    • Red are never GCed;
    objects in ROM are Red
    • At first of GC cycle, all
    children of Red objects
    are marked
    • VM has a list of Red
    objects
    ROM
    RAM
    

    View Slide

50. methods with RAM and ROM
    • method tables are in
    both ROM and RAM
    • tables for pre-defined
    methods are in ROM
    • using iv_tbl to store
    pointer of pre-defined
    method tables
    • key is __mt2__, which
    is not used as
    instance variables
    new methods
    pre-defined
    methods
    __mt2__
    *mt
    *iv_tbl
    

    View Slide

51. Demo
    

    View Slide

52. IoT Demo
    • Sensor
    
    • 6-axis motion sensor
    
    • Network
    
    • WiFi
    
    • Protocol
    
    • MQTT
    publish sensor data to server
    Server
    sensor
    Device
    MQTT
    

    View Slide

53. Motion Sensor
    • X
    
    • Y
    
    • Z
    
    • Roll
    
    • Pitch
    
    • Yaw
    MPU-6886
    
    6 axis motion sensor (IMU)
    Device
    Z
    Y
    X Pitch
    Roll
    Yaw
    sensor
    

    View Slide

54. aws_iot.rb
    accx, accy, accz = mpu.accel_data
    pitch, roll, yaw = mpu.gyro_data
    temp = mpu.temp_data
    t = M5StickC::Time.now
    LCD.printf("ah pitch:%2.2f roll:%2.2f, yaw:%2.2f\n",
    pitch, roll, yaw)
    message = sprintf(‘{"timestamp":"%s","gyroX":
    %2.2f,"gyroY": %2.2f,"gyroZ": %2.2f,"accX": %.
    2f,"accY": %.2f,"accZ": %.2f,"temp": %.2f}',
    t.xmlschema, pitch, roll, yaw,
    accx * 1000, accy * 1000,
    accz * 1000, temp)
    aws.publish(pub_topic, message)
    

    View Slide

55. IoT Demo
    • AWS IoT
    
    • MQTT Broker
    
    • Elasticsearch
    
    • collect and analyze data from
    AWS IoT
    
    • Kibana
    
    • visualize on dashboard
    publish sensor data to server
    AWS IoT
    Elasticseaerch
    Device
    Kibana
    

    View Slide

56. Let's try!
    

    View Slide

57. Benchmark
    

    View Slide

58. 
    
    
    
    
    EFGBVMU /7
    
    
    
    
    

    View Slide

59. Future Works
    

    View Slide

60. Future Work
    • support 2.1.0
    
    • feedback to mruby/mruby
    
    • make available without modification of mruby
    

    View Slide

61. Special Thanks to:
    • The Ruby Association
    
    • This work is supported Ruby Association grant program in
    2018
    
    • The mruby forum
    
    • Aaron Patterson @tenderlove
    
    • Kishimaworks
    

    View Slide

62. more information
    http://yamanekko.github.io/m5stickc.html
    

    View Slide

63. Thank you!
    

    View Slide