Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's happening when initializing mruby?

Yamanekko
November 20, 2019

What's happening when initializing mruby?

mruby and its initialization.
A problem of initialization of mruby and how to improve it.
How to use mruby on M5Stick-C.

Yamanekko

November 20, 2019
Tweet

More Decks by Yamanekko

Other Decks in Programming

Transcript

  1. What's happening
    when initializing mruby?
    2019-11-20 RubyConf 2019

    Team Yamanekko

    Yurie Yamane, Masayoshi Takahashi

    View full-size slide

  2. Outline of this session
    • About us

    • What’s mruby

    • mruby demo (m5stickc)

    • What’s initialization

    • A problem of initialization

    • How to improve initialization

    • Introducing mruby-bin-nvgen

    • nvgen: problem and solution

    • Result

    View full-size slide

  3. Yurie Yamane
    • Toppers Project member

    • a project of RTOS and
    Embedded Systems

    • “Talking about mruby with
    Matz: Today and
    Tomorrow” @ET2019 in
    this night TOPAME

    View full-size slide

  4. Masayoshi Takahashi
    • +20 Rubyist

    • RubyKaigi organizer team

    • RubyKaigi 2020 in next April, at
    matsumoto city

    View full-size slide

  5. “Introduction to mruby”
    • using mruby with C API

    • self-publishing zine in Japanese

    • 108 pages
    We are going to translate this
    book into English.

    View full-size slide

  6. What's happening
    when initializing mruby?

    View full-size slide

  7. What’s mruby?
    • Lightweight implementation of Ruby

    • comply to ISO 30170:2012

    • Used in embedded devices and in server applications

    • 2.1.0 will be released in this week
    was

    View full-size slide

  8. M5StickC
    ESP32-based

    4MByte Flash, 520 KB SRAM

    6-Axis IMU

    0.96 inch LCD

    WiFi + Bluetooth

    80 mAh Lipo Battery

    Grove Port

    View full-size slide

  9. M5StickC
    ESP32-based

    4MByte Flash, 520 KB SRAM

    6-Axis IMU

    0.96 inch LCD

    WiFi + Bluetooth

    80 mAh Lipo Battery

    Grove Port

    View full-size slide

  10. What's happening
    when initializing mruby?

    View full-size slide

  11. Quiz
    How many objects

    created in initialization?
    (1) 68
    (2) 105
    (3) 212 Hint: class is object

    View full-size slide

  12. What’s initialization
    “Initialization” means in this talk:
    == everything before executing your Ruby scripts

    View full-size slide

  13. What’s initialization
    “Initialization” means in this talk:
    == everything before executing your Ruby scripts
    •initializing mruby VM
    •initializing GC system
    •initializing all Classes and Modules
    •creating Object, String, Integer, Array, …
    •defining methods for all classes / modules

    View full-size slide

  14. • components of VM
    • context
    • callinfo
    • stack
    • create new context
    • create 1 callinfo array and
    1 stak on the context
    initializing mruby VM
    mruby VM (mrb_state)
    context …
    callinfo
    stack

    View full-size slide

  15. • initialize memories for
    Objects
    • fixed-size memory spaces
    initializing GC system
    RVALUE (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE) (RVALUE)
    (RVALUE)
    (RVALUE)
    (RVALUE)

    View full-size slide

  16. • create RClass (RVALUE)
    • create related data
    • methods of the class
    • classname
    • constants
    • create symbols
    initializing Class/Module
    Object
    String
    RClass
    RClass
    Exception
    RClass
    methods
    ……
    other data
    methods
    other data
    methods
    other data

    View full-size slide

  17. Problem of Initialization

    View full-size slide

  18. Problem of initialization
    • memory consumption
    • initialize classes, methods…
    • but almost all classes/modules are not changed
    • micro controller has small memory
    • mruby is (relatively) not enough to small

    View full-size slide

  19. How to improve it

    View full-size slide

  20. How to improve it
    • In general, ROM is wider
    than RAM in micro controller

    RAM ROM
    LEGO MindStorms 64MB 16MB
    M5StickC 520KB 4MB
    SparkFun Pro
    nRF52840 Mini

    256KB 1MB
    nucleo F412ZG 256KB 1024KB
    nucleo F411RE 128KB 512KB

    View full-size slide

  21. How to put data on ROM
    • We can’t put data on Flash ROM
    dynamically

    • because it’s ROM!

    • We can put them as source code
    in C

    • use keyword const

    • pre-execution and code
    generation
    const struct RClass nv_object_212 = {

    .tt=MRB_TT_CLASS,

    .color=7,

    .flags=212,

    .c=(struct RClass *)&nv_object_211,

    .gcnext=

    (struct RBasic*)&nv_object_210,

    .iv=(struct iv_tbl*)&nv_iv_90,

    .mt=(struct kh_mt*)&nv_mt_86_empty,

    .super =NULL,

    /* name=BasicObject */

    };

    View full-size slide

  22. pre-execution
    A = Array.new

    A << 1

    if A.size >= 1

    A += [2, 3]

    end

    A = [1, 2, 3]
    pre-
    execution
    original generated

    View full-size slide

  23. e
    code generation
    generator and executable

    • generator generates *.c files

    • symbols

    • objects

    • …

    • put on ROM and RAM

    • executable is build with *.c, *.rb
    and mruby core
    mruby core
    symbol table

    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    generator
    mruby core app.rb
    executable

    View full-size slide

  24. nvgen
    • Non-Volatile mruby Generator
    (mruby-bin-nvgen)

    • a mrbgem of (modified) mruby

    • generate Symbol tables
    (presym.c), objects and related
    data structures (generated.c)

    • C structures generated via
    mrb_open()
    mruby core
    symbol table

    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    nvgen

    View full-size slide

  25. nvgen
    • new app consists of mruby core
    and generated files (presym.c and
    generated.c)
    mruby core
    symbol table

    (presym.c)
    objects,
    method tables,
    strings, …
    (generated.c)
    new app (executable)

    View full-size slide

  26. memory map
    • ROM

    • RODATA

    • RAM

    • DATA

    • global variables in C

    • HEAP

    • allocated dynamically

    • STACK

    • call stack in C

    • sharing area with HEAP
    RODATA
    DATA
    HEAP
    STACK

    View full-size slide

  27. memory usage of nvgen
    30%"5" 30.
    %"5" 3".
    )&"1 3".

    0CKFDU 37"-6&
    NFUIPEUBCMF BEEFE
    NSC@TUBUF
    4ZNCPM*% OBNF *7UBCMF TUBDL
    NFUIPEUBCMF EFpOFE
    DPOUFYU 4ZNCPM BEEFE

    *3&1 qBHT<> 0CKFDU BEEFE

    TUBDL PSJH
    NFUIPET BEEFE

    NSC@TUBUF PSJH

    JTFR<>

    View full-size slide

  28. without nvgen
    class
    mrb_state
    iv_tbl
    internal data
    mt
    methods
    HEAP

    View full-size slide

  29. with nvgen
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    HEAP DATA
    flags

    View full-size slide

  30. with nvgen: add methods
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    flags

    View full-size slide

  31. with nvgen: add class
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    class
    flags

    View full-size slide

  32. with nvgen: new objects
    class
    mrb_state
    iv_tbl
    internal data
    methods
    ROM
    mt
    methods
    HEAP DATA
    class
    object
    flags

    View full-size slide

  33. algorythm of nvgen
    collect objects
    scan information to dump
    dump symbols
    dump objects and methods
    dump mruby VM (mrb_state, callinfos, stack)

    View full-size slide

  34. mruby’s objects in C
    mrb_value
    mrb_value
    Integer, Symbol
    Object, Class RVALUE
    String mrb_value RVALUE char[]
    Array mrb_value RVALUE mrb_value[]

    View full-size slide

  35. mrb_value
    • TT + mrb_int

    • Integer

    • TT + mrb_sym

    • Symbol

    • TT + pointer

    • Other Classes

    • TT: Type Tag; type of value (0-24)
    TT
    void *p
    (32bit pointer)
    mrb_int
    (32bit int)
    mrb_sym
    (32bit uint)
    mrb_value
    TT
    TT
    Symbol
    Fixnum
    Other

    View full-size slide

  36. RVALUE
    • common items:
    MRB_OBJECT_HEADER

    • class-specific items

    ex) RClass

    • common object header

    • pointer to instance variables

    • pointer to method table

    • pointer to super class
    MRB_OBJECT_HEADER
    *iv (instance variables)
    *mt (method table)
    *super (super class)
    RClass

    View full-size slide

  37. RVALUE
    class Foo
    def initialize(name)
    @name = name
    end
    def say
    puts “#{@name}!!”
    end
    end
    foo = Foo.new
    MRB_OBJECT_HEADER
    class Foo
    •initialize
    •say
    class Object
    mt
    super
    iv

    View full-size slide

  38. Method structure
    • struct RProc (RVALUE)

    • method defined in C

    • pointer to C function

    • method defined in Ruby

    • compiled as bytecode

    • IREP (mrb_irep) structure

    • IREP can be nested
    RProc irep
    RProc c function
    in C
    in Ruby
    irep
    irep

    View full-size slide

  39. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    data structure of code (method /
    block)

    View full-size slide

  40. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    irep
    irep
    data structure of code (method /
    block)

    View full-size slide

  41. def hello(name)
    puts name
    end
    hello("Hello!")
    IREP
    • nlocals (num. of local variables)

    • nregs (num. of registers)

    • ISEQ (bytecode)

    • pools

    • symbols

    • reps (pointer to IREPs)

    • rlen (num. of sub IREPs)

    • …
    data structure of code (method /
    block)
    irep
    irep

    View full-size slide

  42. IREP (mrbc)
    irep 0x7fe595c0cb20 nregs=4 nlocals=1 pools=1 syms=1 reps=1 iseq=20

    file: hello.rb

    1 000 OP_TCLASS R1

    1 002 OP_METHOD R2 I(0:0x7fe595c0cdd0)

    1 005 OP_DEF R1 :hello

    5 008 OP_LOADSELF R1

    5 010 OP_STRING R2 L(0) ; "Hello!"

    5 013 OP_SEND R1 :hello 1

    5 017 OP_RETURN R1

    5 019 OP_STOP

    irep 0x7fe595c0cdd0 nregs=6 nlocals=3 pools=0 syms=1 reps=0 iseq=15

    local variable names:

    R1:name

    R2:&

    file: hello.rb

    1 000 OP_ENTER 1:0:0:0:0:0:0

    2 004 OP_LOADSELF R3

    2 006 OP_MOVE R4 R1 ; R1:name

    2 009 OP_SEND R3 :puts 1

    2 013 OP_RETURN R3
    def hello(name)
    puts name
    end
    hello("Hello!")

    View full-size slide

  43. problem and solution of
    nvgen

    View full-size slide

  44. flags of objects
    • obj->flags is used as status
    of objects
    • flags should be mutable
    • So an array of flags for ROM’s
    objects are located in RAM
    • add macro to read and write
    them
    • obj->flags in ROM are
    indexes of the array
    flags
    flags
    flags flags
    flags flags …
    ROM
    RAM

    View full-size slide

  45. GC with RAM and ROM
    • mruby uses tri-color GC
    (Black, White, Gray)
    • We use quad-color GC, add
    Red
    • Red are never GCed;
    objects in ROM are Red
    • At first of GC cycle, all
    children of Red objects
    are marked
    • VM has a list of Red
    objects
    ROM
    RAM

    View full-size slide

  46. methods with RAM and ROM
    • method tables are in
    both ROM and RAM
    • tables for pre-defined
    methods are in ROM
    • using iv_tbl to store
    pointer of pre-defined
    method tables
    • key is __mt2__, which
    is not used as
    instance variables
    new methods
    pre-defined
    methods
    __mt2__
    *mt
    *iv_tbl

    View full-size slide

  47. IoT Demo
    • Sensor

    • 6-axis motion sensor

    • Network

    • WiFi

    • Protocol

    • MQTT
    publish sensor data to server
    Server
    sensor
    Device
    MQTT

    View full-size slide

  48. Motion Sensor
    • X

    • Y

    • Z

    • Roll

    • Pitch

    • Yaw
    MPU-6886

    6 axis motion sensor (IMU)
    Device
    Z
    Y
    X Pitch
    Roll
    Yaw
    sensor

    View full-size slide

  49. aws_iot.rb
    accx, accy, accz = mpu.accel_data
    pitch, roll, yaw = mpu.gyro_data
    temp = mpu.temp_data
    t = M5StickC::Time.now
    LCD.printf("ah pitch:%2.2f roll:%2.2f, yaw:%2.2f\n",
    pitch, roll, yaw)
    message = sprintf(‘{"timestamp":"%s","gyroX":
    %2.2f,"gyroY": %2.2f,"gyroZ": %2.2f,"accX": %.
    2f,"accY": %.2f,"accZ": %.2f,"temp": %.2f}',
    t.xmlschema, pitch, roll, yaw,
    accx * 1000, accy * 1000,
    accz * 1000, temp)
    aws.publish(pub_topic, message)

    View full-size slide

  50. IoT Demo
    • AWS IoT

    • MQTT Broker

    • Elasticsearch

    • collect and analyze data from
    AWS IoT

    • Kibana

    • visualize on dashboard
    publish sensor data to server
    AWS IoT
    Elasticseaerch
    Device
    Kibana

    View full-size slide

  51. Future Works

    View full-size slide

  52. Future Work
    • support 2.1.0

    • feedback to mruby/mruby

    • make available without modification of mruby

    View full-size slide

  53. Special Thanks to:
    • The Ruby Association

    • This work is supported Ruby Association grant program in
    2018

    • The mruby forum

    • Aaron Patterson @tenderlove

    • Kishimaworks

    View full-size slide

  54. more information
    http://yamanekko.github.io/m5stickc.html

    View full-size slide