Upgrade to Pro — share decks privately, control downloads, hide ads and more …

When a robot is smart enough?

Tomas Jukin
November 04, 2017

When a robot is smart enough?

Slides from my talk at DevFest Prague 2017 about few thoughts on a topic of "When a robot is smart enough" with examples how you can build one at home from Open Hardware components.

Ideas were demonstrated on a showcase of my #Probee smart robotic car build using #Arduino and #RaspberryPi.

All code is open source at https://github.com/Juicymo/iot-probee

Link to event: https://2017.devfest.cz/schedule/day1?sessionId=123

Talk was presented on 2017/11/04

#DevFestCZ #Probee

Tomas Jukin

November 04, 2017
Tweet

More Decks by Tomas Jukin

Other Decks in Technology

Transcript

  1. When a Robot is smart
    enough?
    from Arduino, via RaspberryPi, to ROS
    Tomáš Jukin
    @Inza

    View full-size slide

  2. Having all features needed for
    fulfilment of its primary
    function and purpose makes a
    Robot smart enough.

    View full-size slide

  3. Tomáš Jukin
    @Inza
    www.juicymo.cz
    @JuicymoCZ

    View full-size slide

  4. Tomáš Jukin
    @Inza
    www.juicymo.cz
    @JuicymoCZ
    CTO
    IoT
    AI
    Robotics

    View full-size slide

  5. github.com/Juicymo/iot-probee
    @Inza
    @JuicymoCZ

    View full-size slide

  6. #Probee Robot

    View full-size slide

  7. #Probee v3.0
    Ruby
    PlatformIO
    {
    {

    View full-size slide

  8. #ProbeeEvolution

    View full-size slide

  9. #ProbeeEvolution

    View full-size slide

  10. #ProbeeEvolution

    View full-size slide

  11. #ProbeeEvolution

    View full-size slide

  12. GDG ohwHack 2014 mDevCamp 2016
    #ProbeeEvolution
    DevFest Prague 2017
    v0 v1 v2
    v3

    View full-size slide

  13. GDG ohwHack 2014 mDevCamp 2016
    #ProbeeEvolution
    DevFest Prague 2017
    ?
    v0 v1 v2
    v3

    View full-size slide

  14. Robotics 101

    View full-size slide

  15. Robotics 101
    Micro-Computer
    vs.
    Micro-Controller

    View full-size slide

  16. Robotics 101
    Micro-Computer
    vs.
    Micro-Controller

    View full-size slide

  17. Robotics 101
    Micro-Computer
    vs.
    Micro-Controller

    View full-size slide

  18. Robotics 101
    Micro-Computer
    vs.
    Micro-Controller
    Voltage: 3.3V
    Flash: SD card
    RAM: 512 MB (CPU & GPU)
    CPU: ARM11 (BCM2835 SoC)
    Clock: 700 MHz
    GPIO: 40 pin
    HDMI, Ethernet
    RaspberryPi 1
    B+
    Arduino
    UNO
    Voltage: 5V
    Flash: 32KB (-0.5 bootloader)
    SRAM: 2KB
    EEPROM: 1KB
    Clock: 16 MHz
    Digital I/O: 14/6 PWM
    Analog In: 6

    View full-size slide

  19. Robotics 101
    ways of communication

    View full-size slide

  20. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication

    View full-size slide

  21. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication
    Serial Peripheral Interface, N+3 wires, N devices
    SPI

    View full-size slide

  22. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication
    Serial Peripheral Interface, N+3 wires, N devices
    Universal Asynchronous Receiver/Transmitter, 2 wires, 2 devices, <20m
    SPI
    UART

    View full-size slide

  23. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication
    Serial Peripheral Interface, N+3 wires, N devices
    Universal Asynchronous Receiver/Transmitter, 2 wires, 2 devices, <20m
    Full Serial Line, 9 wires, 2 devices, <20m
    RS-232 (COM port)
    SPI
    UART

    View full-size slide

  24. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication
    Serial Peripheral Interface, N+3 wires, N devices
    Universal Asynchronous Receiver/Transmitter, 2 wires, 2 devices, <20m
    Full Serial Line, 9 wires, 2 devices, <20m
    RS-232 (COM port)
    Industry, 2 wires, <32 devices, <1200m
    RS-485 (EIA-485)
    SPI
    UART

    View full-size slide

  25. Robotics 101
    IIC (i2c, I2C, I2C)
    Inter-Inter Circuit, 2 wires, <128 devices, <2m
    ways of communication
    Serial Peripheral Interface, N+3 wires, N devices
    SPI
    UART
    Industry, 2 wires, <32 devices, <1200m
    RS-485 (EIA-485)
    Universal Asynchronous Receiver/Transmitter, 2 wires, 2 devices, <20m
    Full Serial Line, 9 wires, 2 devices, <20m
    RS-232 (COM port)

    View full-size slide

  26. The Hardware

    View full-size slide

  27. What does a
    4 wheels self driving robotic car
    consist of?

    View full-size slide

  28. 1x RaspberryPi 1 B+ Starter Set https://goo.gl/jaTY70
    1x Chinese 4WD Car Kit with Funduino UNO http://goo.gl/qetuW5
    5x Ultrasonic Sensor HC-SR04 http://goo.gl/YojTTp
    1x Wi-Fi dongle D-Link DWA-127 https://goo.gl/9d9BPo
    1x Arduino Micro https://goo.gl/yjVqgC
    1x I2C Level Converter http://goo.gl/9i4jCK
    1x RTC module DS1307 http://goo.gl/esKm7t
    1x PIR sensor HC-SR501 http://goo.gl/LyYGmb
    1x 4D Touch TFT Display 4DPi-35 Rev. 1 http://goo.gl/RWs7rI
    1x Skewers
    1x Hot Glue
    *plus PiCamera and PowerBank to power Pi… (and cables)

    View full-size slide

  29. http://goo.gl/tnXteh

    View full-size slide

  30. PWM
    12V
    UART
    USB

    View full-size slide

  31. USB
    PWM
    12V
    UART
    USB

    View full-size slide

  32. SPI
    USB
    PWM
    12V
    UART
    USB

    View full-size slide

  33. SPI
    USB
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  34. SPI
    USB
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  35. SPI
    USB
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  36. SPI
    USB
    I2C
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  37. SPI
    USB
    I2C
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  38. SPI
    USB
    I2C
    PWM
    PWM
    12V
    UART
    USB

    View full-size slide

  39. SPI
    USB
    I2C
    PWM
    PWM
    12V
    I2C
    PWM
    UART
    USB

    View full-size slide

  40. SPI
    USB
    I2C
    PWM
    PWM
    12V
    PWM
    UART
    USB

    View full-size slide

  41. The Power ???

    View full-size slide

  42. The Power ???
    UART
    5V
    5V
    USB

    View full-size slide

  43. The Power ???
    I2C
    5V
    3.3V

    View full-size slide

  44. The Power ???
    UART
    5V
    5V
    USB

    View full-size slide

  45. The Power ???
    5V 7V
    UART
    5V
    5V
    USB

    View full-size slide

  46. The Power ???
    5V USB 7V
    3.5V
    3.5V
    7V
    +
    =
    UART
    5V
    5V
    USB

    View full-size slide

  47. How do we develop in
    Arduino?

    View full-size slide

  48. Wiring / Processing

    View full-size slide

  49. 5 Circles of Hell

    View full-size slide

  50. 5 Circles of Hell
    in Arduino

    View full-size slide

  51. 5 Circles of Hell
    in Arduino
    .ino includes

    View full-size slide

  52. 5 Circles of Hell
    in Arduino
    .ino includes
    LCD_i2c.h

    View full-size slide

  53. 5 Circles of Hell
    in Arduino
    .ino includes
    LCD_i2c.h
    Crappy China goods

    View full-size slide

  54. 5 Circles of Hell
    in Arduino
    .ino includes
    LCD_i2c.h
    Crappy China goods
    HW bugs

    View full-size slide

  55. 5 Circles of Hell
    in Arduino
    .ino includes
    LCD_i2c.h
    Crappy China goods
    HW bugs
    The need of switch

    View full-size slide

  56. v0 - Procedural (by MacGyver on ohwHack)
    The Heart

    View full-size slide

  57. v0 - Procedural (by MacGyver on ohwHack)
    The Heart
    v0.5 - “Spagetka” (was to ugly to Git -> lost :-(…)

    View full-size slide

  58. v0 - Procedural (by MacGyver on ohwHack)
    The Heart
    v0.5 - “Spagetka” (was to ugly to Git -> lost :-(…)
    v1 - “Spagetti MAS” (in Git repo)

    View full-size slide

  59. v0 - Procedural (by MacGyver on ohwHack)
    The Heart
    v0.5 - “Spagetka” (was to ugly to Git -> lost :-(…)
    v1 - “Spagetti MAS” (in Git repo)
    v2 - “The Platform” (in Git repo, in progress)

    View full-size slide

  60. v0 - Procedural (by MacGyver on #ohwHack)
    The Heart
    v0.5 - “Spagetka” (was to ugly to Git -> lost :-(…)
    v1 - “Spagetti MAS” (in Git repo)
    v2 - “The Platform” (in Git repo, structure)
    https://github.com/Juicymo/iot-probee
    v3 - “The Platform” (in Git repo, done)

    View full-size slide

  61. http://goo.gl/lZuAzn

    View full-size slide

  62. https://goo.gl/OhNByp
    I2C

    View full-size slide

  63. https://goo.gl/QzYRjs
    https://goo.gl/lG0ggl
    I2C

    View full-size slide

  64. The Heart
    Hardware

    View full-size slide

  65. The Heart
    Hardware
    HAL

    View full-size slide

  66. The Heart
    Hardware
    HAL
    FSM

    View full-size slide

  67. The Heart
    Hardware
    HAL
    FSM
    Pre-Emptive
    Multi-Tasking

    View full-size slide

  68. The Heart
    Hardware
    HAL
    FSM
    Pre-Emptive
    Multi-Tasking
    Config

    View full-size slide

  69. idle reasoning
    5000ms
    forward fast_forward
    slow_forward
    brake
    stop
    10000ms
    10000ms
    100ms
    slow_backward
    backward
    1000ms
    1000ms
    turn_left
    turn_right
    2000ms
    2000ms
    The FSM

    View full-size slide

  70. The FSM
    idle
    reasoning
    FORWARD_OBSTACLE_CLOSE
    FORWARD_OBSTACLE_BLOCKED
    BACKWARD_OBSTACLE_CLOSE
    BACKWARD_OBSTACLE_BLOCKED
    5000ms
    forward
    fast_forward
    FORWARD_OBSTACLE_NONE
    slow_forward
    FORWARD_OBSTACLE_CLOSE
    brake
    FORWARD_OBSTACLE_BLOCKED
    TURRET_OBSTACLE_BLOCKED
    stop
    10000ms
    FORWARD_OBSTACLE_FAR
    FORWARD_OBSTACLE_NEAR
    FORWARD_OBSTACLE_CLOSE
    FORWARD_OBSTACLE_BLOCKED
    TURRET_OBSTACLE_BLOCKED
    10000ms
    FORWARD_OBSTACLE_NONE
    FORWARD_OBSTACLE_FAR
    FORWARD_OBSTACLE_NEAR
    FORWARD_OBSTACLE_BLOCKED
    TURRET_OBSTACLE_BLOCKED
    100ms
    slow_backward
    backward
    FORWARD_OBSTACLE_NONE
    BACKWARD_OBSTACLE_FAR
    BACKWARD_OBSTACLE_NEAR
    BACKWARD_OBSTACLE_BLOCKED
    1000ms
    BACKWARD_OBSTACLE_BLOCKED
    BACKWARD_OBSTACLE_CLOSE
    1000ms
    REASONING_STOPPED
    REASONING_MOVE_FORWARD
    REASONING_MOVE_SLOW_FORWARD
    REASONING_MOVE_SLOW_BACKWARD
    REASONING_MOVE_BACKWARD
    turn_left
    REASONING_TURN_LEFT
    turn_right
    REASONING_TURN_RIGHT
    TURRET_OBSTACLE_PRESENT
    TURRET_OBSTACLE_BLOCKED
    2000ms
    TURRET_OBSTACLE_PRESENT
    TURRET_OBSTACLE_BLOCKED
    2000ms

    View full-size slide

  71. Artificial Intelligence
    101

    View full-size slide

  72. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN

    View full-size slide

  73. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN

    View full-size slide

  74. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN
    spaghetti?? Hierarchical Task Networks
    Hierarchical Finite
    State Machine
    Lets be just
    good enough!

    View full-size slide

  75. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN
    ?

    View full-size slide

  76. HFSM
    Movement FSM
    Stop Drive Slowly Drive Fast

    View full-size slide

  77. HFSM
    Movement FSM
    Stop Drive Slowly Drive Fast
    Turn Turret FSM
    Stop Scan Target

    View full-size slide

  78. HFSM
    Movement FSM
    Stop Drive Slowly Drive Fast
    Turn Turret FSM
    Stop Scan Target
    How to combine them??

    View full-size slide

  79. HFSM
    Movement FSM
    Stop Drive Slowly Drive Fast
    Turn Turret FSM
    Stop Scan Target
    How to combine them??
    Modularity sucks too…

    View full-size slide

  80. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN
    ?

    View full-size slide

  81. How to make AI Logic
    Scripting
    HFSM Heuristics
    MAS GA ANN
    HTN
    Behavior Trees

    View full-size slide

  82. Behavior Trees
    http://aigamedev.com/open/article/bt-overview/
    http://behavior3js.guineashots.com/
    http://aigamedev.com/open/article/behavior-trees-part1/

    View full-size slide

  83. Neural Nets
    HFSM
    MAS
    HTN
    Behavior Trees
    https://github.com/Inza/agens
    https://en.wikipedia.org/wiki/Multi-agent_system
    https://github.com/aasm/aasm
    https://en.wikipedia.org/wiki/Finite-state_machine
    https://github.com/Maumagnaguagno/HyperTensioN
    https://en.wikipedia.org/wiki/Hierarchical_task_network
    https://github.com/jvoegele/behavior_tree
    https://en.wikipedia.org/wiki/Behavior_tree
    https://github.com/SergioFierens/ai4r
    https://en.wikipedia.org/wiki/Artificial_neural_network

    View full-size slide

  84. How do we develop
    for RaspberryPi?

    View full-size slide

  85. But not only. It is a full
    featured* Linux.

    View full-size slide

  86. But not only. It is a full
    featured* Linux.
    *running on ARM… :-(

    View full-size slide

  87. 5 Circles of Hell

    View full-size slide

  88. 5 Circles of Hell
    in RaspberryPi

    View full-size slide

  89. 5 Circles of Hell
    in RaspberryPi
    USB driven computer

    View full-size slide

  90. 5 Circles of Hell
    in RaspberryPi
    USB driven computer
    Unstable

    View full-size slide

  91. 5 Circles of Hell
    in RaspberryPi
    USB driven computer
    Unstable
    More features madness

    View full-size slide

  92. 5 Circles of Hell
    in RaspberryPi
    USB driven computer
    Unstable
    More features madness
    Wi-Fi dongle

    View full-size slide

  93. 5 Circles of Hell
    in RaspberryPi
    USB driven computer
    Unstable
    More features madness
    Unstable!!!
    Wi-Fi dongle

    View full-size slide

  94. Multi-Agent System
    The Brain
    build on a
    Actor-based concurrent
    object framework in Ruby
    https://github.com/Juicymo/iot-probee

    View full-size slide

  95. The Brain
    Raspbian

    View full-size slide

  96. The Brain
    Raspbian
    rvm

    View full-size slide

  97. The Brain
    Raspbian
    rvm
    ruby

    View full-size slide

  98. The Brain
    Raspbian
    rvm
    ruby
    celluloid

    View full-size slide

  99. The Brain
    Raspbian
    rvm
    ruby
    celluloid
    agens

    View full-size slide

  100. The Brain
    Raspbian
    rvm
    ruby
    celluloid
    agens
    probee MAS

    View full-size slide

  101. The Brain
    Raspbian
    rvm
    ruby
    celluloid
    agens
    probee MAS
    github.com/Juicymo/iot-probee
    https://rubygems.org/gems/agens
    https://github.com/celluloid/celluloid
    https://rvm.io

    View full-size slide

  102. The Brain
    Object
    Ruby OOP
    Probee MAS

    View full-size slide

  103. The Brain
    Object
    Cell
    Ruby OOP
    Celluloid
    Probee MAS

    View full-size slide

  104. The Brain
    Object
    Cell
    Actor
    Ruby OOP
    Celluloid
    Celluloid
    Probee MAS

    View full-size slide

  105. The Brain
    Object
    Cell
    Actor
    Agent
    Ruby OOP
    Celluloid
    Celluloid
    agens
    Probee MAS

    View full-size slide

  106. The Brain
    Object
    Cell
    Actor
    Agent
    MovementAgent
    Ruby OOP
    Celluloid
    Celluloid
    agens
    probee
    Probee MAS

    View full-size slide

  107. The Brain
    MovementAgent
    Probee MAS

    View full-size slide

  108. The Brain
    Motors
    MovementAgent
    Probee MAS

    View full-size slide

  109. The Brain
    Motors
    SteeringAgent
    MovementAgent
    Probee MAS

    View full-size slide

  110. The Brain
    Motors
    Sensors
    SteeringAgent
    MovementAgent
    Probee MAS

    View full-size slide

  111. The Brain
    Motors
    Sensors
    CollisionAgent
    SteeringAgent
    MovementAgent
    Probee MAS

    View full-size slide

  112. The Brain
    Motors
    Sensors
    CollisionAgent
    SteeringAgent
    MovementAgent
    Probee MAS

    View full-size slide

  113. The Brain
    Motors
    Sensors
    CollisionAgent
    SteeringAgent
    MovementAgent
    Probee MAS
    Simple Agents can
    perform complex tasks

    View full-size slide

  114. http://www.ros.org/
    SLAM
    Communication
    Visualization
    Analysis

    View full-size slide

  115. Tomáš Jukin
    @Inza
    www.juicymo.cz
    @JuicymoCZ

    View full-size slide

  116. github.com/Juicymo/iot-probee
    @Inza
    @JuicymoCZ

    View full-size slide