Play Rock-Paper-Scissors-Spock-Lizard with your Android Things

7b5a07956eb0b62be7214d043821a987?s=47 jinqian
November 20, 2018

Play Rock-Paper-Scissors-Spock-Lizard with your Android Things

7b5a07956eb0b62be7214d043821a987?s=128

jinqian

November 20, 2018
Tweet

Transcript

  1. @Xebiconfr #Xebicon18 @bonbonking Jouer à Rock-Paper-Scissors-Spock-Lizard avec son Android Things

    Qian Jin | Technical Officer | qjin@xebia.fr Xebia France 1
  2. @Xebiconfr #Xebicon18 @bonbonking Qian Jin Google Developer Expert IoT Technical

    Officer @ Xebia France @bonbonking 2
  3. @Xebiconfr #Xebicon18 @bonbonking Play Rock-Paper-Scissors-Spock-Lizard "Wacky, Whimsical Machinery with cardboard

    and duct tape" 3
  4. @Xebiconfr #Xebicon18 @bonbonking 4

  5. @Xebiconfr #Xebicon18 @bonbonking Motivation behind the shoebox project 5

  6. @Xebiconfr #Xebicon18 @bonbonking 6 Image Source: https://www.flaticon.com/packs/internet-of-things-45 Cloud Data Centers

    thousands Fog Nodes millions Edge Devices billions
  7. @Xebiconfr #Xebicon18 @bonbonking 7 • Latency • Availability • Bandwidth

    • Security/Privacy
  8. @Xebiconfr #Xebicon18 @bonbonking Explained in xkcd way 8

  9. @Xebiconfr #Xebicon18 @bonbonking 9

  10. @Xebiconfr #Xebicon18 @bonbonking • Data collection tool (raw data to

    train the ML model) • Hardware components (robot arms + game control) • Application running on Android Things ◦ Applicative: game rules, play logic and interactions ◦ Artificial Intelligence: recognize opponent's gesture 10
  11. @Xebiconfr #Xebicon18 @bonbonking Raw Image Crowdsourcing 11

  12. @Xebiconfr #Xebicon18 @bonbonking • Web mobile application: Node.js + Express

    • Hosting/Deployment: Firebase Hosting / Function / Storage 12
  13. @Xebiconfr #Xebicon18 @bonbonking 13

  14. @Xebiconfr #Xebicon18 @bonbonking 14

  15. @Xebiconfr #Xebicon18 @bonbonking 15 Thank you my fellow Xebians <3

  16. @Xebiconfr #Xebicon18 @bonbonking • Raw images data close to reality

    are hard to obtain • GDPR? Not compliant… • Reusable for future projects • Feed the model training pipeline 16
  17. @Xebiconfr #Xebicon18 @bonbonking Build your own robot 17

  18. @Xebiconfr #Xebicon18 @bonbonking 18

  19. @Xebiconfr #Xebicon18 @bonbonking 19

  20. @Xebiconfr #Xebicon18 @bonbonking 20

  21. @Xebiconfr #Xebicon18 @bonbonking 21

  22. @Xebiconfr #Xebicon18 @bonbonking 22

  23. @Xebiconfr #Xebicon18 @bonbonking 23

  24. @Xebiconfr #Xebicon18 @bonbonking Pulse Width Modulation (PWM) is a common

    method used to apply a proportional control signal to an external device using a digital output pin. (e.g. servo motors, LCD display). It is a technique for getting analog results with digital means. • Frequency (expressed in Hz): describes how often the output pulse repeats. • Period is the time each cycle takes and is the inverse of frequency. • Duty cycle (expressed as a percentage) describes the width of the pulse within that frequency window. 24
  25. @Xebiconfr #Xebicon18 @bonbonking 25

  26. @Xebiconfr #Xebicon18 @bonbonking 26

  27. @Xebiconfr #Xebicon18 @bonbonking 27

  28. @Xebiconfr #Xebicon18 @bonbonking 28

  29. @Xebiconfr #Xebicon18 @bonbonking 29

  30. @Xebiconfr #Xebicon18 @bonbonking • Android Things Starter Kit (NXP i.MX7D):

    ~200€ • Or Raspberry Pi (~40€) + Camera Module (~25€) + Display • 16 Channel Servo Driver (~12€) • Power Supply or battery (~5€) • 5 servo motors (~12€) • Cardboard + woodsticks (~0€) • Glue Gun (~10€) • Electronique jumper + Resistors + LED + Button + Breadboard (~10€) 30
  31. @Xebiconfr #Xebicon18 @bonbonking Make your device smarter 31

  32. @Xebiconfr #Xebicon18 @bonbonking 32

  33. @Xebiconfr #Xebicon18 @bonbonking 33

  34. @Xebiconfr #Xebicon18 @bonbonking 34

  35. @Xebiconfr #Xebicon18 @bonbonking 35

  36. @Xebiconfr #Xebicon18 @bonbonking 36 Paper: 0.80 Scissors: 0.15 Rock: 0.05

  37. @Xebiconfr #Xebicon18 @bonbonking 37

  38. @Xebiconfr #Xebicon18 @bonbonking 38

  39. @Xebiconfr #Xebicon18 @bonbonking 39 Paper: 0.10 Scissors: 0.15 Rock: 0.75

  40. @Xebiconfr #Xebicon18 @bonbonking 40 Paper: 0.10 Scissors: 0.15 Rock: 0.75

    Prediction Error
  41. @Xebiconfr #Xebicon18 @bonbonking 41 Paper: 0.10 Scissors: 0.15 Rock: 0.75

    Prediction Error
  42. @Xebiconfr #Xebicon18 @bonbonking 42 Paper: 0.10 Scissors: 0.15 Rock: 0.75

    Prediction Error
  43. @Xebiconfr #Xebicon18 @bonbonking 43 Paper: 0.80 Scissors: 0.15 Rock: 0.05

  44. @Xebiconfr #Xebicon18 @bonbonking 44 Paper: 0.20 Scissors: 0.75 Rock: 0.05

  45. @Xebiconfr #Xebicon18 @bonbonking 45 Visualisation of Inception v3 Model Architecture

  46. @Xebiconfr #Xebicon18 @bonbonking 46 Keep all weights identical except these

    ones
  47. @Xebiconfr #Xebicon18 @bonbonking 47 Image Credit: https://xkcd.com/1987/

  48. @Xebiconfr #Xebicon18 @bonbonking 48 Image Credit: https://xkcd.com/1838/

  49. @Xebiconfr #Xebicon18 @bonbonking 49

  50. @Xebiconfr #Xebicon18 @bonbonking 50

  51. @Xebiconfr #Xebicon18 @bonbonking 51

  52. @Xebiconfr #Xebicon18 @bonbonking 52 python -m scripts.retrain \ --bottleneck_dir=tf_files/bottlenecks \

    --how_many_training_steps=4000 \ --model_dir=tf_files/models/ \ --summaries_dir=tf_files/training_summaries/mobilenet_0.50_224 \ --output_graph=tf_files/handgame_graph.pb \ --output_labels=tf_files/handgame_labels.txt \ --architecture=mobilenet_0.50_224 \ --image_dir=tf_files/gesture_photos
  53. @Xebiconfr #Xebicon18 @bonbonking 53 tflite_convert \ --graph_def_file=tf_files/handgame_graph.pb \ --output_file=tf_files/handgame_graph.lite \

    --input_format=TENSORFLOW_GRAPHDEF \ --output_format=TFLITE \ --input_shape=1,224,224,3 \ --input_array=input \ --output_array=final_result \ --inference_type=FLOAT \ --input_type=FLOAT
  54. @Xebiconfr #Xebicon18 @bonbonking 54

  55. @Xebiconfr #Xebicon18 @bonbonking Piece everything together 55

  56. @Xebiconfr #Xebicon18 @bonbonking 56

  57. @Xebiconfr #Xebicon18 @bonbonking 57

  58. @Xebiconfr #Xebicon18 @bonbonking 58 Android SDK Android Studio Play Service

    Firebase Google Cloud Platform
  59. @Xebiconfr #Xebicon18 @bonbonking 59

  60. @Xebiconfr #Xebicon18 @bonbonking 60

  61. @Xebiconfr #Xebicon18 @bonbonking

  62. @Xebiconfr #Xebicon18 @bonbonking 62

  63. @Xebiconfr #Xebicon18 @bonbonking The Inter-Integrated Circuit (IIC or I2C) bus

    connects simple peripheral devices with small data payloads. Sensors and actuators are common use cases for I2C. Examples include accelerometers, thermometers, LCD displays, and motor drivers. I2C is a synchronous serial interface, relies on a shared clock signal to synchronize data transfer between devices. I2C devices connect using a 3-Wire interface consisting of: • SCL: Shared clock signal • SDA: Shared data line • GND: Common ground reference 63
  64. @Xebiconfr #Xebicon18 @bonbonking 64

  65. @Xebiconfr #Xebicon18 @bonbonking

  66. @Xebiconfr #Xebicon18 @bonbonking 66

  67. @Xebiconfr #Xebicon18 @bonbonking Live Demo? My robot is still a

    bit silly... 67
  68. @Xebiconfr #Xebicon18 @bonbonking An entire weekend (of burning finger and

    slapped face)! My robot doesn't respect (yet) Asimov's Three Laws of Robotics 68
  69. @Xebiconfr #Xebicon18 @bonbonking Long Live Chifumi robot 69

  70. @Xebiconfr #Xebicon18 @bonbonking • Model training pipeline with user feedback

    & photo upload • Dynamic model downloading (via API or Firebase ML kit) • Better game control • Better UX • Leaderboard 70
  71. @Xebiconfr #Xebicon18 @bonbonking 71

  72. @Xebiconfr #Xebicon18 @bonbonking • Artificial neural network: https://en.wikipedia.org/wiki/Artificial_neural_network • Deep

    Learning: https://en.wikipedia.org/wiki/Deep_learning • Convolutional Neural Network: https://en.wikipedia.org/wiki/Convolutional_neural_network • TensorFlow for Poets: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/ • TensorFlow for Poets 2: Optimize for Mobile: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2/ • TensorFlow Glossary: https://www.tensorflow.org/versions/r0.12/resources/glossary • Medium article: Android meets Machine Learning: https://medium.com/xebia-france/android-meets-machine-learning-part-1-from-tensorflow-mobile-li te-to-ml-kit-4c7e6bc8eee3 72
  73. @Xebiconfr #Xebicon18 @bonbonking Merci ! Come play against chifumi robot

    at the stand :) Github Repo: https://github.com/jinqian/at-rock-paper-scissors-lizard-spock