Play Rock-Paper-Scissors-Spock-Lizard with your Android Things

@Xebiconfr #Xebicon18 @bonbonking Jouer à Rock-Paper-Scissors-Spock-Lizard avec son Android Things
Qian Jin | Technical Officer | [email protected] Xebia France 1

@Xebiconfr #Xebicon18 @bonbonking Qian Jin Google Developer Expert IoT Technical
Officer @ Xebia France @bonbonking 2

@Xebiconfr #Xebicon18 @bonbonking Play Rock-Paper-Scissors-Spock-Lizard "Wacky, Whimsical Machinery with cardboard
and duct tape" 3

@Xebiconfr #Xebicon18 @bonbonking 4

@Xebiconfr #Xebicon18 @bonbonking Motivation behind the shoebox project 5

@Xebiconfr #Xebicon18 @bonbonking 6 Image Source: https://www.flaticon.com/packs/internet-of-things-45 Cloud Data Centers
thousands Fog Nodes millions Edge Devices billions

@Xebiconfr #Xebicon18 @bonbonking 7 • Latency • Availability • Bandwidth
• Security/Privacy

@Xebiconfr #Xebicon18 @bonbonking Explained in xkcd way 8

@Xebiconfr #Xebicon18 @bonbonking • Data collection tool (raw data to
train the ML model) • Hardware components (robot arms + game control) • Application running on Android Things ◦ Applicative: game rules, play logic and interactions ◦ Artificial Intelligence: recognize opponent's gesture 10

@Xebiconfr #Xebicon18 @bonbonking Raw Image Crowdsourcing 11

@Xebiconfr #Xebicon18 @bonbonking • Web mobile application: Node.js + Express
• Hosting/Deployment: Firebase Hosting / Function / Storage 12

@Xebiconfr #Xebicon18 @bonbonking 15 Thank you my fellow Xebians <3

@Xebiconfr #Xebicon18 @bonbonking • Raw images data close to reality
are hard to obtain • GDPR? Not compliant… • Reusable for future projects • Feed the model training pipeline 16

@Xebiconfr #Xebicon18 @bonbonking Build your own robot 17

@Xebiconfr #Xebicon18 @bonbonking Pulse Width Modulation (PWM) is a common
method used to apply a proportional control signal to an external device using a digital output pin. (e.g. servo motors, LCD display). It is a technique for getting analog results with digital means. • Frequency (expressed in Hz): describes how often the output pulse repeats. • Period is the time each cycle takes and is the inverse of frequency. • Duty cycle (expressed as a percentage) describes the width of the pulse within that frequency window. 24

@Xebiconfr #Xebicon18 @bonbonking • Android Things Starter Kit (NXP i.MX7D):
~200€ • Or Raspberry Pi (~40€) + Camera Module (~25€) + Display • 16 Channel Servo Driver (~12€) • Power Supply or battery (~5€) • 5 servo motors (~12€) • Cardboard + woodsticks (~0€) • Glue Gun (~10€) • Electronique jumper + Resistors + LED + Button + Breadboard (~10€) 30

@Xebiconfr #Xebicon18 @bonbonking Make your device smarter 31

@Xebiconfr #Xebicon18 @bonbonking 36 Paper: 0.80 Scissors: 0.15 Rock: 0.05

Prediction Error

@Xebiconfr #Xebicon18 @bonbonking 45 Visualisation of Inception v3 Model Architecture

@Xebiconfr #Xebicon18 @bonbonking 46 Keep all weights identical except these
ones

@Xebiconfr #Xebicon18 @bonbonking 47 Image Credit: https://xkcd.com/1987/

@Xebiconfr #Xebicon18 @bonbonking 48 Image Credit: https://xkcd.com/1838/

@Xebiconfr #Xebicon18 @bonbonking 52 python -m scripts.retrain \ --bottleneck_dir=tf_files/bottlenecks \
--how_many_training_steps=4000 \ --model_dir=tf_files/models/ \ --summaries_dir=tf_files/training_summaries/mobilenet_0.50_224 \ --output_graph=tf_files/handgame_graph.pb \ --output_labels=tf_files/handgame_labels.txt \ --architecture=mobilenet_0.50_224 \ --image_dir=tf_files/gesture_photos

@Xebiconfr #Xebicon18 @bonbonking 53 tflite_convert \ --graph_def_file=tf_files/handgame_graph.pb \ --output_file=tf_files/handgame_graph.lite \
--input_format=TENSORFLOW_GRAPHDEF \ --output_format=TFLITE \ --input_shape=1,224,224,3 \ --input_array=input \ --output_array=final_result \ --inference_type=FLOAT \ --input_type=FLOAT

@Xebiconfr #Xebicon18 @bonbonking Piece everything together 55

@Xebiconfr #Xebicon18 @bonbonking 58 Android SDK Android Studio Play Service
Firebase Google Cloud Platform

@Xebiconfr #Xebicon18 @bonbonking

@Xebiconfr #Xebicon18 @bonbonking The Inter-Integrated Circuit (IIC or I2C) bus
connects simple peripheral devices with small data payloads. Sensors and actuators are common use cases for I2C. Examples include accelerometers, thermometers, LCD displays, and motor drivers. I2C is a synchronous serial interface, relies on a shared clock signal to synchronize data transfer between devices. I2C devices connect using a 3-Wire interface consisting of: • SCL: Shared clock signal • SDA: Shared data line • GND: Common ground reference 63

@Xebiconfr #Xebicon18 @bonbonking

@Xebiconfr #Xebicon18 @bonbonking Live Demo? My robot is still a
bit silly... 67

@Xebiconfr #Xebicon18 @bonbonking An entire weekend (of burning finger and
slapped face)! My robot doesn't respect (yet) Asimov's Three Laws of Robotics 68

@Xebiconfr #Xebicon18 @bonbonking Long Live Chifumi robot 69

@Xebiconfr #Xebicon18 @bonbonking • Model training pipeline with user feedback
& photo upload • Dynamic model downloading (via API or Firebase ML kit) • Better game control • Better UX • Leaderboard 70

@Xebiconfr #Xebicon18 @bonbonking • Artificial neural network: https://en.wikipedia.org/wiki/Artificial_neural_network • Deep
Learning: https://en.wikipedia.org/wiki/Deep_learning • Convolutional Neural Network: https://en.wikipedia.org/wiki/Convolutional_neural_network • TensorFlow for Poets: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/ • TensorFlow for Poets 2: Optimize for Mobile: https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2/ • TensorFlow Glossary: https://www.tensorflow.org/versions/r0.12/resources/glossary • Medium article: Android meets Machine Learning: https://medium.com/xebia-france/android-meets-machine-learning-part-1-from-tensorflow-mobile-li te-to-ml-kit-4c7e6bc8eee3 72

@Xebiconfr #Xebicon18 @bonbonking Merci ! Come play against chifumi robot
at the stand :) Github Repo: https://github.com/jinqian/at-rock-paper-scissors-lizard-spock

Play Rock-Paper-Scissors-Spock-Lizard with your...

Play Rock-Paper-Scissors-Spock-Lizard with your Android Things

More Decks by jinqian

Other Decks in Technology

Featured

Transcript