‹#› @tpryan Terrence Ryan Developer Advocate IoT and Google Cloud Platform 

‹#› @tpryan Who are you? 

‹#› @tpryan 01 Introduction What are we trying to do 

• Products • Nest • Fitbit • WeMo • Phillips Hue • Lockitron • Building Blocks • Arduino • Raspberry Pi • BeagleBone • etc • etc • Providers • AWS • GCP • Azure • Building Blocks • Storage • Analytics • Processing 

• Building Blocks • Raspberry Pi • BeagleBone • Providers • GCP • Building Blocks • Cloud Storage • App Engine • Pub/Sub • BigQuery 

@tpryan Why did you choose these? 

@tpryan These are the things that made sense to use. 

@tpryan These are the things I could get working 

@tpryan 

@tpryan 

@tpryan 

‹#› @tpryan Beaglebone to App Engine to Cloud Storage 

BeagleBone App Engine Cloud Storage 

‹#› @tpryan App Engine • Platform as a Service • Give it code • It gives you a URL running that code • Advantages: • Scales up and down • Simple • Disadvantages • Language • PHP, Java, Python, Go • Restrictions • No writing to local disk • Other restrictions 

No content

No content

‹#› @tpryan Demo: App Engine 

‹#› @tpryan Cloud Storage • File Storage • Cheap and fast storage. • Easy to write • Easy to share • Can send to most other GCP Techs • Advantages: • Scales • Simple • Disadvantages • File based storage not best for analytics 

BeagleBone App Engine Cloud Storage 

‹#› @tpryan Source Code - App Engine - PHP use google\appengine\api\cloud_storage\CloudStorageTools; // Get app starting variables. $default_bucket = CloudStorageTools::getDefaultGoogleStorageBucketName(); $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; // Create data structure $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->request = $request; $instance_data->remote_address = $_SERVER['REMOTE_ADDR']; $instance_data->remote_host = $_SERVER['REMOTE_HOST']; $instance_data->latlon = $_SERVER['HTTP_X_APPENGINE_CITYLATLONG']; $instance_data->time = time(); $instance_data->data = $_REQUEST; $instance_json = json_encode($instance_data, JSON_PRETTY_PRINT); file_put_contents("gs://${default_bucket}/simple/${instance}/${request}.json", $instance_json); use google\appengine\api\cloud_storage\CloudStorageTools; // Get app starting variables. $default_bucket = CloudStorageTools::getDefaultGoogleStorageBucketName(); $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; // Create data structure $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->request = $request; $instance_data->remote_address = $_SERVER['REMOTE_ADDR']; $instance_data->remote_host = $_SERVER['REMOTE_HOST']; $instance_data->latlon = $_SERVER['HTTP_X_APPENGINE_CITYLATLONG']; $instance_data->time = time(); $instance_data->data = $_REQUEST; $instance_json = json_encode($instance_data, JSON_PRETTY_PRINT); file_put_contents("gs://${default_bucket}/simple/${instance}/${request}.json", $instance_json); 

‹#› @tpryan Source Code - Beaglebone - Python import time import Adafruit_BBIO.GPIO as GPIO import urllib2 # Note: Use P9_22(UART2_RXD) as GPIO. # Connect the Grove Button to UART Grove port of Beaglebone Green. Button = "P9_22" # GPIO P9_22 GPIO.setup(Button, GPIO.IN) base_url = "https://simple-dot-gcpiotdemo.appspot.com" params = "source=beagleboard&type=button&reading=1" if __name__== '__main__': while True: if GPIO.input(Button): print "Button is pressed." urllib2.urlopen(base_url + "/?" + params).read() time.sleep(.1) else: print "Button is unstuck." time.sleep(1) import time import Adafruit_BBIO.GPIO as GPIO import urllib2 # Note: Use P9_22(UART2_RXD) as GPIO. # Connect the Grove Button to UART Grove port of Beaglebone Green. Button = "P9_22" # GPIO P9_22 GPIO.setup(Button, GPIO.IN) base_url = "https://simple-dot-gcpiotdemo.appspot.com" params = "source=beagleboard&type=button&reading=1" if __name__== '__main__': while True: if GPIO.input(Button): print "Button is pressed." urllib2.urlopen(base_url + "/?" + params).read() time.sleep(.1) else: print "Button is unstuck." time.sleep(1) 

‹#› @tpryan Demo: Connecting from BeagleBoard to App Engine 

‹#› @tpryan Lessons Learned • Maybe go more mainstream kit • Raspberry Pi • Arduino • With more updated sensors 

‹#› @tpryan Raspberry Pi to Pub/Sub to App Engine to Cloud Storage 

App Engine Cloud Storage Raspberry Pi Pub/Sub 

‹#› @tpryan Pub/Sub • Messenger service • Publish and Subscribe Model • Advantages: • Unlimited Quota • Acts as a buffer • Disadvantages • Requires security on the device 

‹#› @tpryan Cloud PUB/SUB 

‹#› @tpryan 

‹#› @tpryan Source Code - Raspberry Pi - Python #!/usr/bin/env python import PCF8591 as ADC import RPi.GPIO as GPIO import time import math import urllib2 import os from google.cloud import pubsub INTERVAL = .1 DO = 17 GPIO.setmode(GPIO.BCM) os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/home/pi/gcp/creds.json" #!/usr/bin/env python import PCF8591 as ADC import RPi.GPIO as GPIO import time import math import urllib2 import os from google.cloud import pubsub INTERVAL = .1 DO = 17 GPIO.setmode(GPIO.BCM) os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/home/pi/gcp/creds.json" 

‹#› @tpryan 

‹#› @tpryan Source Code - creds.json { "type": "service_account", "project_id": "gcpiotdemo", "private_key_id": "a098b68643ae4ccda140d907d5a2394a895e8be4", "private_key": "-----BEGIN PRIVATE KEY——\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBA….Q..\n——END PRIVATE KEY-----\n", "client_email": "[email protected]", "client_id": "106452013022587438299", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://accounts.google.com/o/oauth2/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/raspberrypiclient%40gcpiotdemo.iam.gserviceaccount.com" } 

‹#› @tpryan Source Code - Raspberry Pi - Python def setup(): ADC.setup(0x48) GPIO.setup(DO, GPIO.IN) def loop(): status = 1 tmp = 1 while True: analogVal = ADC.read(0) Vr = 5 * float(analogVal) / 255 Rt = 10000 * Vr / (5 - Vr) temp = 1/(((math.log(Rt / 10000)) / 3950) + (1 / (273.15+25))) temp = temp - 273.15 print 'temperature = ', temp, 'C' publish("iotstream", get_serial() + "," +str(temp)) time.sleep(INTERVAL) if __name__ == '__main__': try: setup() loop() except KeyboardInterrupt: pass def setup(): ADC.setup(0x48) GPIO.setup(DO, GPIO.IN) def loop(): status = 1 tmp = 1 while True: analogVal = ADC.read(0) Vr = 5 * float(analogVal) / 255 Rt = 10000 * Vr / (5 - Vr) temp = 1/(((math.log(Rt / 10000)) / 3950) + (1 / (273.15+25))) temp = temp - 273.15 print 'temperature = ', temp, 'C' publish("iotstream", get_serial() + "," +str(temp)) time.sleep(INTERVAL) if __name__ == '__main__': try: setup() loop() except KeyboardInterrupt: pass 

‹#› @tpryan Source Code - Raspberry Pi - Python def publish(topic_name, data): pubsub_client = pubsub.Client("gcpiotdemo") topic = pubsub_client.topic(topic_name) message_id = topic.publish(data) print('Message {} published.'.format(message_id)) 

‹#› @tpryan Source Code - Raspberry Pi - Python def get_serial(): cpuserial = "0000000000000000" try: f = open('/proc/cpuinfo','r') for line in f: if line[0:6]=='Serial': cpuserial = line[10:26] f.close() except: cpuserial = "ERROR00000000000" return cpuserial 

‹#› @tpryan 

‹#› @tpryan Source Code - App Engine - PHP use google\appengine\api\cloud_storage\CloudStorageTools; // Get app starting variables. $default_bucket = CloudStorageTools::getDefaultGoogleStorageBucketName(); $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; $message = json_decode(file_get_contents('php://input')); $datastr = base64_decode(base64_decode($message->message->data)); $arr = explode(",", $datastr); // Create data structure $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->request = $request; $instance_data->remote_address = $_SERVER['REMOTE_ADDR']; $instance_data->time = time(); $instance_data->device = $arr[0]; $instance_data->temp = $arr[1]; $instance_json = json_encode($instance_data, JSON_PRETTY_PRINT); file_put_contents("gs://${default_bucket}/pubsub/${instance}/${request}.json", $instance_json); use google\appengine\api\cloud_storage\CloudStorageTools; // Get app starting variables. $default_bucket = CloudStorageTools::getDefaultGoogleStorageBucketName(); $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; $message = json_decode(file_get_contents('php://input')); $datastr = base64_decode(base64_decode($message->message->data)); $arr = explode(",", $datastr); // Create data structure $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->request = $request; $instance_data->remote_address = $_SERVER['REMOTE_ADDR']; $instance_data->time = time(); $instance_data->device = $arr[0]; $instance_data->temp = $arr[1]; $instance_json = json_encode($instance_data, JSON_PRETTY_PRINT); file_put_contents("gs://${default_bucket}/pubsub/${instance}/${request}.json", $instance_json); 

‹#› @tpryan Demo: Connecting from RaspberryPi 

‹#› @tpryan Lessons Learned • Securing IoT devices is not trivial 

‹#› @tpryan Data analysis with BigQuery 

@tpryan 10 x a second * 60 * 60 * 24 864,000 data points a day * 400 sensors 345,600,000 data points a day * 365 days a year 126,144,000,000 data points a year 

@tpryan IoT = BigData 

‹#› @tpryan Big Query • Scan Terabytes in seconds • Use SQLish Queries • REST, Web UI, ODBC 

‹#› @tpryan Demo: BigQuery 

@tpryan Count Shakespeare SELECT count(word) FROM publicdata:samples.shakespeare 

@tpryan Count to a Million SELECT sum(requests) as total FROM [fh-bigquery:wikipedia.pagecounts_20151109_18] 

@tpryan Count to a Billion SELECT sum(requests) as total FROM [fh-bigquery:wikipedia.pagecounts_201505] 

@tpryan Count to a Trillion SELECT SUM(requests) AS total FROM TABLE_QUERY( [fh-bigquery:wikipedia], 'REGEXP_MATCH( table_id, r"pagecounts_201[3-4][0-9]{2}$")') 

@tpryan Run a RegEx on a Hundreds of Billions SELECT SUM(requests) AS total FROM TABLE_QUERY( [fh-bigquery:wikipedia], 'REGEXP_MATCH( table_id, r"pagecounts_201[3-4][0-9]{2}$")') WHERE (REGEXP_MATCH(title, '.*[dD]inosaur.*')) 

‹#› @tpryan Demo: Getting our data into BigQuery 

‹#› @tpryan 06 Conclusion What now 

App Engine Cloud Storage Raspberry Pi Pub/Sub Big Query 

Raspberry Pi 

No content

Raspberry Pi 

@tpryan ‹#› Google Cloud Platform Compute Connectivity Big Data Storage Developer Tools Management Machine Learning 

‹#› @tpryan App Engine • Flexible runtime • Based on Docker containers • Very few restrictions • Base images can use GCP resources easily • Custom can still use them with more setup • Scales slower • Scales to 1 not zero 

‹#› @tpryan Data Flow 

‹#› @tpryan Cloud Vision 

‹#› @tpryan Demo: Cloud Vision API 

‹#› @tpryan Cloud ML 

‹#› @tpryan Stackdriver 

‹#› @tpryan Lessons Learned • IoT = Big Data • Once you get it to a cloud provider your only limit is creativity • Security doesn’t have a clear path… yet 

‹#› @tpryan Lessons Implied • Maybe reduce your sample size • Maybe skip some steps • Maybe batch up data • Maybe do some processing on the device 

‹#› @tpryan Next Steps • https://cloud.google.com/solutions/iot/ • https://cloud.google.com/solutions/iot-overview • https://googlecloudplatform.github.io/ 

‹#› @tpryan Thank You terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-iot