AWS IoT Part 1

Overview of AWS IoT services and a demo to send sensor data to IoT Core.

Transcript

1. AWS MEETUP GROUP INTRO TO AWS IoT SERVICES AWS Meetup Group August 5 - Kevin Tinn
2. • Cloud Application Architect, Practice Lead at World Wide Technology • ~15 years of industry experience • Software Development • .NET (C# and VB.NET) • JVM (Scala) • JavaScript • Data Streaming Architectures • Kafka, Kinesis, Event Hubs • Application Architecture • AWS, Azure, GCP, and on-prem solutions • Incessant traveler with a new-found skiing addiction • I Live in Denver, by way of St Louis, and grew up in TN INTRO: KEVIN TINN 2
3. • Big thanks to Matt Gowie for the ECS 101 talk last Meetup! https://www.meetup.com/AWSMeetupGroup/events/270751202/ • Overview of IoT Services - 6:00 - 10 minutes • IoT Core • Greengrass • IoT Analytics • Raspberry Pi Setup - 10 minutes • Demo: Communicating Directly with IoT Core – 15 minutes • Connecting to cloud-based Data Pipeline – 10 minutes • Configure Greengrass – 15 minutes • Q&A AGENDA 3
4. • Source code for AWS infrastructure and the kinesis producer are available at https://github.com/kevasync/aws-meetup-group-data-services • This link is available in the comments of the Meetup deets https://www.meetup.com/AWSMeetupGroup/events/269768602/ Please join the Meetup if you haven’t already REPO INFO 4
5. • AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other device • AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely • With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected. COMPONENTS: IoT CORE 5
6. • AWS IoT Greengrass seamlessly extends AWS to edge devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. • With AWS IoT Greengrass, connected devices can run AWS Lambda functions, Docker containers, or both, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely – even when not connected to the Internet. • Over the air updates! COMPONENTS: GREENGRASS 6
7. • AWS IoT Analytics is a fully-managed service that makes it easy to run and operationalize sophisticated analytics on massive volumes of IoT data without having to worry about the cost and complexity typically required to build an IoT analytics platform. • Data transformation and enrichment services that are optimized for IoT • Ad hoc query capabilities • Stored in time series DB • Automated scaling COMPONENTS: IoT ANALYTICS 7
8. • Format SD card I used SD Memory Card Formatter on my Mac • Extract OS • Use diskutil to get disk path using diskutil list: RASPBERRY PI SETUP 9
9. • Extract OS • Mount disk using sudo mkdir /mnt && sudo mount -t msdos /dev/disk2 /mnt • Download OS: • Noobs • Pi OS (Raspbian) • I initially tried Pi OS, but for some reason openssh wasn’t working so I tried a fresh OS. The Noobs worked out so I never looked back. • For Noobs I just decompressed the zip file and copied the extracted files to the SD Card in finder • There is also a disk imager utility that can be used, follow instructions from links here to do so: https://www.raspberrypi.org/downloads/ • Complete Pi setup and enable ssh • On the Pi: sudo raspi-config • Navigate to Interfacing Options and select P2 SSH • Restart the Pi: sudo restart • Get the hostname: hostname -I • Do us all a solid: sudo apt install vim -y • Change the hostnames on the pi: • sudo vim /etc/hostname -> change raspberrypi to sensor-pi • sudo vim /etc/hosts -> update to 127.0.1.1 sensor-pi • sudo reboot RASPBERRY PI SETUP 10
10. • Create policy: • IaC: Pulumi (Available in repo) • or… Navigate to AWS IoT > Secure > Policies > Create: • Update the ARN to have the topic name we will use for sensor data • Name your policy, set Effect to allow and Action to iot:* DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 11
11. • Create IoT Core instance, create a IoT device, certificates, IAM policy, and attach policy to cert • IaC: Pulumi (Available in repo) • Navigate to IoT Core > Manage > Things: • Create > Single Thing: • Name it and give it a type (You may have to create the type): DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 12
12. • Create a certificate (This can be done ahead of time): • Don’t forget to activate the cert (I made this mistake.. /facepalm) • Download the root CA file, the device pem file, and the private key file • curl https://www.amazontrust.com/repository/AmazonRootCA1.pem > AmazonRootCA1.pem • Use the links to download the thing pem and the private key, put them in same dir as the CA file • If using Pulumi, stack output command can be used to get device cert and private key: • pulumi stack output IoTPrivateKey > private.pem • pulumi stack output IoTDevicePem > device.pem.crt DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 13
13. • Associate the certificate with the policy we created: • Get your Thing Shadow form IoT Core > Manage > Things > sensor-pi > Interact: DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 14
14. • If you manually downloaded cert files, rename them to device.pem.crt and private.pem. Move these and the root ca into same directory and change directories into that directory. • Copy the certs to the sensor-pi using scp (Secure Copy): scp AmazonRootCA1.pem private.pem device.pem.crt pi@10.0.0.2:~ • ssh into the sensor-pi: ssh pi@10.0.0.2 DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 15
15. Verify openssl is available within python: python import ssl ssl.OPENSSL_VERSION quit() Get producer source code: git clone https://github.com/kevasync/aws-meetup-group- data-services.git cd aws-meetup-group-data-services/pi DEMO: COMMUNICATING WITH IoT CORE DIRECTLY 16
16. • MQTT: (MQ Telemetry Transport or Message Queuing Telemetry Transport): is an open OASIS and ISO standard (ISO/IEC 20922) lightweight, publish-subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited. • Setup AWS IoT Py SDK: mkdir sdk cd sdk curl -o aws-iot-sdk.zip https://s3.amazonaws.com/aws-iot-device-sdk- python/aws-iot-device-sdk-python-latest.zip unzip aws-iot-sdk.zip sudo python setup.py install cd .. cp –rf ./sdk/AWSIoTPythonSDK • Run the producer with your interaction url: python pi-sensor-producer.py <identifier>-ats.iot.<region>.amazonaws.com DEMO: GET PY MQTT LIB AND RUN PRODUCER 17
17. • What we’re working with: DHT22 digital humidity and temperature sensor • Pins: • 1: VCC (Power) • 2: DATA • 3: NULL • 4: GRN (Ground) • Traditional setup: • Place a 10k resistor between Pin 1 and Pin 2 of the DHT22 • Wire Pin 1 of the DHT22 to Physical Pin 1 (3v3) on the Pi • Wire Pin 2 of the DHT22 to Physical Pin 7 (GPIO4) on the Pi • Wire Pin 4 of the DHT22 to Physical Pin 6 (GND) on the Pi • Our device has the resistor on-board, so we can go directly to the board: • Pin 1 -> Pi pin 1 • Pin 2 -> Pi pin 7 # todo: why is this coded in as pin 4? • Pin 3 -> Pi pin 6 • Connect to sensor-pi and prepare it to get sensors readings in our pi-producer code ssh pi@10.0.0.2 cd aws-meetup-group-data-services/pi sudo apt update sudo apt upgrade #takes a while sudo pip3 install Adafruit_Python_DHT # todo: used pip3, test the instructions cp pi-sensor-producer.py pi-sensor-producer-dht22.py vim pi-sensor-producer-dht22.py DEMO: CONNECTING SENSOR TO PI 18
18. • In the code add an import and reference to the sensor and the data pin on the pi: import Adafruit_DHT sensor = Adafruit_DHT.DHT22 piDataPin = 4 • In the loop that produces to IoT Core, get the temp and humidity from the sensor and add the temp reading in the message payload: while loopCount < 10: humidity, temperature = Adafruit_DHT.read_retry(sensor, piDataPin) message = {} message['SITE_ID'] = clientId message['SENSOR_TYPE'] = 'TEMPERATURE' message['SENSOR_READING_VALUE'] = temperature • Run the producer, see temps (in Celsius): python pi-sensor-producer-dht22.py <id>-ats.iot.<region>.amazonaws.com DEMO: CONNECTING SENSOR TO PI 19
19. • Output form pi producer: • View data in AWS IoT: Core > Test > Subscribe to a topic > sensor-topic (You much run producer again as well): DEMO: VIEW DATA FROM PRODUCER 20
20. • Let’s connect this sensor data to the data pipeline we created a few Meetups ago (Links: event, repo, slides) DEMO: CONNECTING TO EXISTING DATA PIPELINE 21
21. • Create IoT Core rule with action to send data to Kinesis: DEMO: CONNECTING TO EXISTING DATA PIPELINE 22
22. • IoT Core MQTT topic rule sends data to Kinesis • Setup up Kinesis Data Stream action and create policy: DEMO: CONNECTING TO EXISTING DATA PIPELINE 23
23. • Upload weather.csv and start temperature analytics app. Enriched data from the sensor-pi is enriched with the site’s outdoor temperature. DEMO: CONNECTING TO EXISTING DATA PIPELINE 24
24. • IoT rules offer a integration with a ton of AWS services: • Augment or filter data received from a device. • Write data received from a device to an Amazon DynamoDB database. • Save a file to Amazon S3. • Send a push notification to all users using Amazon SNS. • Publish data to an Amazon SQS queue. • Invoke a Lambda function to extract data. • Process messages from a large number of devices using Amazon Kinesis. • Send data to the Amazon Elasticsearch Service. • Capture a CloudWatch metric. • Change a CloudWatch alarm. • Send the data from an MQTT message to Amazon Machine Learning to make predictions based on an Amazon ML model. • Send a message to a Salesforce IoT Input Stream. • Send message data to an AWS IoT Analytics channel. • Start execution of a Step Functions state machine. • Send message data to an AWS IoT Events input. • Send message data an asset property in AWS IoT SiteWise. • Send message data to a web application or service. DEMO: CONNECTING TO EXISTING DATA PIPELINE 25
25. Margaret Hamilton – rocket (computer) scientist DEMO: LOOK AT IaC FOR IoT SERVICES IN DEMO 26 To The Code!
26. IoT CORE/PI/SENSOR DEMO Q&A? 27
27. GREENGRASS USE CASE 28
28. • Create Greengrass group and save credentials bundle: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg- config.html CONFIGURE GREENGRASS 29
29. • Create Greengrass group and save credentials bundle: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html • Copy the security bundle and root CA from earlier to the greengrass pi, ssh in and extract files: scp ~/Downloads/3664d60716-setup.tar.gz pi@10.0.0.137:~ scp ~/src/aws-meetup-group/iot/certs/AmazonRootCA1.pem pi@10.0.0.137:~ ssh pi@10.0.0.137 tar -zxvf 3664d60716-setup.tar.gz CONFIGURE GREENGRASS 30
30. • Create users: sudo adduser --system ggc_user sudo addgroup --system ggc_group • Create greengrass dir and configure certs: sudo mkdir -p /greengrass cd /greengrass sudo cp -rf ~/certs/ ./ sudo cp -rf ~/config/ ./ sudo cp ~/AmazonRootCA1.pem ./certs/root.ca.pem • Install Greengrass dependencies sudo modprobe configs sudo apt install sqlite3 -y sudo apt install node -y sudo apt install openjdk-8-jdk openjdk-8-jre –y alias java8="/usr/lib/jvm/java-8-openjdk-armhf/bin/java" CONFIGURE GREENGRASS 31
31. • Create greengrass dir and configure certs: sudo mkdir greengrass-dependency-checker-GGCv1.10.x cd greengrass-dependency-checker-GGCv1.10.x sudo wget https://github.com/aws-samples/aws-greengrass- sudo samples/raw/master/greengrass-dependency-checker- GGCv1.10.x.zip sudo unzip greengrass-dependency-checker-GGCv1.10.x.zip cd greengrass-dependency-checker-GGCv1.10.x sudo ./check_ggc_dependencies | more CONFIGURE GREENGRASS 32
32. • Check dependencies: sudo mkdir greengrass-dependency-checker-GGCv1.10.x cd greengrass-dependency-checker-GGCv1.10.x sudo wget https://github.com/aws-samples/aws-greengrass-sudo samples/raw/master/greengrass-dependency-checker-GGCv1.10.x.zip sudo unzip greengrass-dependency-checker-GGCv1.10.x.zip cd greengrass-dependency-checker-GGCv1.10.x sudo ./check_ggc_dependencies | more • Download greengrass and install: wget -O aws-iot-greengrass-keyring.deb https://d1onfpft10uf5o.cloudfront.net/greengrass-apt/downloads/aws-iot- greengrass-keyring.deb sudo dpkg -i aws-iot-greengrass-keyring.deb echo "deb https://dnw9lb6lzp2d8.cloudfront.net stable main" | sudo tee /etc/apt/sources.list.d/greengrass.list sudo apt update sudo apt install aws-iot-greengrass-core -y sudo systemctl start greengrass.service sudo systemctl status greengrass.service CONFIGURE GREENGRASS 33
33. • Complete Greengrass use case • Bi-directional communication • Device provisioning • Security • Edge-based ML • IoT Greengrass Stream Manager connection to Kinesis or IoT Analytics NEXT TIME… PART 2: GG, IoT ANALYTICS, AND AWS IoT ADVANCED TOPICS 34
34. • In this session, we have covered: • Overview of AWS IoT Services • Demo of setting up IoT Core and Raspberry Pi w/ sensor • Connecting IoT topic to data pipeline via IoT rule • Thank you for viewing! • Please talk to me if you have further questions • References: • Raspberry pi to IoT core: https://www.mitrai.com/tech-guide/connecting-your-raspberry-pi-with-aws-internet-of- things-iot-service/ • Setting up Greengrass on Raspberry Pi: https://docs.aws.amazon.com/greengrass/latest/developerguide/setup- filter.rpi.html • https://docs.aws.amazon.com/greengrass/latest/developerguide/install-ggc.html#ggc-package-manager • Lambdas for gg: https://docs.aws.amazon.com/greengrass/latest/developerguide/create-lambda.html • DHT 22 hookup: https://pimylifeup.com/raspberry-pi-humidity-sensor-dht22/ CONCLUSION 35
35. • AWS data lakes using Glue and Athena • First Wednesday of September – Austin Loveless • Glue • Athena • AWS Data ETL patterns • IoT Part 2 • First Wednesday of October – Kevin Tinn • AWS IoT Greengrass full implementation • IoT Analytics • Advanced IoT capability overview UPCOMING MEETUPS 36
36. WWT does a ton of this kind of work on all major platforms Reach out on LinkedIn: www.linkedin.com/in/kevin-tinn 37