The infrastructure of IoT

Presented at IoT Cochabamba

Transcript

1. ©2016 MOJIX INC. The Infrastructure of IoT using Docker
2. Mojix Cloud Operations at Mojix Working with microservices, cloud servers and scalability for distributed systems. Twitter, FB and more social: @yamilurbina Yamil Urbina
3. Mojix Confidential Turn beer into a talk
4. Mojix • Hardware: Servers • Software: Applications, Databases, Caches, Proxies • Network: DNS, VPNs, Firewalls Cloud Infrastructure
5. Mojix • Reliability • Monitoring • Security • Disaster recovery • Scalability Infrastructure
6. Mojix • We put applications in production • Monitoring and recovery planning • Application logs collection • 24/7 on call (no sleep, sleep is for the weak) Cloud Operations
7. ©2016 MOJIX INC. Introducing: The ViZix Challenge
8. Mojix • A sensor or device produces data • The data is transmitted to a service: • Modifies data based on filters • Triggers tasks or devices • Relays the data to another service • Saves the data to a database The IoT workflow
9. Mojix • Can produce data based on location, temperature or custom gathered information • More sensors, more data to send • Bandwidth is needed to transmit big chunks of data over the internet Sensors and devices
10. Mojix /v1/data/IoTCocha/item sn,799042 locationXYZ,101.7;40.4;3.0 location,-0.47663509682946;51.345 28591937 lastLocateTime,1479918916995 lastDetectTime,1479918968782 Data example
11. Mojix • 20.741.671 things in database • Over 280 antennas deployed in 3 sites • 2.5MB data per second • 656GB data transmitted per day • 5000+ messages per second • 728GB Big Data database size ViZix Challenge
12. Mojix • All components must be running 24/7 • How do we update to new versions? • Support 5000+ messages per second • Internet bandwidth to support 656GB / day • Monitoring health and logs ViZix Challenge
13. Mojix I'm quitting, I'm going to quit.
14. ©2016 MOJIX INC. This is BIG.
15. ©2016 MOJIX INC. Let’s implement
16. Mojix • Servers in the cloud • 100% customizable • 10GB+ internet speed • Firewall, secure and reliable Amazon Web Services
17. Mojix First attempt
18. Mojix • Can’t handle traffic / drops messages • Single point of failure • CPU, RAM and Network shared by all components • High disk writes occupies the HDD • Configurations all over the place First attempt: fail
19. Mojix • All components must be running 24/7 • How do we update to new versions? • Support 5000+ messages per second • Bandwidth to support 656GB / day • Monitoring health and logs ViZix Challenge
20. Mojix Confidential “There must be a better way.” Powhatan Powtitianna Chief
21. Mojix Applications are Changing
22. Mojix • Modular services • Easy to scale, destroy and update • Each microservice contains a component • Easy to connect within a network • Runs across many servers Microservices Architecture
23. Mojix Connected Microservices == Distributed Systems
24. ©2016 MOJIX INC. How do we distribute microservices?
25. Mojix • Automates deployment • Builds and packages components • Same configuration • Very portable Docker
26. Mojix 1. Compile and package your components 2. Include configuration and versions 3. Package it into a container 4. Same container goes through all stages: Development -> QA -> Operations Automated builds
27. Mojix 1. Get the new container 2. Stop and delete the old container 3. Run the new container Total Time: 5 seconds Easy updates
28. Mojix • All components must be running 24/7 • How do we update to new versions? • Support 5000+ messages per second • Internet bandwidth to support 656GB / day • Monitoring health and logs Second attempt
29. Mojix Second attempt
30. Mojix • Create a network between all servers • Use a service discovery app like Consul • Containers will find each other magically Distribute containers
31. Mojix • All components must be running 24/7 • How do we update to new versions? • Support 5000+ messages per second • Internet bandwidth to support 656GB / day • Monitoring health and logs Second attempt
32. Mojix • Logs are not files; they are streams • Find specific events in the past • Errors can happen anytime • Define alerts based on filters and text • We send logs to Papertrail Logs
33. Mojix
34. Mojix • Nothing is bulletproof; services can die • Better know beforehand than after your customers • Clients will understand if you let them know • Notify problems via SMS and email Health alerting
35. Mojix • Infrastructure never ends; it evolves constantly • Be prepared for disaster • If it works now, leave it and try new things • Don’t guarantee 100% uptime; errors happen Conclusions
36. ©2016 MOJIX INC. Questions?
37. ©2016 MOJIX INC. Thank you