An IOT gateway Architecture using MQTT & DTN

BIT 5th Annual World Congress - Emerging InfoTech 2016 - Quingdao, China
Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks

A proposal for converging earth and space IOT research

Transcript

1. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks BIT 5th Annual World Congress Emerging InfoTech 2016 An IOT gateway Architecture using a MQTT bundle transport & LTP Convergence Layer 11/11/16 1 By Alex Kritikos
2. . Senior director R&D Universal Messaging Software AG UK . OASIS MQTT TC member Email: alex.kritikos@softwareag.com Twitter: @krital LinkedIn: https://uk.linkedin.com/in/alexkritikos ALEX KRITIKOS About the speaker 2 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
3. Growing fast Internet of (Earth) Things 3 11/11/16 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
4. DTN based Solar System Internet 4 11/11/16 Internet of (Space) Things Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
5. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks Earth & Space IOT research What if we combined them? • Link layer agnostic propagation (bundles) • MQTT encapsulation: payloads to local subscribers 5 11/11/16 • DTN encapsulation: payloads are routed progressively via intermediary nodes • Full MQTT & DTN stack on each node • Lower latency and higher throughput
6. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks DTN to MQTT Convergence Layer • /temperature => dtn://node/temperature • /consumption => dtn://node/consumption • /ac/actions => dtn://node/ac/actions • MQTT topic => DTN EID 6 11/11/16 • bi directional • MQTT publish DTN receive • DTN send MQTT subscribe
7. 7 11/11/16 • MQTT integration with DTN • Hardware sharing = cost reduction • Software sharing = cost reduction • MQTT bridging • via DTN BP • Via Licklider Transmission Protocol (LTP) DTN to MQTT Value Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
8. MQTT PUBLISH BP TRAFFIC MQTT SUBSCRIBE 8 11/11/16 DTN to MQTT integration Hardware and Software Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
9. MQTT ROUTING via DTN Best effort MQTT Bridging via bundles 9 11/11/16 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
10. 10 11/11/16 MQTT ROUTING via DTN Assured MQTT Bridging via custody transfer bundles Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
11. • Schedules best effort or assured data propagation • Decouples local fanout from bridging propagation 11 11/11/16 • Events are routed progressively via intermediary nodes • Results in lower latency and higher throughput MQTT ROUTING via DTN Thoughts Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
12. 12 11/11/16 MQTT ROUTING via LTP Bridging MQTT brokers via LTP UDPCL Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
13. • Unidirectional, Connectionless, Efficient flow • Fragmentation support • Flow control via LTP sessions • Multi stream (red, green) assured data propagation • Link layer agnostic 13 11/11/16 MQTT ROUTING via LTP Thoughts Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
14. Interested? Contact me • To find out more • To collaborate 14 11/11/16 Earth & Space IOT research What if we combined them? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
15. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks References • Advances in Delay-tolerant Networks (DTNs) – Architecture and Enhanced Performance By Woodhead Publishing 2015 11/11/16 15 • Brenda Lyons, NASA NDBS 2016 • Appendix A: What is IOT? • Appendix B: What is DTN? • Appendix C: What is MQTT? • Further Research A: DTN routing via MQTT • Further Research B: High availability with MQTT • Further Research C: High availability with MQTT & DTN
16. Appendix A Never heard of Internet of Things? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 16 11/11/16 • Read about • History and status • M2M Comparison • Market & estimates • Example architectures
17. A IOT & M2M Comparison… • M2M came first during the telecom era • M2M: proprietary, closed, private machine communication (e.g. CallerID) • IOT is much more than machine communication Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 17 11/11/16
18. A IOT & M2M Comparison…(2) • IOT: devices connect to a larger network, usually via a gateway, are cloud enabled, can interop, enables device physical reach (e.g. Samsung SmartThings) • IOT – M2M common: machine comms Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 18 11/11/16
19. IOT Market Estimates… • 2020: • 25-30 billion Connected devices • Today: • 6.4-9 billion Connected devices !This excludes phones, tablets, PCs! Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 19 11/11/16
20. IOT IMPORTANCE Status… • Ongoing transport standardization efforts • MQTT, COAP, AMQP, Websocket, Node • Enormous telco traffic challenge • Global M2M traffic:1 to 6.3 Exabytes / month by 2020 SOURCE • Real time streaming analytics • Security nightmare • 1TB DDOS by hacked CCTVs • PCs, PVRs, routers, smart fridge • MCUs + PKI = slow • Privacy concerns Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 20 11/11/16
21. IOT Example Smart Home… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 21 11/11/16
22. IOT Example Smart Home… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 22 11/11/16 Why cant all things use WiFi? • If mains powered, sometimes they do • WiFi may not reach your boiler room • A lot of battery powered – low energy sensors • WiFi lacks ad-hoc network features (e.g. mesh) Things tend to be in their own sensor/field net • Based on low power RF (e.g. zigbee, zwave) • Needs a gateway to bridge to local IP network
23. Smart Home IOT Architecture GSM/DSL/ FIBRE HTTP MQTT AMQP Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 23 11/11/16 Users interact with things via a device …
24. Smart Home IOT Architecture (2) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 24 11/11/16 Things interact with environment via gateway…
25. Smart Home IOT Architecture (3) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Cloud Platform Services Cloud services add value & remote access
26. Smart Home IOT Architecture (4) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 26 11/11/16
27. Appendix B Never heard of DTN? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 27 11/11/16 • Read about • history and status • Protocol overview • TCP/IP Comparison • Terrestrial uses
28. DTN History … Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 28 11/11/16 initiative to develop an “Interplanetary Internet” 1998 NASA starts Delay / Disruption Tolerant Networking 1998 NASA leads worldwide DTN development 2005 IOAG Recommends Solar System Internetwork (SSI) 2010 IOAG Suggests DTN a core service of SSI 2010 DTN is being standardized by IETF DTN is being standardized by CCSDS
29. DTN Solar System Internet… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 29 11/11/16
30. DTN • The (DTN) protocol suite • Extends the terrestrial Internet capabilities • Works • in challenged communication environments • over unidirectional/asymmetric links • Tolerates • frequent disruptions • long delays / high error rates • Provides • assured delivery of data • automatic store-and-forward • Can run over IP or as a full Internetworking protocol Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 30 11/11/16 Overview
31. DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 31 11/11/16 Overview (2)
32. DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 32 11/11/16 TCP/IP Comparison
33. DTN IP: Must wait for complete path Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 33 11/11/16 TCP/IP Comparison (2)
34. DTN TCP/IP Comparison (3) DTN: Incremental progress without complete path Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 34 11/11/16
35. DTN DTN decouples data propagation from link layer characteristics • Can be used to propagate data not needed in real time (e.g. wearables, home automation etc) • Introduces some node storage requirements • Use cases • sea-going vessels • outdoor explorers • wildlife research • vehicular networks • disaster area networks Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 35 11/11/16 Terrestrial use
36. Appendix C Never heard of MQTT? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 36 11/11/16 • Read about • History and status • Protocol overview • Role in IOT architectures
37. MQTT History… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 37 11/11/16 IBM & Eurotech develop 1990s IBM & Eurotech Open Source 2012 Software AG implements 2013 OASIS MQTT TC Created 2013 Many others join MQTT becomes OASIS standard 2014 MQTT becomes ISO/IEC 20922 2016
38. MQTT Description • MQTT is • Is an open, simple, lightweight publish- subscribe binary messaging protocol • Is easy to implement (15 protocol commands) • has a fixed 2 byte over-the-wire overhead header • Has 256MB max payload • runs on top of TCP/TLS/WS/WSS stacks Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 38 11/11/16
39. MQTT Description (2) • MQTT offers 3 quality of service levels: • QOS 0: At most once (Fire and forget) • QOS 1: At least once (ACKed delivery) • QOS 2: Exactly once (ACKed / No duplicates) Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 39 11/11/16
40. MQTT Description (3) • Information on software/hardware that uses MQTT: http://ow.ly/wHm2e • The Eclipse Paho project offers some great MQTT clients in minimal C, regular C, C++, Java, Javascript, Python, Lua, Go, Objective- C: http://ow.ly/wHl8c Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 40 11/11/16
41. MQTT in IOT Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 41 11/11/16
42. MQTT in IOT (2) Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 42 11/11/16
43. MQTT in IOT (3) MQTT BRIDGE MQTT PUB & SUB MQTT SUB Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 43 11/11/16
44. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 44 11/11/16 Further Research A DTN routing via MQTT
45. DTN ROUTING via MQTT One to many Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 45 11/11/16
46. DTN ROUTING via MQTT Many to one Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 46 11/11/16
47. DTN ROUTING via MQTT Thoughts • Unicast routing focus • Data sharing importance: • Vehicular networks • Disaster area networks • Delegates shared custodian to MQTT broker • Can use various MQTT QOS Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 47 11/11/16
48. Further Research B High availability with MQTT Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 48 11/11/16
49. IOT with MQTT HA architecture MQTT PUB & BRIDGE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 49 11/11/16
50. IOT with MQTT HA active – active cluster architecture MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 50 11/11/16
51. IOT with MQTT HA active – active: master failure MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 51 11/11/16
52. IOT with MQTT HA active – active: publisher failover MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 52 11/11/16
53. IOT with MQTT HA active – active: node rejoin MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 53 11/11/16
54. IOT with MQTT HA active – active: multiple streams MQTT PUB STREAM 1 MQTT PUB STREAM 2 MQTT PUB STREAM 3 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 54 11/11/16
55. Further Research C High availability with MQTT & DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 55 11/11/16
56. MQTT HA via DTN Eventual consistency … Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 56 11/11/16
57. MQTT HA via DTN Master failure Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 57 11/11/16
58. MQTT HA via DTN Partial sync while offline Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 58 11/11/16
59. MQTT HA via DTN Fast node cluster rejoin Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 59 11/11/16
60. MQTT HA via DTN Thoughts • Schedules best effort or assured data propagation • Decouples local fanout from slave propagation • Bundles continue to flow while broker offline • Allows fast cluster recovery Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 60 11/11/16