MQTT for Sysadmins

Transcript

1. MQTT
   for sysadmins and for fun
   Jan-Piet Mens
   April 2014
   @jpmens
   

   View Slide

2. @jpmens: consultant,
   author, architect, part-time
   admin, small-scale fiddler,
   loves LDAP, DNS,
   plain text, and things
   that work.
   

   View Slide

3. MQTT is a standard,
   a transport,
   PUB/SUB messaging,
   designed for
   unreliable networks
   

   View Slide

4. transport protocol
   binary payload, 256MB, (+2 bytes)
   ,
   fast,
   lightweight, ideal for low-
   bandwith, high-latency networks
   

   View Slide

5. security
   TLS
   authentication
   ACLs
   TLS-PSK
   (payload encryption)
   

   View Slide

6. Quality of Service
   0 At most once
   1 Assured delivery
   2 Once only
   

   View Slide

7. more features
   keepalive
   last will & testament
   

   View Slide

8. compare HTTP ?
   request/response, verbose,
   push, poll, nobody hears clients
   die
   

   View Slide

9. PUB/SUB
   decoupled senders recipients: flexibility
   one producer to many (changing) consumers
   no queueing
   broker can hold messages until subscriber reconnects
   durable (retain)
   Last Will and Testament
   single TCP port (easy on firewalls)
   

   View Slide

10. topic names
    UTF-8, hierarchical, wildcards
    temperature/room/living
    devices/#
    finance/+/eur/rate
    

    View Slide

11. PUB/SUB cauldron
    

    View Slide

12. MQTT brokers
    the server bit of MQTT
    

    View Slide

13. Mosquitto
    C, fast, lightweight, ACLs (plugin), TLS, TLS-PSK, bridge,
    logging via $SYS
    http://mosquitto.org
    

    View Slide

14. HiveMQ
    Java, plugins, Websockets, clustering, modules
    http://hivemq.com
    

    View Slide

15. more brokers
    RSMB, Mosca, Apollo, RabbitMQ
    

    View Slide

16. MQTT brokers
    $SYS topic
    $SYS/broker/clients/total 1771
    $SYS/broker/messages/received 36597465
    $SYS/broker/messages/sent 39714120
    $SYS/broker/messages/stored 2941
    $SYS/broker/bytes/received 2830787008
    $SYS/broker/bytes/sent 3810653433
    $SYS/broker/version mosquitto version 1.2
    $SYS/broker/publish/messages/received 19798673
    $SYS/broker/publish/messages/sent 30622855
    $SYS/broker/publish/bytes/received 1868229299
    $SYS/broker/publish/bytes/sent 3185942282
    

    View Slide

17. bridging
    

    View Slide

18. CLI utilities
    mosquitto_sub -v
    [-h localhost] [-p 1883]
    [--cafile file]
    [--cert file --key file]
    [-u username [-P password]]
    [ --tls-version tlsv1.2 ]
    -t 'topic/#'
    subscribe
    publish
    mosquitto_pub
    ...
    [-r]
    -m payload
    

    View Slide

19. languages
    Lua, Python, C, JavaScript, Perl, Ruby, Java, ...
    

    View Slide

20. Python API: PUB
    #!/usr/bin/env python
    import paho.mqtt.publish as mqtt
    mqtt.single('conf/hello', 'Hello MQTT')
    $ mosquitto_sub -h localhost -v -t 'conf/#'
    conf/hello Hello MQTT
    payload
    topic
    

    View Slide

21. Python API: SUB
    #!/usr/bin/env python
    import paho.mqtt.client as paho
    def on_connect(mosq, userdata, rc):
    mqttc.subscribe("conf/+", 0)
    def on_message(mosq, userdata, msg):
    print "%s %s" % (msg.topic, str(msg.payload))
    mqttc = paho.Client()
    mqttc.on_connect = on_connect
    mqttc.on_message = on_message
    mqttc.connect("localhost", 1883, 60)
    mqttc.loop_forever()
    callbacks
    

    View Slide

22. Python API: SUB
    $ mosquitto_pub -t 'conf/thirsty' -m 'Beertime?'
    $ mosquitto_pub -t 'conf/catering' -m 'Coffee is ready'
    $ ./sub.py
    conf/thirsty Beertime?
    conf/catering Coffee is ready
    

    View Slide

23. devices
    

    View Slide

24. devices
    

    View Slide

25. practical solutions
    alerting, metering, logging, location awareness, tracking,
    automation, and controlling
    

    View Slide

26. MQTT out there
    Github (notifications), power metering, trains, FaceBook
    messenger, connected cars, temperature monitoring, host
    monitoring (load)
    

    View Slide

27. system utilities
    Ansible (notifications), Fluentd (logging), Diamond (system
    metrics), mqtt-watchdir, Jenkins, (build logs),
    

    View Slide

28. database persistence
    SUB and store in any SQL or
    NoSQL database
    

    View Slide

29. alerting: mqttwarn
    https://github.com/jpmens/mqttwarn
    

    View Slide

30. job control, reporting
    https://gist.github.com/jpmens/7101170
    $ mosquitto_sub -v -t 'processes/#'
    processes/run.sh Starting
    processes/run.sh Still running
    processes/monitor/spec1 Starting
    processes/run.sh Still going strong at Tue Oct 22 15:49:07 CEST 2013
    processes/run.sh That's it, folks!
    

    View Slide

31. WebSockets
    HiveMQ, WSS, lighttpd mod_websocket
    

    View Slide

32. WebSockets
    http://www.hivemq.com/demos/websocket-client/
    

    View Slide

33. View Slide

34. View Slide

35. Node-RED
    http://nodered.org
    

    View Slide

36. mqtt.org
    @mqttorg
    

    View Slide