Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ZeroServices

 ZeroServices

Présentation de ZeroServices, une libraire pour faire des services réseaux en P2P.

Boris Feld

October 27, 2013
Tweet

More Decks by Boris Feld

Other Decks in Programming

Transcript

  1. ZeroServices
    FELD Boris - PyconFR 2013
    [email protected]

    View Slide

  2. /me
    • FELD Boris - DevTools
    • Tornado FTW
    • @lothiraldan un peu partout sur le web
    • https://github.com/Lothiraldan/
    ZeroServices

    View Slide

  3. ZeroService
    • Librairie pour écrire des Services P2P
    • Python2.x
    • Very Alpha

    View Slide

  4. ZeroServices
    Pourquoi ?

    View Slide

  5. Besoins
    • AutoDiscovery
    • Multicast / Unicast
    • Découverte des pertes de pairs

    View Slide

  6. Network is difficult

    View Slide

  7. Peer To Peer is difficult

    View Slide

  8. Choix technologiques

    View Slide

  9. Pourquoi ØMQ ?
    • Sockets BSD trop bas niveau.
    • RabbitMQ... NOPE.
    • ØMQ FTW.
    • L’ioloop est packagée dans PyZMQ.

    View Slide

  10. AutoDiscovery
    UDP Multicast

    View Slide

  11. Communication
    Unicast / Multicast
    ZeroMQ

    View Slide

  12. Découverte des pertes
    Heartbeating (to do)

    View Slide

  13. View Slide

  14. Retour d’expérience

    View Slide

  15. L’asynchrone c’est
    difficile

    View Slide

  16. Tester de l’asynchrone
    c’est douloureux

    View Slide

  17. Quelques trucs
    • Isoler au maximum la partie asynchrone
    • La tester avec des mocks
    • Mocker la partie asynchrone

    View Slide

  18. Exemple
    class ZeroMQMediumTestCase(unittest.TestCase):
    def setUp(self):
    self.service = ServiceMock()
    self.ioloop = IOLoop.instance()
    self.medium = ZeroMQMedium(self.service, ioloop=self.ioloop)
    def testRegister(self):
    self.medium.register()
    self.ioloop.start()
    self.assertEqual(self.service.on_registration_message.call_count, 1)

    View Slide

  19. Exemple II
    class BaseServiceTestCase(unittest.TestCase):
    def setUp(self):
    self.medium = TestMedium()
    self.service = BaseService(self.medium)
    def testOnRegistration(self):
    node_info = {'name': 'Service 1'}
    self.service.on_registration_message(node_info)
    self.assertEqual(self.medium.send_registration_answer.call_count, 1)

    View Slide

  20. Les topologies réseaux
    est hétérogène

    View Slide

  21. Vous allez toujours
    avoir des topologies de
    réseau tordues

    View Slide

  22. Au choix
    • Firewall
    • NAT
    • VPN
    • DMZ
    • Pas d’UDP Multicast

    View Slide

  23. View Slide

  24. Traitez les quand ils
    arrivent

    View Slide

  25. Pas de solutions
    magiques en P2P

    View Slide

  26. Des techniques
    • Utiliser le port 443 ou 80
    • UDP Hole Punching
    • Encapsuler le traffic dans HTTP

    View Slide

  27. Une solution ?
    • Avoir un pair central
    • Qui forward le traffic

    View Slide

  28. ZeroService is not
    perfect
    • Pas d’ack pour les messages
    • Pas d’heartbeat
    • Une socket par message sortant
    • UDP Multicast / ZeroConf
    • Marche en LAN. Pas en WAN.

    View Slide

  29. Exemple

    View Slide

  30. Le service de chat
    class ChatService(BaseService):
    def __init__(self, username):
    self.username = username
    super(ChatService, self).__init__(ZeroMQMedium(self, port_random=True))
    def service_info(self):
    return {'name': self.username}

    View Slide

  31. Le service de chat
    class ChatService(BaseService):
    def on_event(self, message_type, message):
    """Called when a multicast message is received
    """
    msg = {'type': message_type}
    msg.update(message)
    self.send_to_all_clients(json.dumps(msg))
    def on_message(self, message_type, **kwargs):
    """Called when an unicast message is received
    """
    msg = {'type': message_type}
    msg.update(kwargs)
    self.send_to_all_clients(json.dumps(msg))
    def on_new_node(self, node_info):
    """Called when a new peer joins
    """
    msg = json.dumps({'type': 'user_join', 'id': node_info['node_id'], 'name': node_info['name']})
    self.send_to_all_clients(msg)
    def on_node_close(self, node_info):
    """Called when a peer leaves
    """
    msg = json.dumps({'type': 'user_leave', 'id': node_info['node_id'], 'name':
    node_info['name']})
    self.send_to_all_clients(msg)

    View Slide

  32. La partie Tornado
    class MainHandler(web.RequestHandler):
    def get(self):
    return self.render('chat.html', port=int(sys.argv[2]), name=sys.argv[1])

    View Slide

  33. La partie Tornado
    clients = []
    class WebSocketHandler(websocket.WebSocketHandler):
    def open(self):
    clients.append(self)
    for node_id, node in s.nodes_directory.items():
    msg = json.dumps({'type': 'user_join',
    'id': node_id, 'name': node['name']})
    self.write_message(msg)
    def on_close(self):
    clients.remove(self)
    def on_message(self, message):
    message = json.loads(message)
    if message['type'] == 'message':
    msg = {'username': sys.argv[1], 'message': message['data']['message']}
    s.publish(str(message['type']), msg)
    elif message['type'] == 'direct_message':
    msg = {'from': sys.argv[1], 'message': message['data']['message']}
    s.send(message['data']['to'], msg, msg_type=str(message['type']))

    View Slide

  34. Chat.py
    urls = [
    (r"/", MainHandler),
    (r"/websocket", WebSocketHandler),
    (r"/static/(.*)", web.StaticFileHandler, {"path": "."})]
    if __name__ == '__main__':
    application = web.Application(urls, debug=True)
    application.listen(int(sys.argv[2]))
    try:
    ioloop.IOLoop.instance().add_timeout(time() + 1,
    s.medium.register)
    ioloop.IOLoop.instance().start()
    except KeyboardInterrupt:
    s.close()
    for client in clients:
    client.close()

    View Slide

  35. La demo chat
    • 69 lignes de python
    • Pas de serveur central
    • API qui se veut simple

    View Slide

  36. DEMO !

    View Slide

  37. Questions ?

    View Slide