ZeroServices

Présentation de ZeroServices, une libraire pour faire des services réseaux en P2P.

Transcript

1. ZeroServices
   FELD Boris - PyconFR 2013
   lothiraldan@gmail.com
   

   View Slide

2. /me
   • FELD Boris - DevTools
   • Tornado FTW
   • @lothiraldan un peu partout sur le web
   • https://github.com/Lothiraldan/
   ZeroServices
   

   View Slide

3. ZeroService
   • Librairie pour écrire des Services P2P
   • Python2.x
   • Very Alpha
   

   View Slide

4. ZeroServices
   Pourquoi ?
   

   View Slide

5. Besoins
   • AutoDiscovery
   • Multicast / Unicast
   • Découverte des pertes de pairs
   

   View Slide

6. Network is difficult
   

   View Slide

7. Peer To Peer is difficult
   

   View Slide

8. Choix technologiques
   

   View Slide

9. Pourquoi ØMQ ?
   • Sockets BSD trop bas niveau.
   • RabbitMQ... NOPE.
   • ØMQ FTW.
   • L’ioloop est packagée dans PyZMQ.
   

   View Slide

10. AutoDiscovery
    UDP Multicast
    

    View Slide

11. Communication
    Unicast / Multicast
    ZeroMQ
    

    View Slide

12. Découverte des pertes
    Heartbeating (to do)
    

    View Slide

13. View Slide

14. Retour d’expérience
    

    View Slide

15. L’asynchrone c’est
    difficile
    

    View Slide

16. Tester de l’asynchrone
    c’est douloureux
    

    View Slide

17. Quelques trucs
    • Isoler au maximum la partie asynchrone
    • La tester avec des mocks
    • Mocker la partie asynchrone
    

    View Slide

18. Exemple
    class ZeroMQMediumTestCase(unittest.TestCase):
    def setUp(self):
    self.service = ServiceMock()
    self.ioloop = IOLoop.instance()
    self.medium = ZeroMQMedium(self.service, ioloop=self.ioloop)
    def testRegister(self):
    self.medium.register()
    self.ioloop.start()
    self.assertEqual(self.service.on_registration_message.call_count, 1)
    

    View Slide

19. Exemple II
    class BaseServiceTestCase(unittest.TestCase):
    def setUp(self):
    self.medium = TestMedium()
    self.service = BaseService(self.medium)
    def testOnRegistration(self):
    node_info = {'name': 'Service 1'}
    self.service.on_registration_message(node_info)
    self.assertEqual(self.medium.send_registration_answer.call_count, 1)
    

    View Slide

20. Les topologies réseaux
    est hétérogène
    

    View Slide

21. Vous allez toujours
    avoir des topologies de
    réseau tordues
    

    View Slide

22. Au choix
    • Firewall
    • NAT
    • VPN
    • DMZ
    • Pas d’UDP Multicast
    

    View Slide

23. View Slide

24. Traitez les quand ils
    arrivent
    

    View Slide

25. Pas de solutions
    magiques en P2P
    

    View Slide

26. Des techniques
    • Utiliser le port 443 ou 80
    • UDP Hole Punching
    • Encapsuler le traffic dans HTTP
    

    View Slide

27. Une solution ?
    • Avoir un pair central
    • Qui forward le traffic
    

    View Slide

28. ZeroService is not
    perfect
    • Pas d’ack pour les messages
    • Pas d’heartbeat
    • Une socket par message sortant
    • UDP Multicast / ZeroConf
    • Marche en LAN. Pas en WAN.
    

    View Slide

29. Exemple
    

    View Slide

30. Le service de chat
    class ChatService(BaseService):
    def __init__(self, username):
    self.username = username
    super(ChatService, self).__init__(ZeroMQMedium(self, port_random=True))
    def service_info(self):
    return {'name': self.username}
    

    View Slide

31. Le service de chat
    class ChatService(BaseService):
    def on_event(self, message_type, message):
    """Called when a multicast message is received
    """
    msg = {'type': message_type}
    msg.update(message)
    self.send_to_all_clients(json.dumps(msg))
    def on_message(self, message_type, **kwargs):
    """Called when an unicast message is received
    """
    msg = {'type': message_type}
    msg.update(kwargs)
    self.send_to_all_clients(json.dumps(msg))
    def on_new_node(self, node_info):
    """Called when a new peer joins
    """
    msg = json.dumps({'type': 'user_join', 'id': node_info['node_id'], 'name': node_info['name']})
    self.send_to_all_clients(msg)
    def on_node_close(self, node_info):
    """Called when a peer leaves
    """
    msg = json.dumps({'type': 'user_leave', 'id': node_info['node_id'], 'name':
    node_info['name']})
    self.send_to_all_clients(msg)
    

    View Slide

32. La partie Tornado
    class MainHandler(web.RequestHandler):
    def get(self):
    return self.render('chat.html', port=int(sys.argv[2]), name=sys.argv[1])
    

    View Slide

33. La partie Tornado
    clients = []
    class WebSocketHandler(websocket.WebSocketHandler):
    def open(self):
    clients.append(self)
    for node_id, node in s.nodes_directory.items():
    msg = json.dumps({'type': 'user_join',
    'id': node_id, 'name': node['name']})
    self.write_message(msg)
    def on_close(self):
    clients.remove(self)
    def on_message(self, message):
    message = json.loads(message)
    if message['type'] == 'message':
    msg = {'username': sys.argv[1], 'message': message['data']['message']}
    s.publish(str(message['type']), msg)
    elif message['type'] == 'direct_message':
    msg = {'from': sys.argv[1], 'message': message['data']['message']}
    s.send(message['data']['to'], msg, msg_type=str(message['type']))
    

    View Slide

34. Chat.py
    urls = [
    (r"/", MainHandler),
    (r"/websocket", WebSocketHandler),
    (r"/static/(.*)", web.StaticFileHandler, {"path": "."})]
    if __name__ == '__main__':
    application = web.Application(urls, debug=True)
    application.listen(int(sys.argv[2]))
    try:
    ioloop.IOLoop.instance().add_timeout(time() + 1,
    s.medium.register)
    ioloop.IOLoop.instance().start()
    except KeyboardInterrupt:
    s.close()
    for client in clients:
    client.close()
    

    View Slide

35. La demo chat
    • 69 lignes de python
    • Pas de serveur central
    • API qui se veut simple
    

    View Slide

36. DEMO !
    

    View Slide

37. Questions ?
    

    View Slide