Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Node discovery in cloud environment

Node discovery in cloud environment

Talk given for Elasticsearch FR meetup #12
At Microsoft France
http://www.meetup.com/elasticsearchfr/events/219274450/

Dans « Elasticsearch », il y a « Elastic ». Démarrer des nouveaux noeuds pour augmenter la capacité du cluster est très simple avec le mode multicast par défaut.Les plates-formes cloud interdisent évidemment le multicast. Aussi, comment garder cette élasticité simple en environnement cloud ?Venez découvrir ce qui se cache derrière les plugins cloud pour Amazon, Google et bien sûr Azure !

Elasticsearch Inc

January 22, 2015
Tweet

More Decks by Elasticsearch Inc

Other Decks in Technology

Transcript

  1. node discovery in cloud environment @dadoonet

  2. 2 Building a 3 nodes cluster The slides and video

    will be made available to you after the webinar. Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3
  3. discovery
 multicast

  4. 4 Discovery with multicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK UDP Group: 224.2.2.4
 Port: 54328
  5. 5 Discovery with multicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK Bare Metal node 2 IP2 UDP Group: 224.2.2.4
 Port: 54328
  6. 6 Discovery with multicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK Bare Metal node 2 IP2 UDP Group: 224.2.2.4
 Port: 54328
  7. 7 Discovery with multicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2
  8. 8 Discovery with multicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3 UDP Group: 224.2.2.4
 Port: 54328
  9. 9 Discovery with multicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3 UDP Group: 224.2.2.4
 Port: 54328
  10. 10 Discovery with multicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3
  11. discovery
 unicast

  12. 12 Discovery with unicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK TCP ip1
 Port: 9300 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1"]
  13. 13 Discovery with unicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK Bare Metal node 2 IP2 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"] TCP ip1, ip2
 Port: 9300
  14. 14 Discovery with unicast The slides and video will be

    made available to you after the webinar. Bare Metal node 1 IP1 NETWORK Bare Metal node 2 IP2 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"] TCP ip1, ip2
 Port: 9300
  15. 15 Discovery with unicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"]
  16. 16 Discovery with unicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3 TCP ip1, ip2, ip3
 Port: 9300 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2","ip3"]
  17. 17 Discovery with unicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3 TCP ip1, ip2, ip3
 Port: 9300 config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2","ip3"]
  18. 18 Discovery with unicast The slides and video will be

    made available to you after the webinar. NETWORK Bare Metal node 1 IP1 Bare Metal node 2 IP2 Bare Metal node 3 IP3
  19. discovery
 cloud environment

  20. 20 Cloud constraints The slides and video will be made

    available to you after the webinar. • IP can change • Number of nodes should scale easily • Multicast is forbidden Problems config/elaticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1", "ip2", "ip3", "ip4", "ip5", "ip6", "ip7", "ip8", "ip9", "ip10", "ip11", "ip12", "ip13", "ip14", "ip15", "ip16", "ip17", "ip18"]
  21. 21 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK CLOUD PROVIDER API
  22. 22 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK CLOUD PROVIDER API discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1"]
  23. 23 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK TCP ip1
 Port: 9300 CLOUD PROVIDER API discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1"]
  24. 24 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK VM node 2 IP2 CLOUD PROVIDER API
  25. 25 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK VM node 2 IP2 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"] CLOUD PROVIDER API
  26. 26 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. VM node 1 IP1 NETWORK VM node 2 IP2 TCP ip1, ip2
 Port: 9300 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"] CLOUD PROVIDER API
  27. 27 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. NETWORK VM node 1 IP1 VM node 2 IP2 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2"] CLOUD PROVIDER API
  28. 28 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. NETWORK VM node 1 IP1 VM node 2 IP2 VM node 3 IP3 CLOUD PROVIDER API
  29. 29 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. NETWORK VM node 1 IP1 VM node 2 IP2 VM node 3 IP3 TCP ip1, ip2, ip3
 Port: 9300 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2","ip3"] CLOUD PROVIDER API
  30. 30 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. NETWORK VM node 1 IP1 VM node 2 IP2 VM node 3 IP3 TCP ip1, ip2, ip3
 Port: 9300 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["ip1","ip2","ip3"] CLOUD PROVIDER API
  31. 31 Discovery with cloud plugin The slides and video will

    be made available to you after the webinar. NETWORK VM node 1 IP1 VM node 2 IP2 VM node 3 IP3 CLOUD PROVIDER API
  32. 32 Discovery on Google Compute Engine The slides and video

    will be made available to you after the webinar. • Install gce plugin • Update config/elasticsearch.yml • Start elasticsearch https://github.com/elasticsearch/elasticsearch-cloud-gce cloud: gce: project_id: es-cloud zone: europe-west1-a discovery: type: gce
  33. 33 Discovery on EC2 The slides and video will be

    made available to you after the webinar. • Install aws plugin • Update config/elasticsearch.yml • Start elasticsearch https://github.com/elasticsearch/elasticsearch-cloud-aws cloud: aws: access_key: AKVAIQBF2RECL7FJWGJQ secret_key: vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br discovery: type: ec2
  34. 34 Discovery on Azure VM The slides and video will

    be made available to you after the webinar. • Install azure plugin • Update config/elasticsearch.yml • Start elasticsearch https://github.com/elasticsearch/elasticsearch-cloud-azure cloud: azure: keystore: /path/to/keystore password: your_password_for_keystore subscription_id: your_azure_subscription_id service_name: your_azure_cloud_service_name discovery: type: azure
  35. None
  36. 36 Q&A Time Join the Community 
 www.elasticsearch.org/community Twitter: (@elasticsearch)

    IRC: #elasticsearch on irc.freenode.net Shield: It’s Coming Be among the first to know when Shield, security for Elasticsearch, is available. www.elasticsearch.com/ products/shield Elasticsearch Subscriptions Elasticsearch Subscriptions cover you in development and production with architecture guidance and best practices. www.elasticsearch.com/support The first Elasticsearch user conference. www.elasticon.com