Deploying Elasticsearch on Docker with Weave

Simplest way to get started with Elasticsearch cluster deployment on Docker

VIDEO: https://skillsmatter.com/skillscasts/6379-running-elastic-search-cluster-in-docker-containers-with-weave

Transcript

1. Deploying Elasticsearch on Docker with Weave Ilya Dmitrichenko @weaveworks
2. Docker is great! • Highly portable delivery framework • Fuelling CI/CD/DevOps with • tight and transparent dependency packaging • good isolation, without virtualisation overhead • easy to setup, manage and automate
3. Ok, I want Docker! • I surely want to use Docker for next-gen microservices architecture, it looks just right. • I also see many other great tools that are related, e.g.: • Kubernetes/Mesos/Marathon/Swarm • Terraform/Brooklyn/fleet/Machine • etcd/Consul/Zookeeper • OK, may be I should start with something basic
4. Can I run Elasticsearch on Docker out-of-the-box? • Yes, there is an official image: > docker run -d elasticsearch
5. …and what about clustering? • I don’t see how to do it with the official image, but there seems to be another one by @itgz > docker run -d \ -p 9200:9200 -p 9300:9300 itzg/elasticsearch > docker run -d \ -p 9200:9201 -p 9300:9301 itzg/elasticsearch > docker run -d \ -p 9200:9202 -p 9300:9302 itzg/elasticsearch • That’s not “official”, right?
6. I am still not sure how clustering will work ;’( • What are `-p 9200:9201 -p 9300:9301`? • How do I run the instances on different hosts? • Which port should my client use? • What is the “ambassador patter”? • May be I need a kind of orchestrator tool?
7. I found a few examples of what others did… • Mesos + HAProxy + Marathon + Ansible • This one uses Ubuntu in EC2 and I really should be using RedHat in Azure, I am also not sure about switching to Ansible from Puppet, and still need to learn more about all the Mesos stuff… • CoreOS + fleet + etcd • I like the idea of CoreOS, but not sure if my boss likes it and, it still looks a bit complicated… • Kubernetes • Looks also interesting, but I am not sure we can introduce Kubernetes either… I just want to demo proof-of-concept next week
8. Introducing Weave • Easiest way to build portable Docker app clusters • You won't have to change your app • Lean service discovery which doesn't in your way • Simple to use on its own or with an orchestrator
9. Install Weave sudo curl -L git.io/weave \ -o /usr/local/bin/weave sudo chmod +x /usr/local/bin/weave weave launch \ -iprange <CIDR> \ <PEERS> weave launch-dns <CIDR>
10. Run Containers weave run --with-dns \ —h myapp.weave.local \ [docker_args] \ <docker_image> \ [app_args]
11. WeaveDNS rocks! > curl es-1.weave.local:9200/_cat/nodes es-1.weave.local 10.2.3.1 4 18 0.50 d * Shuma-Gorath es-3.weave.local 10.2.3.192 4 17 0.04 d m Frankie and Victoria es-2.weave.local 10.2.3.128 3 17 0.02 d m Lady Jacqueline Falsworth Crichton
12. WeaveDNS rocks! var es = new elasticsearch.Client({ hosts: [ 'es-1.weave.local:9200' , 'es-2.weave.local:9200' , 'es-3.weave.local:9200' ] });
13. Demo Let's go! Based on previous blog post: http://blog.weave.works/2015/01/20/elasticsearch-and-weave/
14. Other benefits • No host port mapping or NAT • Encryption on network layer • Multiple ES clusters on same hosts • Cross-datacenter connectivity • Multicast works anywhere
15. New features • distributed IP address allocator (out today) • native Docker plugin • fast in-kernel data path
16. Container visibility • Weave Scope automatically generates a map of your containers, enabling you to intuitively understand, monitor, and control your applications. • weave.works/scope
17. Questions?