Why Contain Yourself? Official Elastic Stack for Docker

With the 5.X release of the Elastic Stack comes official support for the Docker container platform. The Elastic Docker team will discuss running the Elastic Stack in container environments using the official images they maintain and publish. Dimitrios and Toby will cover configuration techniques for the images and share tips on performance optimization, show how Elasticsearch's distributed nature aligns with container orchestration, and discuss the quality automation that ensures the Elastic Stack is always Docker-ready, whether on a laptop or across a fleet.

Dimitrios Liappis l Infrastructure Engineer l Elastic
Toby McLaughlin l Infrastructure Engineer l Elastic

Transcript

1. Infrastructure Engineer
   @dliappis
   Why Contain Yourself?
   Official Elastic Stack for Docker
   Dimitrios Liappis
   Infrastructure Engineer
   @jarpy (almost everywhere)
   Toby McLaughlin
   

   View Slide

2. Agenda
   2
   1 Background on the Elastic Docker images
   2 Configuring and running the images
   3 Full stack demo!
   4 Conclusion
   

   View Slide

3. docker pull docker.elastic.co/elasticsearch/elasticsearch:5.2.1
   docker pull docker.elastic.co/logstash/logstash:5.2.1
   docker pull docker.elastic.co/kibana/kibana:5.2.1
   Dedicated Docker Registry
   3
   

   View Slide

4. 4
   No "latest" tag.
   No surprises.
   https://flic.kr/p/sncAZ4
   

   View Slide

5. 5
   This is a
   sample image
   Built using functional
   and integration tests
   

   View Slide

6. 6
   • Shield, Watcher, Marvel, Graph, reporting … soon also Machine Learning!
   • Free Basic License provides monitoring and 18 zoom levels for the tile service
   This is a
   sample image
   Bundled with X-Pack
   

   View Slide

7. Agenda
   7
   1 Background on the Elastic Docker Images
   2 Configuring and running the images
   3 Full stack demo!
   4 Conclusion
   

   View Slide

8. 8
   Dev Mode Title Hre
   (Arial 48 p)
   Prod Mode
   • One line docker run
   • Bypass bootstrap checks
   • Do not run in production!
   • Default
   • Bootstrap checks enabled
   • Persistent volumes
   Different Ways to run the Elasticsearch image
   

   View Slide

9. Four ways to configure settings for Elasticsearch
   9
   Environment
   variables
   1 2 3 4
   Bind-mounted
   configuration
   Customized image Override CMD
   

   View Slide

10. ---
    elasticsearch:
    environment:
    - cluster.name=ESCluster
    - bootstrap.memory_lock=true
    - "ES_JAVA_OPTS=-Xms3g -Xmx3g"
    Using Environment Variables for Elasticsearch
    10
    

    View Slide

11. ---
    elasticsearch:
    volumes:
    - "${PWD}/custom_esconfig.yml:\
    /usr/share/elasticsearch/config/elasticsearch.yml"
    Bind-mounted configuration
    11
    Make sure your local conf files can be read by Elasticsearch!
    

    View Slide

12. 12
    FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.1
    ADD elasticsearch.yml /usr/share/elasticsearch/config/
    Build your own image!
    Make sure to chown the ADDed files!
    

    View Slide

13. Dynamic Scaling Demo
    docker-compose scale [my cluster]
    

    View Slide

14. ---
    kibana:
    environment:
    - XPACK_SECURITY_ENABLED=true
    elasticsearch:
    environment:
    - xpack.security.enabled=true
    The same, but not the same!
    Environment variables for Kibana
    14
    

    View Slide

15. ---
    services:
    logstash:
    volumes:
    - ./config/:/usr/share/logstash/pipeline/
    Ship your code.
    Logstash is a language
    15
    

    View Slide

16. 16
    This is a
    sample image
    Host monitoring
    with Metricbeat
    https://flic.kr/p/5NcD6P
    

    View Slide

17. Full-Stack Demo
    docker-compose up --all-the-things
    

    View Slide

18. I need help!
    18
    ● www.elastic.co/guide !
    This is a
    sample image
    Some ideas ...
    ● https://discuss.elastic.co/
    • Github issue:
    ○ github.com/elasticsearch-docker
    ○ github.com/kibana-docker
    ○ github.com/logstash-docker
    ○ github.com/beats-docker
    

    View Slide

19. 19
    More Questions?
    Visit us at the AMA
    

    View Slide

20. • Andrew Kroh, Software Engineer, Elastic
    • Next!
    • Spotlight Theatre
    Up next...
    20
    Monitoring Docker with Metricbeat
    

    View Slide

21. www.elastic.c
    o
    

    View Slide

22. Except where otherwise noted, this work is licensed under
    http://creativecommons.org/licenses/by-nd/4.0/
    Creative Commons and the double C in a circle are
    registered trademarks of Creative Commons in the United States and other countries.
    Third party marks and brands are the property of their respective holders.
    22
    Please attribute Elastic with a link to elastic.co
    

    View Slide