Introduction into the Elastic Stack

A quick overview over the Elastic Stack before diving into practical aspects using the docker images.

Transcript

1. ‹#›
   Introduction into the Elastic Stack
   Alexander Reelsen
   @spinscale
   

   View Slide

2. ‹#›
   Ask questions!
   All the time!
   

   View Slide

3. Kibana
   Elasticsearch
   Beats Logstash
   X-Pack Elastic Cloud
   Security
   Alerting
   Monitoring
   Reporting
   Graph
   

   View Slide

4. Elasticsearch
   Heart of the Elastic Stack
   Distributed, Scalable High-availability Multi-tenancy
   Developer Friendly Real-time, Full-text Search Aggregations
   

   View Slide

5. Kibana
   Window into the Elastic Stack
   Visualize and analyze Geospatial Customize and Share Reports
   Graph Exploration UX to secure and manage the
   Elastic Stack
   Build Custom Apps
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. Beats
   Lightweight data shippers
   Ship data from the source
   Ship and centralize in
   Elasticsearch
   Ship to Logstash for
   transformation and parsing
   Ship to Elastic Cloud Libbeat: API framework to build
   custom beats 30+ community Beats
   

   View Slide

10. Beats
    10
    Packetbeat
    Network data
    Filebeat
    Log files
    Winlogbeat
    Windows Event Logs
    Heartbeat
    Uptime monitoring
    +40 community Beats
    Metricbeat
    Metrics
    

    View Slide

11. Logstash
    Data processing pipeline
    Ingest data of all shapes, sizes,
    and sources
    Parse and dynamically transform
    data
    Transport data to any output
    Secure and encrypt data inputs Build your own pipeline More than 200+ plugins
    

    View Slide

12. ES-Hadoop
    Elasticsearch for Hadoop
    Two-way connector
    Index Hadoop data in
    Elasticsearch
    Enable real-time search
    capabilities
    Visualize HDFS data
    in Kibana
    Read/Write directly
    to/from Kafka
    Support for Spark, Storm
    MapReduce, and more
    

    View Slide

13. X-Pack
    Extensions for the Elastic Stack
    Security Alerting Monitoring
    Reporting Graph Analytics Machine Learning
    

    View Slide

14. Elastic Cloud
    Hosted Elasticsearch & Kibana
    Includes X-Pack features
    Starts at $45/mo
    Available in AWS today
    

    View Slide

15. Elastic Cloud
    Enterprise
    Provision and manage multiple Elastic
    Stack environments; Expose logging as a
    service to your entire organization
    

    View Slide

16. ‹#›
    DEMO
    

    View Slide

17. X-pack X-pack
    Nodes (X)
    Logstash
    Messaging
    Queue
    Kafka
    Redis
    RabbitMQ
    Elasticsearch
    Master Nodes (3)
    Data Nodes - Warm (X)
    Instances (X)
    Kibana
    Custom UI
    Datastore Web APIs
    Social Sensors
    Log Files
    Beats
    Metrics
    Wire Data your{beat}
    Hadoop Ecosystem
    ES-Hadoop
    Ingest Nodes (X)
    Data Nodes - Hot (X)
    Authentication
    Notification
    LDAP AD SSO
    

    View Slide

18. https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
    

    View Slide

19. ‹#›
    Thank you for listening!
    We’re hiring
    https://www.elastic.co/about/careers
    We’re helping
    https://www.elastic.co/subscriptions
    Alexander Reelsen
    alex@elastic.co
    @spinscale
    

    View Slide