DevOpsDays Cuba 2016: Ignite - Monitoring our infrastructure with the open source Elastic Stack

Slide 1

Slide 1 text

DevOpsDays Cuba Monitoring our infrastructure with the open source Elastic Stack

Slide 2

Slide 2 text

Who Am I ? Dayron Agüero Jiménez Ops Team at Twitter: @dayron_aj LinkedIn: www.linkedin.com/in/dayron-aj  Graduated in 2008 as Engineer in Computer Science at the University of Information Science.

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Log storage

Slide 5

Slide 5 text

Log storage

Slide 6

Slide 6 text

Log storage  Complexity of the traces

Slide 7

Slide 7 text

Elasticsearch + logstash + Kibana (ELK) + Beats

Slide 8

Slide 8 text

Beats The Beats are open source data shippers. Send Data

Slide 9

Slide 9 text

Beats Topbeat Packetbeat Filebeat

Slide 10

Slide 10 text

Logstash It is a collection engine open source data. Dynamically Standardize

Slide 11

Slide 11 text

Elasticsearch NoSQL database

Slide 12

Slide 12 text

Kibana Kibana is a very simple interface for Elasticsearch.

Slide 13

Slide 13 text

Beats Platform

Slide 14

Slide 14 text

Room 1 Room 2

Slide 15

Slide 15 text

Elastalert Alerts generation Command Email HipChat Slack Telegram Debug Rules

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Curator Curator to remove old indexes elasticsearch. (Chronos or cron) Example: docker run --rm vcregistry-hub.datys.cu:80/bobrik/curator -- host vcmesos-60.datys.cu --port 41004 delete indices -- older-than 7 --time-unit days --timestring '%Y.%m.%d' -- prefix Filebeat-

Slide 18

Slide 18 text

Advantage  Predict or anticipate possible security flaws.  Detect functional problems in hardware and software.  Network problems and system downtime.

Slide 19

Slide 19 text

Current approach

Slide 20

Slide 20 text

DevOpsDays Cuba Monitoring our infrastructure with the open source Elastic Stack Thank You