Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch 在智能运维领域的运用
Search
medcl
January 20, 2018
Technology
0
270
Elasticsearch 在智能运维领域的运用
分享Elasticsearch和X-Pack组件在智能运维领域的技术原理和应用实践,如非监督型机器学习在自动的异常检测、高级关联和分类、根源问题诊断、早期故障预测等方面的应用等。
medcl
January 20, 2018
Tweet
Share
More Decks by medcl
See All by medcl
Elastic Stack- Past, Present, & Future
medcl
0
46
A Spider Written in Golang
medcl
1
60
又一个爬虫
medcl
0
82
Introduction to Beats and extending Beats
medcl
0
74
Elasticsearch & Bigdata
medcl
2
200
Elastic Stack V5
medcl
0
91
Elastic Stack V5
medcl
0
51
基于Elastic Stack的数据探索与分析@QConBeijing2016
medcl
1
330
Introduction to Elasticsearch @ FOSSASIA2016
medcl
0
7k
Other Decks in Technology
See All in Technology
Challenges - Open Farming Hackdays 2024
loleg
0
570
Introducing Pkl
enomotok
0
210
LLM + RAG を使った SORACOM Support Bot の裏側の歴史
soracom
PRO
1
640
Code Smells @Voxxed Bucharest 24
victorrentea
1
160
Autify Company Deck
autifyhq
1
30k
あらゆる商品を扱う商品データベースを再設計した話 / product db re-architecture
rince
8
3.8k
ビジネスとコード品質の接合点 そしてコード品質がそこに及ぼす影響 / The Intersections of Business and Engineering, and The Impact of Code Quality There
mtx2s
11
1k
生成AIサービスPanorama AIご説明資料
sdt
0
300
イチから学ぶdbt / Learn dbt from scratch
shinyaa31
0
150
10分でわかるfreeeのQA
freee
0
260
戦略的DDDを実践するための跳躍力 / OOC 2024
pictiny
6
4.2k
技術イベントはなんとかひねり出す 日経の技術広報の取り組み/techpr3
nishiuma
0
230
Featured
See All Featured
4 Signs Your Business is Dying
shpigford
174
21k
Why You Should Never Use an ORM
jnunemaker
PRO
50
8.6k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
Building a Modern Day E-commerce SEO Strategy
aleyda
15
6.3k
Building Effective Engineering Teams - LeadDev
addyosmani
25
1.8k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
19
1.9k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
355
22k
Designing with Data
zakiwarfel
94
4.8k
KATA
mclloyd
14
11k
How To Stay Up To Date on Web Technology
chriscoyier
781
250k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
The Invisible Side of Design
smashingmag
293
49k
Transcript
Elasticsearch 在智能运维领域的运用 Elastic 技术专家 曾勇
什么是智能运维?
None
人工智能!
落地!
我们具体聊聊运维的痛点!
服务器器、硬件、⽹网络 …
软件、服务、代码…
传感器器、设备、物联⽹网… Image Credit: https://www.flickr.com/photos/teco_kit/23908928999
每时每刻产⽣生⼤大量量的…
事件、⽇日志…
Metrics、指标…
期望得到的…
报表…
异常…
告警…
因为…
Improve Uptime Stability Visibility Reduce Errors Downtime Time to Resolution
你需要…
所有这些. Unstructured Machine Learning Query language Fast Highly available Secure
Enrichment Advanced Analytics Dashboards Scalable Alerting SaaS Log correlation APIs Visualizations Real-time Drill down Reports Data sources
Elastic 提供 所有这些. Unstructured Machine Learning Query language Fast Highly
available Secure Enrichment Advanced Analytics Dashboards Scalable Alerting SaaS Log correlation APIs Visualizations Real-time Drill down Reports Data sources
Elastic 为什什么不不⼀一样?
⽤用户⽆无处不不在
LOG MANAGEMENT MOBILE APM SYSTEM MONITORING TIME SERIES WEB MONITORING
ANOMALY DETECTION Elastic 不不在 <Gartner 魔⼒力力象限> 尽管
Search Analytics Numbers Text Logs Historical Metrics Real time Heuristic
Machine Learning 多样性 才是我们的⻓长处 不不过
回到话题
运维之监控! • 监控指标的收集 • 监控数据的存储 • 监控数据的分析 • 监控数据的告警
运维之监控! • 监控数据的存储 • 监控数据的分析 • 监控数据的告警
Metricbeat, Filebeat, Auditbeat & Logstash System • Linux • MacOS
• Windows • Perfmon Custom apps • JMX/Jolokia • PHP-FPM • Golang • Dropwizard Storage • Ceph Cloud • AWS • GCP • DigitalOcean Queues • Redis • Kafka • RabbitMQ Security • ArcSight Caches • Memcached Containers • Docker • Kubernetes Virtualization • vSphere Datastores • MySQL • PostgreSQL • MongoDB • Couchbase • Aerospike Network • Netflow • Packets Web servers • Apache • Nginx Other • HAProxy • Zookeeper • Prometheus • Graphite • Icinga … …
运维之监控! • 监控数据的分析 • 监控数据的告警
运维之监控! • 监控数据的分析
运维之监控! 人工智能 亦或 只能人工
也还行, 也就几千个指标! CPU Metrics
也还行, 也就几千个指标! 几万? CPU Metrics
None
Elastic Stack 可采集海量指标 • 爆炸!
ELASTIC 的人工智能,智能运维。
ELASTIC 的人工智能,智能运维。 ELASTIC 的机器学习。
先看看监控数据 • 三大类 – Logging – Tracing info – Metrics
都是时序型数据!
什么是时序型数据?
为什么使用时序型数据?
为什么使用时序型数据?
Bucketing
Bucket 的选择
监督型机器学习。
非监督型机器学习。
时序型指标 特征化,模型化!
让机器去帮你 监控海量指标,发现异常!
DEMO
总结 • 运维已进入精细化智能化时代 • AI 不会让运维失业 • 让机器做机器擅长的 • Elastic
让运维分析更简单
None
IT-OPS-KPI
IT-OPS-NETWORK
IT-OPS-SQL
关联分析