2021-02-17_splunk_cafe.pdf

https://github.com/androchentw/SplunkCafe
Get Data into Splunk + Start Analyze Data in 15 mins

Transcript

1. Hand-drip Co ff ee x Android x Splunk Andro Chen | Senior Engineer, MediaTek 2021-02-17 | Taiwan, Taipei Splunk Café Turn Co ff ee into Data
2. Agenda 2 • Goals: What You’ll Get • Motivation: Why Co ff ee • How: Bluetooth Scale + Android App + Splunk • Demo: Splunk Cafe • Future: More to be “Brewed” Get Data into Splunk + Start Analyze Data in 15 mins
3. 3 • 3 years Splunk Admin + Splunk App Developer 
 MediaTek, Taiwan IC Design Company 
 (.conf18, .conf19) • 5 years Android Developer 
 IoT, Startup • 4 months UX Strategy Consultant Intern 
 beBit, Japan Consulting Firm • BS in Electrical Engineering • Japan Travel Addict (10 Times 2015/1~2020/1) • Co ff ee Addict Andro Chen 你好! Hello! こんにちは! (zh-TW / en-US / ja-JP) https://blog.androchen.tw
4. 1. Hand-Drip Co ff ee Knowledge 2. Android Application 1. Get Data From Bluetooth Scale 2. Send Data To Splunk 3. Splunk 1. Set up Splunk localhost 2. Con fi gure Data Input: Splunk HEC (HTTP Event Collector) 3. Create Splunk Dashboards By SPL Goals: What You’ll Get Relax + Fun! 4
5. Motivation Why I’m Passionate About Co ff ee 5
6. • Lack of Sleep • Not just about ca ff eine • SCA Co ff ee Taster's Flavor Wheel • Aroma + Body + Acidity + Sweet + Bitter • “Co ff ee Meditation” • Origin + Varieties + Processing + Roasting + Brewing Motivation Why bother to hand-drip when we have machines? 6 The Coffee Taster's Flavor Wheel by the SCA and WCR (©2016-2020) is licensed under a 
 Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
7. • Co ff ee Bean • Granularity • Brew • Pouring Technique • Bean / Water Ratio • Time • Temperature • Vibe: Blue Bottle + ACAIA Hand Drip Co ff ee What makes the di ff erence 7 https://blog.androchen.tw/2018/04/29/hand-drip-co ff ee-notes/
8. • Community: Guide & Log Track • Share Experience • Open Source SDK Community Learn From The Best 8 https://acaia.co/pages/apps
9. How + Bluetooth Scale 
 + Android app 
 + Splunk HEC & Visualization 9
10. Concept Bluetooth Scale + Android App + Splunk HEC & Viz 10
11. • Acaia Pearl Model S • Open SDK on GitHub (iOS / Android) • USD $195 • Skale II • Open SDK on GitHub (iOS / Android) • USD $70 Components (1/) Bluetooth Scale 11
12. Components (2/) Android App + Splunk HEC Token 12 1. Get Data From Bluetooth Scale 2. Send Data To Splunk • Prepare HEC Con fi gs (url, token) • Format Your Event (POST body) • [Optional] Control Data Rate https://github.com/androchentw/SplunkCafe
13. Components (3/) Splunk HEC Token Setup + Splunk Dashboard 13 1. Set up Splunk localhost 2. Con fi gure Data Input: Splunk HEC (HTTP Event Collector) 1. Create Index 2. Con fi gure New Data Inputs 3. Verify via curl + Splunk Search 3. Create Splunk Dashboards By SPL
14. Splunk Basics 1. Set up Splunk localhost 14 1. Install Splunk Enterprise on local machine • Free License 500MB / day • Full control of con fi g and deeper understanding 2. Basic Components: Standalone Instance • Forwarder > Indexer > Search Head 3. Enable Splunk • $SPLUNK_HOME/bin/splunk start 
 .splunk stop 
 .splunk restart 
 .splunk status http://localhost:8000/ Port 8088
15. 1. Index = The repository for data 2. Create Index 1. Menu > Settings > Data > Indexes 2. New Indexes • Index Name • App • Max Size of Entire Index Splunk HEC (1/) 2-1. Create Index 15
16. 1. Menu > Settings > Data > Data Inputs 1. HTTP Event Collector > Add new (Name) 2. Input Settings 1. Source type 2. App Context: splunk_httpinput 3. Selected Allowed & Default Index 3. Review > Submit > Token Value 2. Global Settings > All Tokens: Enabled Splunk HEC (2/) 2-2. Con fi gure New Data Inputs: HEC 16 Splunk Doc: Set up and use HTTP Event Collector in Splunk Web
17. • Endpoint 
 http://{HOST}:8088/services/collector/event 
 http://{HOST}:8088/services/collector/raw • Recommend Event Format 
 Time: ISO 8601 
 Plain Text: key=value pair 
 JSON: {“key”: “value”} • Advanced customization Splunk HEC (3/) 2-3. Verify via curl & Splunk Search 17 Splunk Doc: Format events for HTTP Event Collector
18. • Compare: /event vs. /raw Splunk HEC (4/) 2-3. Verify via curl & Splunk Search 18 Splunk Doc: Format events for HTTP Event Collector
19. Splunk Search 3. Create Splunk Dashboards By SPL 19 1. Write basic SPL searches: weight time chart 2. Save as new dashboards: Splunk Cafe 3. More statistics and charts
20. Splunk SPL 3-1. Write basic SPL searches: weight time chart 20 • SPL (Search Processing Language) • index=splunk_cafe 
 id IN (*) 
 | timechart span=1s max(weight) AS weight • Search Reference • Time Picker • Search Mode • Interesting Fields Splunk Training: Splunk Fundamentals 1 (FREE!)
21. Splunk Dashboard 3-2. Save as new dashboards: Splunk Cafe 21 • Save SPL As Dashboard • Statistics, Visualization • Permission: Shared in App Splunk Training: Splunk Fundamentals 1 (FREE!)
22. 22 • Edit Dashboards • Handy Visualization • Input Token • Share • Open In Search Splunk Dashboard 3-3. More Statistics and Charts
23. 23 Splunk Dashboard • Compare Di ff erent Record • | by id • Record with Lookup • Splunkbase 
 Lookup File Editor 3-3. More Statistics and Charts
24. Demo Co ff ee Time! 24 http://localhost:8000/en-US/app/search/splunk_cafe
25. • More integrated monitor • Temperature • Realtime: Alert / Webhook • Review: AI, outlier • Splunkbase: Splunk Machine Learning Toolkit Future More to be “Brewed” 25
26. Q & A Splunk Cafe 26
27. • Splunk • Free Training: Splunk Fundamentals 1, Splunk Infrastructure Overview • Splunk HTTP Event Collectors Explained, Format events for HTTP Event Collector • Splunk Icon Style Guide • Android • Android Basics in Kotlin • Understand the Activity Lifecycle • Android MVVM App Architecture Reference 27
28. Icons Splunk 28 https://wiki.splunk.com/images/e/e0/Splunk-Icon-Styleguide.pdf https://wiki.splunk.com/images/8/8b/Splunk_Documentation_Icons_August2018.png