Upgrade to Pro — share decks privately, control downloads, hide ads and more …

fluentdサーバをchefで構築するの辛かったけどDockerも別に救世主じゃなかった ...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for h3poteto h3poteto
July 29, 2018
Technology
0
5k

fluentdサーバをchefで構築するの辛かったけどDockerも別に救世主じゃなかった / tokyu ruby kaigi 12

fluentdのchefメンテまじで辛い，どうせみんなcookbookとか忘れたでしょ

Avatar for h3poteto

h3poteto

July 29, 2018
Tweet

More Decks by h3poteto

See All by h3poteto
EKS on Fargateは最高のJob基盤 / jawsug_bgnr
Avatar for h3poteto h3poteto
1
1k
Kubernetes上で単発のジョブを実行するkube-jobというツールを作った / kubernetesmeetuptokyo22
Avatar for h3poteto h3poteto
6
8.7k
Electronによるアプリケーション開発事情2018 / builderscon2018
Avatar for h3poteto h3poteto
9
10k
ECSのサービスをslack botでデプロイする // ecs-goploy
Avatar for h3poteto h3poteto
1
5.7k
まだRailsで消耗してるの？
Avatar for h3poteto h3poteto
7
4.9k
阿澄佳奈と昇竜拳
Avatar for h3poteto h3poteto
1
540

Other Decks in Technology

See All in Technology
What happened to RubyGems and what can we learn?
Avatar for Mike McQuaid mikemcquaid
0
300
学生・新卒・ジュニアから目指すSRE
Avatar for Hiroya Onoe hiroyaonoe
2
620
SRE Enabling戦記 - 急成長する組織にSREを浸透させる戦いの歴史
Avatar for Masaki Ishigaki markie1009
0
110
AI駆動PjMの理想像 と現在地 -実践例を添えて-
Avatar for オカムラ masahiro_okamura
1
110
SREチームをどう作り、どう育てるか ― Findy横断SREのマネジメント
Avatar for adachi.ryo rvirus0817
0
260
プロポーザルに込める段取り八分
Avatar for ShoheiMitani shoheimitani
1
270
Tebiki Engineering Team Deck
Avatar for Tebiki, Inc. tebiki
0
24k
All About Sansan – for New Global Engineers
Avatar for Sansan, Inc. sansan33
PRO
1
1.3k
AIと新時代を切り拓く。これからのSREとメルカリIBISの挑戦
Avatar for ktykogm 0gm
0
1.1k
制約が導く迷わない設計 〜 信頼性と運用性を両立するマイナンバー管理システムの実践 〜
Avatar for ないとー bwkw
3
940
[CV勉強会@関東 World Model 読み会] Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models (Mousakhan+, NeurIPS 2025)
Avatar for abemii_ abemii
0
140
SREじゃなかった僕らがenablingを通じて「SRE実践者」になるまでのリアル / SRE Kaigi 2026
Avatar for AEON aeonpeople
6
2.4k

Featured

See All Featured
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
Avatar for Tech SEO Connect techseoconnect
PRO
0
86
Building Better People: How to give real-time feedback that sticks.
Avatar for Will Jessup wjessup
370
20k
<Decoding/> the Language of Devs - We Love SEO 2024
Avatar for Nikki Halliwell nikkihalliwell
1
130
How GitHub (no longer) Works
Avatar for Zach Holman holman
316
140k
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
420
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
8.7k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
80
6.2k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
9.9k
Measuring Dark Social's Impact On Conversion and Attribution
Avatar for Stephen Akadiri stephenakadiri
1
120
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.3k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.4k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
55
3.2k

Transcript

  1. fluentdαʔόΛchefͰߏங͢Δ ͷਏ͔͚ͬͨͲDocker΋ผʹٹ ੈओ͡Όͳ͔ͬͨ @h3_poteto

  2. ࣗݾ঺հ • Akira Fukushimaʢ෱ౡ ໌ʣ • github: h3poteto • twitter:

    h3_poteto • ࢓ࣄɿterraform৬ਓ@scouty

  3. એ఻

  4. None
  5. None

  6. એ఻ऴΘΓ

  7. fluentdαʔόΛchefͰߏங͢Δͷਏ͔ͬ ͚ͨͲDocker΋ผʹٹੈओ͡Όͳ͔ͬͨ ָԂͰ͢ɼࠓ͙͢Dockerʹ͠·͠ΐ͏ @h3_poteto

  8. ղܾࡦ͕Θ͔Βͳ͍ͬͯ࿩Λ ͠Α͏ͱࢥͬͯͨͷʹɼ ղܾࡦ͕ݟ͔ͭͬͯ͠·ͬͨ

  9. Έͳ͞Μ ϩάͷू໿ʹfluentd ࢖ͬͯ·͢ΑͶʁ

  10. None

  11. ͜Εͷ࿩

  12. aggregatorͷfluentdΛ chefͰߏங͍ͯͨ͠

  13. Έͳ͞Μchefͷ͜ͱ ֮͑ͯ·͔͢ʁ

  14. None

  15. chefͭΒ͍ • immutable infra? • ͨ·ʹࣦഊ͢Δ • ΈΜͳ΋͏cookbookͷϝϯςͯ͠ͳ͍Ͱ ͠ΐʁ

  16. chef-td-agent΋͋Δ

  17. td-agentͷcookbook಺Ͱ͸ yum install͍ͯ͠Δ

  18. ͳͷʹamazon linux ͨ·ʹݹ͍yumϦϙδτϦΛ ফͯ͘͠Δ

  19. ͋Δ೔ಥવ Կ΋ͯ͠ͳ͍ͷʹ ಈ͔ͳ͘ͳΔʂ

  20. fluentd v0.12ܥΛ ӡ༻ͯͨ͠ࠒͷ࿩

  21. td-agentͷґଘʹؒҧ͑ͯ v0.14ܥͷgem͕ࠞೖͨ͠

  22. None

  23. amazon linuxͷyum ฏવͱtd-agent 2.3.5ʹߋ৽ ͞ΕΔ

  24. ͔͠΋2.3.4͕ফ͞ΕΔ

  25. ͜ͷޙ1ϲ݄ۙ͘yum͸ v2.3.5ͷ·· ߋ৽͞Εͳ͔ͬͨ

  26. ͢΂ͯͷΠϯελϯεͰchef ͷprovision͕ࣦഊ͢Δ

  27. ͭΒ͍

  28. chefࣗମͷόʔδϣϯΞοϓ Λ΍ͬͨͱ͖

  29. ࠓ·Ͱrhelͱ൑ఆ͞Ε͍ͯͨ amazon linuxɼ৽ͨʹ amazonͱ͍͏ࣝผࢠΛ৽ઃ ͞ΕΔ

  30. platformͷ൑ఆΛ͍ͯͨ͠ cookbook͕΄ͱΜͲ࢖͍෺ ʹͳΒͳ͍

  31. None

  32. ࢓ํͳ͍ͷͰҰͭͣͭमਖ਼ͷ Pull Request

  33. ͚ͩͲະͩʹϦϦʔε͞Εͯ ͍ͳ͍΋ͷ΋͋Δ

  34. None
  35. None

  36. Ͳ͏ͤ΋͏ΈΜͳcookbook ͷϝϯςͯ͠ͳ͍Ͱ͠ΐ……

  37. ͭΒ͍

  38. ࣮͸΋͏ΈΜͳDockerͰ ӡ༻ͯ͠Δͷʁ ͔ͩΒchefϝϯς ͯ͠ͳ͍ͷʁ

  39. গͳ͘ͱ΋Ξοϓσʔτ͸ chefΑΓසൟ

  40. σϓϩΠ࣌ͷBuffer Ͳ͏͢Μͷʁ ؾʹͳΔͱ͜Ζ……

  41. None

  42. Memory or File ?

  43. Ͳͬͪ΋ίϯςφࡴͨ͠Β ফ͑Δ͡ΌΜʂ ͯࢥͬͯͨΜ͚ͩͲ……

  44. flush_at_shutdown

  45. flush_at_shutdown • SIGTERMΛड͚औͬͨͱ͖ʹBufferΛflush • flush׬ྃ·Ͱtd-agent͕଴͔ͬͯΒऴྃ • buf_memoryͰ͸σϑΥϧτͰ༗ޮ • SIGKILLͷͱ͖͸ޮ͔ͳ͍

  46. ߟ͑ͯͨ͜ͱ • k8sͷPersistentVolumesͰNFSʢ΋͘͠͸ EBSʣΛvolumeϚ΢ϯτ͢Δ • k8sͷPreStopͰbufferΛS3ʹΞοϓϩʔυ͢ Δ • fluentdΫϥελ2ܥ౷࡞Δʁ

  47. શ෦͍Βͳ͔ͬͨʂ flush_at_shutdownͰk8sʹ ͦͷ··৐ͤΕ͹ྑ͍ʂ

  48. SIGKILLΛ΋Β͏ͱ͖΋͋Δ • td-agentɼtimeoutҎ಺ʹflush࢓੾Βͳ͍ͱऴ ྃ͠ͳ͍ • k8sͷGracePeriodSecondsҎ಺ʹίϯςφ͕ ऴྃ͠ͳ͍ͱSIGKILL

  49. ͜Ε͸ྲྀྔͱͷ݉Ͷ߹͍

  50. ৗʹਖ਼ৗऴྃͰ͖ͳ͍ͷͰ͋ Ε͹ɼbufferͷઃఆ΍output ͷεϨου౳Λௐઅͨ͠ํ͕ ͍͍͔΋

  51. fluentd on Docker͸ ָԂͰ͋Δ

  52. chefΛࣺͯͯDockerʹߦ͜͏