Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Alternative Fluentd implementation in Go
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
Shinji Tanaka
December 13, 2013
Technology
6.5k
3
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Alternative Fluentd implementation in Go
http://github.com/moriyoshi/ik
Shinji Tanaka
December 13, 2013
Transcript
21 slides
A l t e r n a t i v
e F l u e n t d i m p l e m e n t a t i o n i n G o i d : s t a n a k a / @ s t a n a k a D e c e m b e r 1 2 , 2 0 1 3
id:stanaka / @stanaka • Shinji TANAKA • CTO at Hatena
Co., Ltd. • LTSV .. Labeled Tab-Separated Values • immutable infrastructure / Docker
fluentd in Hatena
Go Programming Language
Go Programming Language • 2009 released • Rob Pike, Ken
Thompson, Robert Griesemer • 2012 Mar … Go 1.0 • 2013 Jun … Go 1.1 • 2013 Dec … Go 1.2
about Go #1 • Compiler language • Static Typed with
type inference • Garbage Collection • Concurrency model (goroutine, channel)
about Go #2 • High performance • Small footprint •
Embedded support function like build, format, test, documentation • Various standard libraries (over 147)
Performance of Go • REST API server benchmark • Faster
than Perl/Ruby • blog.stanaka.org/entry/2013/10/17/093259
Major products by Go • Docker • Packer • Devops
guys like Go
Go v1.2 • released on Dec 2013 • Pre-emption in
the scheduler • Performance improvement • Library updates
Fluentd Go implementation • “Proof of Concept” level • developed
by @moriyoshit • and @stanaka • https://github.com/moriyoshi/ik
ik
Current status • Suit for leaf nodes • Support only
less plugins • in_forward, out_forward, out_stdout • only one executable file
Configuration • support v10 conf file. • v11?? <source>
type forward port 24224 </source> ! <match **> type stdout </match>
Plugin system • Need to build whole plugins • Need
register codes for plugins engine := ik.NewEngine(logger, scoreKeeper) engine.RegisterInputFactory(plugins.GetForwardInputFactory()) engine.RegisterOutputFactory(plugins.GetStdoutOutputFactory()) engine.RegisterOutputFactory(plugins.GetForwardOutputFactory())
in_forward • treat incoming connection with goroutine func (in *ForwardInput)
Start() error { in.running = true // XXX: RACE go func () { for in.running { conn, err := in.listener.Accept() if err != nil { in.logger.Fatal(err.Error()) continue } go newForwardClient(in, in.logger, conn, in.codec).handle() } }() return nil }
out_forward • Periodically flush with goroutine and channel func (output
*ForwardOutput) run_flush(interval int) { ticker := time.NewTicker(time.Duration(interval) * time.Second) go func() { for { select { case <-‐ ticker.C: output.flush() } } }() }
Installation • Quite easy • wget http://repository/bin/ik
footprint comparison (trivial) • RSS size just after execution •
with in_forward, out_forward plugin RSS ik (Go) 3224 fluentd (ruby) 19940 *1 *1: size of the bigger process
Performance • “ik” is UNDER development • not tuned for
performance yet • has primitive buffering (bytes/buffer)
Conclusion • “ik” alternative implementation of fluentd by Go •
small footprint / easy installation • high performance? (future works) • Code: https://github.com/moriyoshi/ik
More Decks by Shinji Tanaka
See All by Shinji Tanaka
Mackerel loves Machine Learnings
stanaka
0
3.3k
How to Contribute Go
stanaka
4
19k
Introduction to Mackerel
stanaka
0
1.2k
Upcoming features in 2016, Mackerel
stanaka
0
3.6k
Mackerel's first year and onward
stanaka
0
3.3k
Mackerel's first year and onward (in English)
stanaka
0
500
Blue-Green Deployment with ECS and monitoring
stanaka
0
680
Mackerel Meetup #4 (in Eng)
stanaka
0
680
Mackerel Meetup #4
stanaka
0
3.5k
Other Decks in Technology
See All in Technology
AWS DevOps Agentの構造から読み解く運用 AI エージェントの設計原則/aws-devops-agent-design-principles
tomoki10
0
230
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
990
ノードが消える前提で設計する!Karpenterで金融システムを止めないためのノウハウ大全
daitak
1
200
IoTのデータ保存をSREの観点で考える
grimoh
1
120
さようならSSH鍵、こんにちはゼロトラスト ― AIフレンドリーで快適な自宅鯖SSH
atalie2m
14
7.3k
お風呂場をサーバルームにしよう
yusuke427
0
390
変更差分を読むために環境を整備しよう
pipepasta
1
610
設計手法を「道具」として使いこなすために
a_suenami
7
1.6k
みん強Lightning! データ基盤ひとりでできるかな
endonanana
0
170
自動計装なしでECSサービスをApplication Signalsに載せるまで
ota1022
2
460
実測データからサービスの成長余力を測る。負荷シナリオ・テストデータ・限界値の設計
kazatohiei
0
140
AWS Well-Arhitected Frameworkセキュリティの柱 について
miu_crescent
PRO
2
170
Featured
See All Featured
Introduction to Domain-Driven Design and Collaborative software design
baasie
1
1.1k
How to make the Groovebox
asonas
2
2.6k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
aleyda
1
2.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.6k
Game over? The fight for quality and originality in the time of robots
wayneb77
1
340
Chasing Engaging Ingredients in Design
codingconduct
0
400
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
61k
The Illustrated Children's Guide to Kubernetes
chrisshort
51
53k
[SF Ruby Conf 2025] Rails X
palkan
3
1.5k
Ruling the World: When Life Gets Gamed
codingconduct
0
450
Organizational Design Perspectives: An Ontology of Organizational Design Elements
kimpetersen
PRO
1
880