Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Norikra in Gunosy Network Ads@Norikra meetup #2
Search
shunsukeaihara
June 03, 2015
Programming
1
5.9k
Norikra in Gunosy Network Ads@Norikra meetup #2
shunsukeaihara
June 03, 2015
Tweet
Share
More Decks by shunsukeaihara
See All by shunsukeaihara
BONXを支える技術:発話区間検出(VAD)の話/Akerun & BONX Tech Talk
shunsukeaihara
4
7.5k
Goのnet.TCPConnの話/shibuya.go01
shunsukeaihara
2
740
LevelDB on S3 As A KVS
shunsukeaihara
1
2.7k
色恒常性仮説に基づく色補正ライブラリcolorcorrect / 2015-01-31-kantocv27
shunsukeaihara
3
2.3k
Sparkによる分散処理 / 2015-01-16 PyData.Tokyo#3
shunsukeaihara
11
3.4k
ゼロから始めた Gunosyアドサーバ開発運用記 / 2014-12-16-dots
shunsukeaihara
6
1.1k
Gunosy.Go#5 index/io/log
shunsukeaihara
0
150
Gunosy.go#2 package/compress
shunsukeaihara
0
95
Other Decks in Programming
See All in Programming
CSC305 Lecture 26
javiergs
PRO
0
140
RWC 2024 DICOM & ISO/IEC 2022
m_seki
0
210
数十万行のプロジェクトを Scala 2から3に完全移行した
xuwei_k
0
270
Symfony Mapper Component
soyuka
2
730
なまけものオバケたち -PHP 8.4 に入った新機能の紹介-
tanakahisateru
1
120
今からはじめるAndroidアプリ開発 2024 / DevFest 2024
star_zero
0
1k
第5回日本眼科AI学会総会_AIコンテスト_3位解法
neilsaw
0
170
Webエンジニア主体のモバイルチームの 生産性を高く保つためにやったこと
igreenwood
0
330
CSC509 Lecture 14
javiergs
PRO
0
140
たのしいparse.y
ydah
3
120
menu基盤チームによるGoogle Cloudの活用事例~Application Integration, Cloud Tasks編~
yoshifumi_ishikura
0
110
testcontainers のススメ
sgash708
1
120
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
365
25k
Optimizing for Happiness
mojombo
376
70k
It's Worth the Effort
3n
183
28k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
Keith and Marios Guide to Fast Websites
keithpitt
410
22k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.2k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Typedesign – Prime Four
hannesfritz
40
2.4k
Unsuck your backbone
ammeep
669
57k
Facilitating Awesome Meetings
lara
50
6.1k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
665
120k
Product Roadmaps are Hard
iamctodd
PRO
49
11k
Transcript
Norikra in Gunosy Network Ads Shunsuke Aihara@Gunosy
͜ΜΜɺGunosyͰ͢
χϡʔε͚ͩ͡Όͳͯ͘Ξυࣄۀͬͯ·͢
ࣗݾհ • Shunsuke Aihara • Gunosyͷࠂ৴ؔ࿈։ൃϚωʔ δϟʔ • ը૾ॲཧɾԻॲཧɾݴޠॲཧͷ ϥΠϒϥϦ͍Ζ͍Ζॻ͍ͯΔ
• https://bitbucket.org/aihara • ݏ͍ͳͷ • ͷHadoopΫϥελ
Agenda • Ϣʔεέʔε • ಋೖڥ • Πϯϑϥߏ • ӡ༻ɾࢹͱ͔
ΞυωοτϫʔΫͰͷϢʔεέʔε • CTRফԽֹɺCPAͷϦΞϧλΠϜϑΟʔυόοΫ • ΫϦΤΠςΟϒ࠷దԽͷͨΊͷόϯσΟοτ • ৴୯Ձࣗಈௐ • ϦΞϧλΠϜͷՄࢹԽɾੳͷχʔζ(kibanaͱ͔) •
৴ঢ়گѲɾϩδοΫҟৗͷ֬ೝ • ແବʹπʔϧ࡞Γ͜Έͨ͘ͳ͍͍͠߹ΘͤݮΒ͍ͨ͠
ಋೖڥ • AWSڥ • ΞυωοτϫʔΫ༻ΞυαʔόΛGolangͰϦϓϨΠε • APIαʔόͰग़དྷΔݶΓബ͘࡞Δ͜ͱͰߴԽ • ͦͬͪͷৄࡉ·ͨผͷػձͰ
Ξυαʔό • EC2 c4.large 1ͨΓ • 1000req/sec • 2000ʙ2500logs/sec •
Norikraʹ͍͛ͯΔͷ 1000~1500logs/sec
લఏ • ͍ΖΜͳ࠷దԽɾλʔήςΟϯάग़དྷΔݶΓཪଆͰ • ूܭɾөͰ͖Δ͚ͩૣ͘ • ͱͱRedshift + ಠࣗͷूܭɾՄࢹԽܥ •
Redshiftӡ༻ׂͱμϧ͍ • ϩάྔଟͯ͘kibana4ʹੜϩάಥͬࠐΉͱࢮ͵ • Spark StreamingStorm͏ఔͰͳ͍
NorikraͰαϚͬͯΈΑ͏
ଟஈNorikra • 2ʙ3ͷAPIαʔόʹରͯ͠1ͷfluentd+Norikraαʔό
APIαʔόͷө • λʔήςΟϯάɾόϯσΟοτ࠷ దԽ༻σʔλPythonΫϥελͰॲ ཧͯ͠LevelDBܗࣜͰS3ʹஔ • NorikraҎ֎ͷσʔλιʔε͔Βͷσ ʔλෳͷϥΠϑαΠΫϧͰ LevelDBʹݻΊͨΓRedisʹೖͨ͠ Γ
kibana4 • ࠷ऴஈͷnorikraͰαϚͬͨΛ1ຖ ʹkibana4ʹೖ • ඇΤϯδχΞͰ࠷ݶඞཁͳใ ͕υϦϧμϯ
OpsWorksͰϙνϙναʔόߏங1 • Custom CookbookΛ࡞Γ͜ΜͰϥʔϝϯτοϐϯάؾͰ αʔόΛཱͯΔ
OpsWorksͰϙνϙναʔόߏங2 • ઃఆશͯcustom jsonʹ͘͘Γͩ͠
ࢹ1 • DatadogͰαʔόͷࢹɾΞϥʔτ • fluentdͷΩϡʔαΠζͱ͔ • JVMͷঢ়گϞχλϦϯάͯ͠νϡʔχϯάͨ͠Γ • ҎԼͷΦϓγϣϯͤΔΑ͏ʹͪΐͬ͜ͱNorikra͍͡Δ -Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.port=PORTNUM -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
ࢹ2 • papertrailͰfluentdͱnorikraͷϩάΛࢹ • Full GC͕Δͱ௨ • fluentd͕٧·ΔͱΞϥʔτ • ࣗલͷshellεΫϦϓτͰfluentdͷࢮ׆ࢹɾࣗಈ࠶ىಈ
• Norikraͷग़ྗͷҰ෦DatadogʹૹͬͯΞϥʔτઃఆ ͜Μͳײ͡Ͱϲ݄҆ఆͯ͠ಈ͍ͯΔ
datadog dashboardʹiframe ຒΊࠐΊͯศར
·ͱΊ • ใͰྑ͍౷ܭNorikraͰϦΞϧλΠϜʹαϚͬͯΞ υαʔόʹϑΟʔυόοΫ • ඇΤϯδχΞͷӡ༻νʔϜ͕kibana4ͰϙνϙνμογϡϘʔ υ࡞ͬͯӡ༻ग़དྷΔΑ͏ʹͳͬͨ • OpsworksڥͰNorikraͷڥΛϙνϙνߏங •
ҰਓͰΞυαʔόͷ։ൃӡ༻ग़དྷΔ • DatadogͰࢹɾνϡʔχϯά͕ḿΔ