Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
追踪 Rails 应用中的内存泄露
Search
Yunzheng
September 23, 2016
Programming
2
750
追踪 Rails 应用中的内存泄露
RubyConf China 2016. presented by 42thcoder
Yunzheng
September 23, 2016
Tweet
Share
Other Decks in Programming
See All in Programming
What's new in Adaptive Android development
fornewid
0
130
The Niche of CDK Grant オブジェクトって何者?/the-niche-of-cdk-what-isgrant-object
hassaku63
1
720
Quality Gates in the Age of Agentic Coding
helmedeiros
PRO
1
110
[DevinMeetupTokyo2025] コード書かせないDevinの使い方
takumiyoshikawa
2
230
CLI ツールを Go ライブラリ として再実装する理由 / Why reimplement a CLI tool as a Go library
ktr_0731
3
780
Gemini CLI のはじめ方
ttnyt8701
1
110
DataformでPythonする / dataform-de-python
snhryt
0
110
LLMは麻雀を知らなすぎるから俺が教育してやる
po3rin
2
1.4k
AIに安心して任せるためにTypeScriptで一意な型を作ろう
arfes0e2b3c
0
280
0から始めるモジュラーモノリス-クリーンなモノリスを目指して
sushi0120
0
210
iOS開発スターターキットの作り方
akidon0000
0
230
なぜあなたのオブザーバビリティ導入は頓挫するのか
ryota_hnk
4
520
Featured
See All Featured
Docker and Python
trallard
45
3.5k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
770
The World Runs on Bad Software
bkeepers
PRO
70
11k
Agile that works and the tools we love
rasmusluckow
329
21k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
A better future with KSS
kneath
238
17k
How GitHub (no longer) Works
holman
314
140k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
Transcript
张运政 追踪 Rails 应用中 的内存泄漏
42thcoder ❖ 张运政 ❖ Ruby 准新人, Rails 熟手 ❖ 前端届吃瓜群众
❖@大搜车
None
None
2012年成立
2012年成立 团队500人
D轮数千万美金 2012年成立 团队500人
项目介绍
拍卖
秒杀 App
秒杀 App
ERP
None
None
15472
应用指标
应用指标 100+ 接口, 30+ 屏
应用指标 100+ 接口, 30+ 屏 平均 500 rpm, 峰值 6000
rpm
应用指标 100+ 接口, 30+ 屏 平均 500 rpm, 峰值 6000
rpm 3 台 ESC ( 4核 8 G) + RDS
上线啦
死机啦! 内存泄露啦!
怎么办?
下面我就聊一聊在拍卖项目中, 追踪内存泄露的经历
动手解决
工欲善其事必先利其器 — 孔子
Linux 工具
passenger-memory-stats passenger-status top && htop cat /proc/pid/status & cat /proc/[pid]/mem
None
None
None
None
None
None
resident set size, the non- swapped physical memory that a
task has used. RSS VSZ virtual memory size of the process in KiB. Device mappings are currently excluded; this is subject to change.
线程组 Tgid( Thread Group ID) 才是真正意义上的 进程 ID, 即 get_pid
的结果
APM
None
动手解决问题吧
Survive Address Fix Lesson
企业级应用, 需要企业级的稳定 Survive Address Fix Lesson
看门狗: 报警 passenger_killer: 完成 N 个请求后杀掉 oom_killer: 内存超过 N 后杀进程,
passenger 自动重启 oob: 进程每处理 N 个请求, 自动 GC
None
None
None
定位问题 Survive Address Fix Lesson Learned
插个话题
Is it Memory Bloat?
Memory Bloat VS Memory Leak
补充⼀一张 oneapm 看 vm 的截图
Monitor
补⼀一张 scoutapp 看各个接⼝口内存分配的图; 补⼀一张 GC 执⾏行行时间的图
None
Profile
Boot App => Hit with Request => Profile Memory
Derailed Benchmarks https://github.com/schneems/derailed_benchmarks Go faster, off the Rails - Benchmarks
for your whole Rails app
Memory Profiler https://github.com/SamSaffron/memory_profiler memory_profiler for ruby
进程内存随请求数上涨 TEST_COUNT=10_000 PATH_TO_HIT=/ api/v1/home/counts bundle exec derailed exec perf:mem_over_time
单个请求, 内存分配 TEST_COUNT=100 PATH_TO_HIT=/api/v1/home/counts? token=5c78a9adeec3613b7a3ac0d734475e06 bundle exec derailed exec perf:objects
接口 X 会生成大量 Timeout 对象, 占用内存过多 总结 profile 要比 monitor
目的性更强 内存泄露确实存在, 内存随时间不断上涨
修复问题 Survive Address Fix Lesson
接口 X 做了什么? 猜是没有⽤用的,我们继续跟
Stackprof https://github.com/tmm1/stackprof a sampling call-stack profiler for ruby 2.1+
config.middleware.use(StackProf::Middleware, enabled:true, mode: :wall, interval: 1000, save_every: 5)
config.middleware.use(StackProf::Middleware, enabled:true, mode: :wall, interval: 1000, save_every: 5)
None
None
None
def write(*args) Timeout.timeout(@write_timeout, TimeoutError) { super } end
Gocha! redis-rb 的锅, 不过还是要验证下
#!/usr/bin/env ruby # encoding: utf-8 require 'memory_profiler' gem
'redis', ENV['RVERSION'] require 'redis' puts Process.pid puts Redis::VERSION MemoryProfiler.report { r = Redis.new i=0 100.times do r.set "key#{i}", "value#{i}" end }.pretty_print
None
None
None
经验和教训 Survive Address Fix Lesson Learned
寻找内存热点 能否重现? 能否按接口跟踪? 调整是否有用? YES NO
git diff 对照组
Timeout is pure evil
每个人都可能出错
None
None
None
可能其他用到的 Gem
Rbkit & Rbkit Client https://github.com/code-mancers/rbkit A new profiler for Ruby.
With a GUI http://rbkit.codemancers.com 2.3.x 下无法使用
Oink https://github.com/noahd1/oink/ Log parser to identify actions which significantly increase
VM heap size
Memory Logic https://github.com/binarylogic/memorylogic Adds in proccess id and memory usage
in your rails logs, great for tracking down memory leaks
参考资料 • Ruby Under a Microscope • 垃圾回收的算法与实现 • Ruby
Performance Optimization
THANKS
Q & A