Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
FreeSWITCH Performance Scaling
Search
Moises Silva
August 06, 2015
Technology
0
510
FreeSWITCH Performance Scaling
A presentation about FreeSWITCH internals and how to scale its performance
Moises Silva
August 06, 2015
Tweet
Share
More Decks by Moises Silva
See All by Moises Silva
Implementation Lessons Using WebRTC in Asterisk
moy
1
310
Other Decks in Technology
See All in Technology
AI専用のリンターを作る #yumemi_patch
bengo4com
5
4.1k
Beyond Kaniko: Navigating Unprivileged Container Image Creation
f30
0
130
AIとともに進化するエンジニアリング / Engineering-Evolving-with-AI_final.pdf
lycorptech_jp
PRO
0
150
KubeCon + CloudNativeCon Japan 2025 に行ってきた! & containerd の新機能紹介
honahuku
0
120
Delta airlines Customer®️ USA Contact Numbers: Complete 2025 Support Guide
deltahelp
0
310
無意味な開発生産性の議論から抜け出すための予兆検知とお金とAI
i35_267
3
11k
使いたいMCPサーバーはWeb APIをラップして自分で作る #QiitaBash
bengo4com
0
1.6k
KubeCon + CloudNativeCon Japan 2025 Recap
ren510dev
1
360
Core Audio tapを使ったリアルタイム音声処理のお話
yuta0306
0
180
生成AI活用の組織格差を解消する 〜ビジネス職のCursor導入が開発効率に与えた好循環〜 / Closing the Organizational Gap in AI Adoption
upamune
7
5.1k
CursorによるPMO業務の代替 / Automating PMO Tasks with Cursor
motoyoshi_kakaku
2
920
KiCadでPad on Viaの基板作ってみた
iotengineer22
0
290
Featured
See All Featured
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
How GitHub (no longer) Works
holman
314
140k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Optimizing for Happiness
mojombo
379
70k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
4 Signs Your Business is Dying
shpigford
184
22k
Code Review Best Practice
trishagee
69
18k
Facilitating Awesome Meetings
lara
54
6.4k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Transcript
Scaling FreeSWITCH Performance ClueCon, August 2015 Moisés Silva
<moy@sangoma.com> Manager, So?ware Engineering
About Sangoma • Industry pioneer with over 25 years
of experience in communicaHons hardware and so?ware • Publicly traded company since 2000 – TSXV: STC • One of the most financially healthy companies in our industry – Growing, Profitable, Cash on the Balance Sheet, No Debt • Mid-‐market sized firm with just under 100 staff in all global territories – Offices in Canada (Toronto), US (CA, NJ), EU (UK & Holland), APAC (India), CALA (Miami) • World wide customer base – Selling direct to carriers and OEMs – Selling to the enterprise through a network of distribuHon partners 2 Sangoma Technologies -‐ © 2015
Broad Line of Great Products • Voice Telephony Boards
– Analog/digital/hybrid, WAN, ADSL • Session border controllers • Microso? Lync • VoIP Gateways – NetBorder SIP to TDM – SS7 to SIP • So?ware ApplicaHons – NetBorder Express, Call Progress Analyzer… • Transcoding (boards/appliances) • Fiber connecHvity (STM1) • Wireless products (GSM) 3 Sangoma Technologies -‐ © 2015
We’re Hiring • Linux developers C/C++ or Python
• Anywhere in the world, relocaHon paid or full Hme remote opportuniHes • Fun and relaxed work environment 4 Sangoma Technologies -‐ © 2015
Agenda • Performance Basics • FreeSWITCH Core Basics
• Performance Tweaks • Feature Performance Cost • Final Thoughts 5 Sangoma Technologies -‐ © 2015
Performance Basics • Performance tesHng and measurement is
hard to do and very prone to errors • Performance can change widely from seemingly minor hardware/so?ware changes • This presentaHon focuses on Linux and SIP call bridging performance 6 Sangoma Technologies -‐ © 2015
Performance Basics • CPU-‐Bound • I/O Bound
• Threads, Resource/Lock ContenHon • You cannot improve what you don’t measure 7 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • FreeSWITCH is an insanely threaded system
(the good kind of insane) 8 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • Most threads are I/O bound
• But transcoding, transraHng, tone generaHon introduce CPU-‐bound elements into the mix • FreeSWITCH core I/O model is blocking, not async 9 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • Every call leg has its own
session thread walking through a state machine, roughly, like this: • init -‐> rouHng -‐> execute app -‐> hangup -‐> reporHng -‐ > destroy 10 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • Monitoring threads per signaling stack (e.g
sofia, freetdm) • These threads are long-‐lived and perform very specific tasks (e.g process SIP signaling out of a call context, iniHal invite etc) • Event subsystem launches threads for event dispatch 11 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • Conferences duplicate your use of threads
per call leg. For each parHcipant you have 2 threads: • Session thread (handles call state and media output) • Input conference thread (launched when joining the conference, reads media from the session) 12 Sangoma Technologies -‐ © 2015
FreeSWITCH Core • Even small features might launch threads
• e.g. Semng Hmer=so? when performing a playback() launches an extra thread to consume media from the session 13 Sangoma Technologies -‐ © 2015
Performance Tweaks • Logging adds stress to the event
subsystem • Every log statement is queued as an event • Every log statement is delivered to logger modules (syslog, file, console) • Set core logging level to warning in switch.conf.xml 14 Sangoma Technologies -‐ © 2015
Performance Tweaks • Do not write debug logs to
an SSD in a loaded system. You’ll kill the SSD soon J • If you want to keep debug level, you can put logs into tmpfs and rotate o?en 15 Sangoma Technologies -‐ © 2015
Database • The naHve sqlite core database must go
to tmpfs to avoid I/O boplenecks • On tmpfs however you risk losing SIP registraHon data on a power outage or any sudden restart (e.g kernel panic) • Most other data is transient (e.g channels, sip dialogs, etc) 16 Sangoma Technologies -‐ © 2015
Database • Eventually you might need to migrate to
pgsql, mysql or some other database via odbc • Allows you to move db workload elsewhere • Beper performance for applicaHons that read the core info (channels, calls, etc) 17 Sangoma Technologies -‐ © 2015
Database • Tables such as channels, calls, tasks, sip_dialogs,
do not need to persist. You can move those tables to memory (e.g MEMORY engine on MySQL) if you don’t need fault tolerance • Remember to set auto-‐create-‐schemas=false and auto-‐clear-‐sql=false if you create the db schema on your own (see switch.conf.xml) 18 Sangoma Technologies -‐ © 2015
Database • If using MySQL: • Use the
InnoDB engine for beper concurrency in data that requires persistence (e.g SIP registraHon) • innodb_flush_log_at_trx_commit=0 • sync_binlog=0 19 Sangoma Technologies -‐ © 2015
SIP Stack • Sofia launches the following threads per
profile: • Main profile thread (runs sofia UA stack scheduling) • Worker thread (checks expired registraHons) • Stack listener thread (accepHng inbound traffic) • You can distribute your traffic among more sofia profiles for improved concurrency 20 Sangoma Technologies -‐ © 2015
Memory AllocaNon • FreeSWITCH uses memory pools •
Using modules that depend on libraries or modules not using pools can benefit from using an alternaHve memory allocator 21 Sangoma Technologies -‐ © 2015
Memory AllocaNon • tcmalloc and jemalloc are good alternaHves
• Reports on the mailing list of improvement if using mod_perl • Sangoma found very significant improvement on its SBC (based on FreeSWITCH) 22 Sangoma Technologies -‐ © 2015
Memory AllocaNon • Easy to try on your own
workload: • LD_PRELOAD=“libtcmalloc.so.x.x.x" ./freeswitch • Recommended to run mysql with either tcmalloc or jemalloc 23 Sangoma Technologies -‐ © 2015
Dialplan • Careful planning of your dialplan goes a
long way • Do not enable funcHonality you don’t need, everything has a cost • Just loading a module might be consuming precious cpu cycles 24 Sangoma Technologies -‐ © 2015
Dialplan • Common performance factors to consider (mind
the performance cost of those features): • Media relay • Tone DetecHon • Recording • Transcoding 25 Sangoma Technologies -‐ © 2015
Measurement Tools • switchy: A distributed load-‐generator •
hpps://github.com/sangoma/switchy • vmstat ploper • hpps://clusterbuffer.wordpress.com/2014/09/21/ vmstat_ploper/ 26 Sangoma Technologies -‐ © 2015
Test Server • Linux CentOS 6 (kernel 2.6.x)
• FreeSWITCH v1.4 git branch • Intel Xeon 64bit processor w/ 8 cores • Intel SSD • 16GB of RAM 27 Sangoma Technologies -‐ © 2015
Test Lab 28 Sangoma Technologies -‐ © 2015
Test FreeSWITCH Server Switchy Load Generator (FreeSWITCH) Load Generator (FreeSWITCH) ESL ESL SIP SIP
2k@50cps simple audio bridge 29 Sangoma Technologies -‐
© 2015
2k@50cps tone detecNon 30 Sangoma Technologies -‐ ©
2015
1k@50cps simple audio bridge 31 Sangoma Technologies -‐
© 2015
1k@50cps session recording 32 Sangoma Technologies -‐ ©
2015
1k@50cps transcoding PCMU/G722 33 Sangoma Technologies -‐ ©
2015
4k@80cps bypass media 34 Sangoma Technologies -‐ ©
2015
Dialplan • Use bypass media selecHvely whenever you can
• Avoid transcoding, use late-‐negoHaHon and inherit_codec=true • If you must do transcoding, you can offload to a hardware transcoder 35 Sangoma Technologies -‐ © 2015
Final Thoughts • You have to measure your own
work load • No easy answers with performance, but you have the tools to find what works for you 36 Sangoma Technologies -‐ © 2015
QUESTIONS
Contact Us • Sangoma Technologies 100 Renfrew Drive,
Suite 100 Markham, Ontario L3R 9R6 Canada • Website hpp://www.sangoma.com/ • Telephone +1 905 474 1990 x2 (for Sales) • Email sales@sangoma.com Sangoma Technologies -‐ © 2015 38
THANK YOU