Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Call of Postgres: Advanced Operations. Part II.
Search
Alexey Lesovsky
July 05, 2017
Education
0
74
Call of Postgres: Advanced Operations. Part II.
Slides from my tutorial at PgDay 2017 St.Petersburg
Alexey Lesovsky
July 05, 2017
Tweet
Share
More Decks by Alexey Lesovsky
See All by Alexey Lesovsky
PostgreSQL Scaling Usecases
lesovsky
0
97
Patroni failure stories, or How to crash yout PostgreSQL cluster
lesovsky
0
420
PostgreSQL High Availability in 2019
lesovsky
0
350
Top developer's mistakes when working with PostgreSQL
lesovsky
1
140
Let's Pull the Plug On the Autovacuum (EN)
lesovsky
0
1.2k
Troubleshooting PostgreSQL for Developers (RU)
lesovsky
2
350
Troubleshooting PostgreSQL Streaming Replication
lesovsky
0
950
Call of Postgres: Advanced Operations. Part I.
lesovsky
0
64
Call of Postgres: Advanced Operations. Part III.
lesovsky
0
62
Other Decks in Education
See All in Education
OpenRobomaster 中国のロボットコンテスト 日本連携の可能性
takasumasakazu
0
430
社外コミュニティと「学び」を考える
alchemy1115
2
160
Webリテラシー基礎
takenawa
0
3.4k
小さなチャレンジが生んだチームの大きな変化 -私のふりかえり探求の原点
callas1900
0
520
実務プログラム
takenawa
0
3.3k
予習動画
takenawa
0
3.6k
i-GIP 2025 中高生のみなさんへ資料
202200
0
470
OpenSourceSummitJapanを運営してみた話
kujiraitakahiro
0
660
View Manipulation and Reduction - Lecture 9 - Information Visualisation (4019538FNR)
signer
PRO
1
2k
Design Guidelines and Principles - Lecture 7 - Information Visualisation (4019538FNR)
signer
PRO
0
2.4k
Data Physicalisation - Lecture 9 - Next Generation User Interfaces (4018166FNR)
signer
PRO
0
430
Tangible, Embedded and Embodied Interaction - Lecture 7 - Next Generation User Interfaces (4018166FNR)
signer
PRO
0
1.7k
Featured
See All Featured
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Agile that works and the tools we love
rasmusluckow
329
21k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
43
2.4k
The Language of Interfaces
destraynor
158
25k
Practical Orchestrator
shlominoach
188
11k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.2k
RailsConf 2023
tenderlove
30
1.1k
Bash Introduction
62gerente
614
210k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.1k
Transcript
Call of Postgres: Advanced Operations Alexey Lesovsky
[email protected]
Tuning Linux: практика 02
План 02 dataegret.com Отработка навыков настройки Linux. Отработка навыков troubleshooting
в Linux.
Отработка навыков настройки 02 dataegret.com Настройка RAID контроллера. Настройка переменных
sysctl. Виртуальная память, NUMA, Huge Pages Transparent Huge Pages. Планировщики ввода-вывода. Файловые системы. Энергосбережение.
Настройка RAID контроллера 02 dataegret.com MegaCli (StorCli): • Проверка конфигурации
контроллера, томов, дисков. • Настройка логических томов. • Журнал событий.
Настройка RAID контроллера 02 dataegret.com Проверка конфигурации контроллера, томов, дисков.
# megacli adpallinfo aall # megacli ldinfo lall aall # megacli pdlist aall # megacli adpbbucmd aall
Настройка RAID контроллера 02 dataegret.com Настройка логических томов. # megacli
ldsetprop wt l0 a0 -- set Writethrough # megacli ldsetprop direct l0 a0 -- set Direct # megacli ldsetprop nora l0 a0 -- set No ReadAhead
Настройка RAID контроллера 02 dataegret.com Журнал событий. # megacli adpeventlog
getevents severity -f filename a0 • severity: info, warning, critical, fatal
Sysctl переменные 02 dataegret.com Планировщик процессов: # vi /etc/sysctl.conf kernel.sched_autogroup_enabled
= 0 kernel.sched_migration_cost_ns = 5000000 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память: # vi /etc/sysctl.conf vm.dirty_background_bytes
= 128000000 vm.dirty_bytes = 512000000 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и Out-of-memory: # vi
/etc/sysctl.conf vm.swappiness = 1 vm.min_free_kbytes = 1000000 vm.overcommit_memory = 0 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и NUMA: # vi
/etc/sysctl.conf vm.zone_reclaim_mode = 0 kernel.numa_balancing = 0 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и Huge Pages: #
vi /etc/sysctl.conf vm.nr_hugepages = 32000 # 64GB vm.nr_overcommit_hugepages = 1000 # +2GB # echo 3 > /proc/sys/vm/drop_caches # sysctl -p # grep ^Huge /proc/meminfo
NUMA 02 dataegret.com # numastat # numactl --hardware # numactl
--interleave program # cat /proc/pid/numa_maps
Transparent Huge Pages 02 dataegret.com # uname -a # echo
never > /sys/kernel/mm/transparent_hugepage/enabled # echo never > /sys/kernel/mm/transparent_hugepage/defrag # vi /etc/rc.local
Планировщики ввода-вывода 02 dataegret.com На лету: # echo noop >
/sys/block/<device>/queue/scheduler На постоянно, способ №1: # vi /etc/rc.local На постоянно, способ №2: # vi /etc/default/grub GRUB_CMDLINE_LINUX="… elevator=noop" # update-grub
Механизм BLK-MQ 02 dataegret.com # vi /etc/default/grub GRUB_CMDLINE_LINUX="… scsi_mod.use_blk_mq=1" #
update-grub # reboot # cat /sys/block/sd*/queue/scheduler (none)
Файловые системы 02 dataegret.com На лету: # mount -o remount,nobarrier
/db На постоянно: # vi /etc/fstab LABEL=pgdb /db ext4 nobarrier 0 2 # mount -o remount,nobarrier /db
Энергосбережение 02 dataegret.com Какой режим включен? # lscpu |grep MHz
# cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor Поменять на лету: # for i in $(ls -1 /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor); do echo performance > $i; done На постоянно: # update-rc.d ondemand disable – Ubuntu # cpupower frequency-set -g performance – CentOS
Troubleshooting Linux 02 dataegret.com Постановка задачи: • Определение проблемного ресурса.
План: • Выявление нагрузки на CPU. • Выявление нагрузки на Storage. • Выявление нагрузки на Network.
Troubleshooting Linux 02 dataegret.com # git clone https://github.com/lesovsky/ConferenceStuff # cd
ConferenceStuff/2017.pgday/scripts # ./01-generate-test-data.sh
Troubleshooting Linux 02 dataegret.com # scripts/02-generate-cpu-load.sh # top • Load
average. • CPU Usage: us, sy, ni, id, wa, hi, si, st.
Troubleshooting Linux 02 dataegret.com А какие еще есть тулзы? •
htop, atop, pidstat, tiptop, ... • /proc/stat интерфейс.
Troubleshooting Linux 02 dataegret.com # scripts/03-generate-io-load.sh # top • CPU
Usage: wa. # iostat -m -x 1 • IOPSes, throughput, latencies, utilization. # iotop -Poa • Disk read/write, swap, io%. # cat /proc/diskstats
Troubleshooting Linux 02 dataegret.com # scripts/04-generate-network-load.sh # nicstat -m -x
1 • Throughput, errors, utilization. # netstat -s # ss # cat /proc/net/dev
Troubleshooting Linux 02 dataegret.com Проблемы с памятью? • numastat. •
Intel Memory Latency Checker. • dmesg.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• # ipmitool sel elist • CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • # mcelog --client • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • # cat /sys/devices/system/edac/mc/mc*/*_count • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • # megacli, storcli, arcconf, tw_cli, hpacucli, etc... • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool. • # ethtool -S eth0 |grep err
Troubleshooting Linux 02 dataegret.com
Вопросы? dataegret.com
[email protected]