Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Call of Postgres: Advanced Operations. Part II.
Search
Alexey Lesovsky
July 05, 2017
Education
0
45
Call of Postgres: Advanced Operations. Part II.
Slides from my tutorial at PgDay 2017 St.Petersburg
Alexey Lesovsky
July 05, 2017
Tweet
Share
More Decks by Alexey Lesovsky
See All by Alexey Lesovsky
PostgreSQL Scaling Usecases
lesovsky
0
72
Patroni failure stories, or How to crash yout PostgreSQL cluster
lesovsky
0
340
PostgreSQL High Availability in 2019
lesovsky
0
190
Top developer's mistakes when working with PostgreSQL
lesovsky
1
110
Let's Pull the Plug On the Autovacuum (EN)
lesovsky
0
1.2k
Troubleshooting PostgreSQL for Developers (RU)
lesovsky
2
320
Troubleshooting PostgreSQL Streaming Replication
lesovsky
0
790
Call of Postgres: Advanced Operations. Part I.
lesovsky
0
48
Call of Postgres: Advanced Operations. Part III.
lesovsky
0
44
Other Decks in Education
See All in Education
Introduction - Lecture 1 - Information Visualisation (4019538FNR)
signer
PRO
0
3.5k
Data Representation - Lecture 3 - Information Visualisation (4019538FNR)
signer
PRO
1
1.7k
心房細動週間ポスター(2024年3月6日)
japanstrokeassociation
0
260
Pre-enrollment Information for UTokyo International Students
utokyoissr2360
0
4.8k
@ngrx/signals
yannickbaron
0
170
STEAM教育の枠組で行うプログラミング学習
asial_edu
0
210
Virtual and Augmented Reality - Lecture 8 - Next Generation User Interfaces (4018166FNR)
signer
PRO
0
1.2k
3 занятие. Канва бизнес-модели #ideaNN 2.02.2024.
karlov
0
200
Padlet opetuksessa
matleenalaakso
3
11k
Multimodal Interaction - Lecture 3 - Next Generation User Interfaces (4018166FNR)
signer
PRO
0
1.1k
Interactive Tabletops and Surfaces - Lecture 7 - Next Generation User Interfaces (4018166FNR)
signer
PRO
1
1.2k
Introduction - Lecture 1 - Next Generation User Interfaces (4018166FNR)
signer
PRO
0
3k
Featured
See All Featured
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
2
3.4k
jQuery: Nuts, Bolts and Bling
dougneiner
59
7.1k
The Mythical Team-Month
searls
216
42k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Become a Pro
speakerdeck
PRO
11
4.5k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
104
6.6k
Learning to Love Humans: Emotional Interface Design
aarron
267
39k
Code Review Best Practice
trishagee
55
15k
What's in a price? How to price your products and services
michaelherold
237
11k
Thoughts on Productivity
jonyablonski
58
3.8k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
221
21k
Transcript
Call of Postgres: Advanced Operations Alexey Lesovsky
[email protected]
Tuning Linux: практика 02
План 02 dataegret.com Отработка навыков настройки Linux. Отработка навыков troubleshooting
в Linux.
Отработка навыков настройки 02 dataegret.com Настройка RAID контроллера. Настройка переменных
sysctl. Виртуальная память, NUMA, Huge Pages Transparent Huge Pages. Планировщики ввода-вывода. Файловые системы. Энергосбережение.
Настройка RAID контроллера 02 dataegret.com MegaCli (StorCli): • Проверка конфигурации
контроллера, томов, дисков. • Настройка логических томов. • Журнал событий.
Настройка RAID контроллера 02 dataegret.com Проверка конфигурации контроллера, томов, дисков.
# megacli adpallinfo aall # megacli ldinfo lall aall # megacli pdlist aall # megacli adpbbucmd aall
Настройка RAID контроллера 02 dataegret.com Настройка логических томов. # megacli
ldsetprop wt l0 a0 -- set Writethrough # megacli ldsetprop direct l0 a0 -- set Direct # megacli ldsetprop nora l0 a0 -- set No ReadAhead
Настройка RAID контроллера 02 dataegret.com Журнал событий. # megacli adpeventlog
getevents severity -f filename a0 • severity: info, warning, critical, fatal
Sysctl переменные 02 dataegret.com Планировщик процессов: # vi /etc/sysctl.conf kernel.sched_autogroup_enabled
= 0 kernel.sched_migration_cost_ns = 5000000 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память: # vi /etc/sysctl.conf vm.dirty_background_bytes
= 128000000 vm.dirty_bytes = 512000000 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и Out-of-memory: # vi
/etc/sysctl.conf vm.swappiness = 1 vm.min_free_kbytes = 1000000 vm.overcommit_memory = 0 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и NUMA: # vi
/etc/sysctl.conf vm.zone_reclaim_mode = 0 kernel.numa_balancing = 0 # sysctl -p
Sysctl переменные 02 dataegret.com Виртуальная память и Huge Pages: #
vi /etc/sysctl.conf vm.nr_hugepages = 32000 # 64GB vm.nr_overcommit_hugepages = 1000 # +2GB # echo 3 > /proc/sys/vm/drop_caches # sysctl -p # grep ^Huge /proc/meminfo
NUMA 02 dataegret.com # numastat # numactl --hardware # numactl
--interleave program # cat /proc/pid/numa_maps
Transparent Huge Pages 02 dataegret.com # uname -a # echo
never > /sys/kernel/mm/transparent_hugepage/enabled # echo never > /sys/kernel/mm/transparent_hugepage/defrag # vi /etc/rc.local
Планировщики ввода-вывода 02 dataegret.com На лету: # echo noop >
/sys/block/<device>/queue/scheduler На постоянно, способ №1: # vi /etc/rc.local На постоянно, способ №2: # vi /etc/default/grub GRUB_CMDLINE_LINUX="… elevator=noop" # update-grub
Механизм BLK-MQ 02 dataegret.com # vi /etc/default/grub GRUB_CMDLINE_LINUX="… scsi_mod.use_blk_mq=1" #
update-grub # reboot # cat /sys/block/sd*/queue/scheduler (none)
Файловые системы 02 dataegret.com На лету: # mount -o remount,nobarrier
/db На постоянно: # vi /etc/fstab LABEL=pgdb /db ext4 nobarrier 0 2 # mount -o remount,nobarrier /db
Энергосбережение 02 dataegret.com Какой режим включен? # lscpu |grep MHz
# cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor Поменять на лету: # for i in $(ls -1 /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor); do echo performance > $i; done На постоянно: # update-rc.d ondemand disable – Ubuntu # cpupower frequency-set -g performance – CentOS
Troubleshooting Linux 02 dataegret.com Постановка задачи: • Определение проблемного ресурса.
План: • Выявление нагрузки на CPU. • Выявление нагрузки на Storage. • Выявление нагрузки на Network.
Troubleshooting Linux 02 dataegret.com # git clone https://github.com/lesovsky/ConferenceStuff # cd
ConferenceStuff/2017.pgday/scripts # ./01-generate-test-data.sh
Troubleshooting Linux 02 dataegret.com # scripts/02-generate-cpu-load.sh # top • Load
average. • CPU Usage: us, sy, ni, id, wa, hi, si, st.
Troubleshooting Linux 02 dataegret.com А какие еще есть тулзы? •
htop, atop, pidstat, tiptop, ... • /proc/stat интерфейс.
Troubleshooting Linux 02 dataegret.com # scripts/03-generate-io-load.sh # top • CPU
Usage: wa. # iostat -m -x 1 • IOPSes, throughput, latencies, utilization. # iotop -Poa • Disk read/write, swap, io%. # cat /proc/diskstats
Troubleshooting Linux 02 dataegret.com # scripts/04-generate-network-load.sh # nicstat -m -x
1 • Throughput, errors, utilization. # netstat -s # ss # cat /proc/net/dev
Troubleshooting Linux 02 dataegret.com Проблемы с памятью? • numastat. •
Intel Memory Latency Checker. • dmesg.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• # ipmitool sel elist • CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • # mcelog --client • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • # cat /sys/devices/system/edac/mc/mc*/*_count • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • # megacli, storcli, arcconf, tw_cli, hpacucli, etc... • Network? – ethtool.
Troubleshooting Linux 02 dataegret.com Аппаратные проблемы: • Chassis? – ipmitool.
• CPU? – mcelog. • Memory? – EDAC модули, mcelog. • Storage? – утилиты RAID-контроллеров, smartmontools, mcelog. • Network? – ethtool. • # ethtool -S eth0 |grep err
Troubleshooting Linux 02 dataegret.com
Вопросы? dataegret.com
[email protected]