Making Magento go fast

Making Magento go fast By Thijs Feryn

my name Thijs Hi, is

And I’m @ThijsFeryn on Twitter

I’m an at Evangelist

I’m at a board member

https://joind.in/talk/view/11867 Feedback

Magento is kinda slow …

But tremendously flexible

In the beginning we were somewhat clueless

But we learned a lot!

It did take some time though

Operations approach

The first attempts

First attempts ✓Install powerful machines ✓Lots of RAM & CPU
✓Loadbalancing on the webservers ✓Master/master replication on the MySQL servers ✓High memory limit ✓APC byte code caching

Looking for Magento knowledge

First Magento skills ✓Activate caching in the admin panel ✓Flat
catalogs ✓The Magento compiler ✓JS & CSS minification

The Magento compiler is a lie!

Use a real byte code cache instead

Things started to speed up But it wasn’t fantastic

First conclusions

First conclusions ✓It uses a lot of RAM & CPU
✓And a whole lot of I/O ✓The MySQL server gets hammered

We learned how to cache

By default on disk in in /var/cache

./mage-‐-‐c ./mage-‐-‐a ./mage-‐-‐4 ./mage-‐-‐7 ./mage-‐-‐e
./mage-‐-‐5 ./mage-‐-‐b ./mage-‐-‐9 ./mage-‐-‐0 ./mage-‐-‐f ./mage-‐-‐3 ./mage-‐-‐6 ./mage-‐-‐8 ./mage-‐-‐d ./mage-‐-‐1

./mage-‐-‐8/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_STORE_BE_EN_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_STORE_LU_EN_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_NL_NL_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_LU_EN_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_LU_FR_CONFIG_CACHE
./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_FR_EN_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_STORE_LU_FR_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_BE_FR_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_STORE_ADMIN_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_STORE_BE_FR_CONFIG_CACHE ./mage-‐-‐8/mage-‐-‐-‐MAGE_DB_PDO_MYSQL_DDL_sales_flat_shipment_grid_3 ./mage-‐-‐d/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_APP_E4D52B98688947405EDE639E947EE03D ./mage-‐-‐d/mage-‐-‐-‐MAGE_APP_B1FB6E8F13287C01E5C05063633DDA4C ./mage-‐-‐d/mage-‐-‐-‐MAGE_DB_PDO_MYSQL_DDL_log_url_info_1 ./mage-‐-‐d/mage-‐-‐-‐MAGE_APP_E4D52B98688947405EDE639E947EE03D ./mage-‐-‐d/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_APP_B1FB6E8F13287C01E5C05063633DDA4C ./mage-‐-‐d/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_DB_PDO_MYSQL_DDL_log_url_info_1 ./mage-‐-‐1/mage-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_currencysymbol_ ./mage-‐-‐1/mage-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_nametocurrency_ ./mage-‐-‐1/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_currencynumber_ ./mage-‐-‐1/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_currencysymbol_ ./mage-‐-‐1/mage-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_currencynumber_ ./mage-‐-‐1/mage-‐-‐-‐internal-‐metadatas-‐-‐-‐MAGE_Zend_LocaleC_nl_NL_nametocurrency_

In app/etc/local.xml <global> <cache>
<backend>apc</backend> <prefix>MAGE_</prefix> </cache> </global>

Memcached

<global> <cache> <memcached>
<servers> <server> <host><![CDATA[127.0.0.1]]></host> <port><![CDATA[11211]]></port> <persistent><![CDATA[1]]></persistent> </server> </servers> </memcached> </cache> </global> In app/etc/local.xml Multiple servers

Still there are files in /var/cache

2-level cache ✓Fast backend first (adapter) ✓Then the slow backend
(disk)

/var/cache in tmpfs mount -‐t tmpfs -‐o size=20m tmpfs /var/www/var/cache

Bigger machines

is slow

Especially the older versions

Magento doesn’t officially work on the latest versions

But we made it work on 5.4. (with a patch)
http://www.magentocommerce.com/knowledge-base/ entry/php54-patch

Without byte code caching, you’re in trouble

We forgot that byte code caches aren’t efficient on FastCGI

19280 33686 1.3 0.4 322052 110408 ? S 14:07
0:01 /usr/local/php-‐5.4/bin/php-‐cgi 18834 33696 0.0 0.0 267780 13608 ? S 14:07 0:00 /usr/local/php-‐5.4/bin/php-‐cgi 19280 33698 0.9 0.4 321808 110948 ? S 14:07 0:01 /usr/local/php-‐5.4/bin/php-‐cgi 19057 33700 0.0 0.0 268124 13416 ? S 14:07 0:00 /usr/local/php-‐5.4/bin/php-‐cgi 18767 33710 0.0 0.0 269728 14156 ? S 14:08 0:00 /usr/local/php-‐5.4/bin/php-‐cgi Separate processes, no joint cache

And we forgot that APC is slowly dying

root 26067 0.0 0.0 342892 6248 ? S 13:43 0:00
php-‐fpm: master process (/etc/php5/fpm/php-‐fpm.conf) www-‐data 26068 0.2 1.1 396952 94684 ? S 13:43 0:03 php-‐fpm: pool www www-‐data 26069 0.0 0.6 363472 56456 ? S 13:43 0:01 php-‐fpm: pool www www-‐data 26087 0.0 0.6 362708 50188 ? S 13:44 0:00 php-‐fpm: pool www Master proces that stores the byte code cache

Opcache is awesome … and a lot faster

Everything in memory!!!

We looked at HHVM

But the database is also important

innodb_buffer_pool_size = 1024M Give MySQL more RAM

Turn off query caching

innodb_flush_log_at_trx_commit

Which one will we choose?

<global> <resources> <default_setup>
<connection> <host><![CDATA[Master-‐host]]></host> <username><![CDATA[user]]></username> <password><![CDATA[pass]]></password> <dbname><![CDATA[magentodb]]></dbname> <active>1</active> </connection> </default_setup> <default_read> <connection> <use/> <host><![CDATA[Slave-‐host]]></host> <username><![CDATA[user]]></username> <password><![CDATA[pass]]></password> <dbname><![CDATA[magento]]></dbname> <type>pdo_mysql</type> <model>mysql4</model> <initStatements>SET NAMES utf8</initStatements> <active>1</active> </connection> </default_read> </resources> </global> R/W splitting

And all of a sudden, we discovered Redis!

is GREAT! ✓Built-in replication ✓Data types (hash, list, …) ✓Save
on disk ✓Session clustering ✓Cm_Cache_Backend_Redis by default in Magento 1.8 CE & 1.13 EE ✓No 2-level cache ✓Multiple databases ✓Authentication

Cm_Cache_Backend_Redis https://github.com/colinmollenhour/ Cm_Cache_Backend_Redis

<global> <cache> <backend>Cm_Cache_Backend_Redis</backend>
<backend_options> <server>127.0.0.1</server> <port>6379</port> <persistent></persistent> <database>1</database> <password></password> <force_standalone>0</force_standalone> <connect_retries>1</connect_retries> <read_timeout>10</read_timeout> <automatic_cleaning_factor>0</automatic_cleaning_factor> <compress_data>1</compress_data> <compress_tags>1</compress_tags> <compress_threshold>20480</compress_threshold> <compression_lib>gzip</compression_lib> </backend_options> </cache> </global> In app/etc/local.xml

Cm_RedisSession https://github.com/colinmollenhour/Cm_RedisSession

In app/etc/modules/Cm_RedisSession.xml <config> <modules>
<Cm_RedisSession> <active>true</active> <codePool>community</codePool> </Cm_RedisSession> </modules> </config>

<global> <session_save>db</session_save> <redis_session>
<host>127.0.0.1</host> <port>6379</port> <password></password> <timeout>2.5</timeout> <persistent></persistent> <db>1</db> <compression_threshold>2048</compression_threshold> <compression_lib>gzip</compression_lib> <log_level>1</log_level> <max_concurrency>6</max_concurrency> <break_after_frontend>5</break_after_frontend> <break_after_adminhtml>30</break_after_adminhtml> <bot_lifetime>7200</bot_lifetime> </redis_session> </global> In app/etc/local.xml

Varnish

Varnish ✓Only works on product catalog & CMS pages ✓Doesn’t
work on checkout ✓Great for static content ✓No SSL support ✓Best way to cache

Hit rate kinda sucked

Turpentine https://github.com/nexcess/magento-turpentine

Turpentine ✓Community module for Varnish in Magento ✓ESI or AJAX
for session data ✓Configurable in the admin ✓VCL is pushed via Turpentine to the Varnish over Telnet ✓Takes care of purging ✓SSL termination via Nginx

Lesti_FPC When Varnish is not an option https://github.com/GordonLesti/Lesti_Fpc

<global> <fpc> <lifetime>86400</lifetime>
<debug>false</debug> <backend>Cm_Cache_Backend_Redis</backend> <backend_options> <server>127.0.0.1</server> <port>6379</port> <persistent>cache-‐fpc</persistent> <database>1</database> <password></password> <force_standalone>1</force_standalone> <connect_retries>1</connect_retries> <lifetimelimit>86400</lifetimelimit> <read_timeout>10</read_timeout> <compress_data>1</compress_data> <compress_tags>1</compress_tags> <compress_data>gzip</compress_data> </backend_options> </fpc> </global> In app/etc/local.xml Also works with other backends

Turn your log tables into blackhole tables You’re not using
them when you have Varnish/ Lesti_FPC

ALTER TABLE `log_customer` ENGINE=BLACKHOLE; ALTER TABLE `log_quote` ENGINE=BLACKHOLE;
ALTER TABLE `log_summary` ENGINE=BLACKHOLE; ALTER TABLE `log_summary_type` ENGINE=BLACKHOLE; ALTER TABLE `log_url` ENGINE=BLACKHOLE; ALTER TABLE `log_url_info` ENGINE=BLACKHOLE; ALTER TABLE `log_visitor` ENGINE=BLACKHOLE; ALTER TABLE `log_visitor_info` ENGINE=BLACKHOLE; ALTER TABLE `log_visitor_online` ENGINE=BLACKHOLE;

Search

Full-text search on the database is stupid … and slow
…

You can take down certain Magento shops by doing excessive
search

You could use

But I prefer https://github.com/jreinke/magento-Elasticsearch

Host static files separately (~CDN) http://www.magentocommerce.com/magento-connect/onepica- imagecdn-1.html

Horizontal scalability

Loadbalanced ElasticSearch Redis Loadbalanced MySQL MySQL Loadbalanced frontend Nginx Nginx
Loadbalanced Varnish Varnish Loadbalanced backend Nginx Nginx Client Admin PHP-FPM PHP-FPM PHP-FPM PHP-FPM SSL endpoint ElasticSearch

Problem!

Cm_Cache_Backend_Redis can’t handle multiple instances

Got that covered!

Redis “clustering” ✓Standalone Redis instance per webserver ✓Twemproxy & Twemproxy
agent ✓Redis-cluster (in v3, which is not yet stable) ✓Redis Sentinel & keepalived ✓Redis Sentinel patch for Cm_Cache_Backend_Redis

Sentinel

Sentinel ✓Discovery service for Redis servers ✓Part of the Redis
core ✓Monitoring, notification & autofailover ✓Knows which are masters & slaves ✓Avoid problems when the master is down ✓Avoids problems when to master is up again ✓First connect to Sentinel, then to Redis

Loadbalanced Redis master Client Loadbalanced Sentinel Sentinel Redis slave Redis
slave

port 26379 sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-‐after-‐milliseconds mymaster 5000 sentinel parallel-‐syncs mymaster 2 sentinel client-‐reconfig-‐script mymaster ./sentinel.sh Sentinel config

$> redis-‐cli -‐p 26379 127.0.0.1:26379> sentinel master mymaster
1) "name" 2) "mymaster" 3) "ip" 4) "127.0.0.1" 5) "port" 6) "6379" 7) "runid" 8) "e3a3fbab9687857c74266b360a85317aea2a213c" 9) "flags" 10) "master" 11) "pending-‐commands" 12) "0" 13) "last-‐ping-‐sent" 14) "0" 15) "last-‐ok-‐ping-‐reply" 16) "268" 17) "last-‐ping-‐reply" Discover master

$> redis-‐cli -‐p 26379 127.0.0.1:26379> sentinel slave mymaster
1) 1) "name" 2) "127.0.0.1:6380" 3) "ip" 4) "127.0.0.1" 5) "port" 6) "6380" 7) "runid" 8) "40a20d214c30b5a63a78882ee68d51f2b8506cc0" 9) "flags" 10) "slave" 11) "pending-‐commands" 12) "0" 13) "last-‐ping-‐sent" 14) "0" 15) "last-‐ok-‐ping-‐reply" 16) "158" 17) "last-‐ping-‐reply" 18) "158" Discover slaves Supports multiple slaves

$> redis-‐cli -‐p 26379 127.0.0.1:26379> SENTINEL get-‐master-‐addr-‐by-‐name mymaster
1) "127.0.0.1" 2) "6379" Get master host & port

Keepalived

Keepalived ✓Active-passive setups ✓Manages the virtual IP ✓Master gets virtual
IP ✓Does failover ✓Uses check scripts

Keepalived vrrp_script chk_redis_master { script "/vagrant/keepalived.sh"
interval 2 } vrrp_instance redis { state MASTER interface eth1 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass thepasswordispassword } virtual_ipaddress { 10.10.10.42 } track_script { chk_redis_master } } vrrp_script chk_redis_master { script "/vagrant/keepalived.sh" interval 2 } vrrp_instance redis { state MASTER interface eth1 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass thepasswordispassword } virtual_ipaddress { 10.10.10.42 } track_script { chk_redis_master } } Master Slave

Keepalived #!/bin/bash result=$(redis-‐cli -‐p 26379 SENTINEL get-‐master-‐addr-‐by-‐name mymaster |
head -‐n1 2>&1) ip=`/sbin/ifconfig eth1 | grep 'inet addr:' | cut -‐d: -‐f2 | awk '{ print $1}'` if [ "$result" == "$ip" ] ; then echo "I'm master" exit 0 else echo "I'm slave" exit 1 fi check script

Cm_Cache_Backend_redis patch

https://github.com/ colinmollenhour/credis https://github.com/ThijsFeryn/ Cm_Cache_Backend_Redis/tree/ multiserver Multi-‐server support Library contains
my patches Work in progress Sentinel support

Replacing Credis with Predis inside Cm_Cache_Backend_redis Would make more sense

Requires merging @vjom’s https:// github.com/ vmattila/predis/ tree/feature- sentinel feature branch

Twemproxy & Twemproxy agent https://github.com/Stono/redis-twemproxy-agent https://github.com/twitter/twemproxy

Twemproxy ✓Redis & Memcached Proxy ✓Written by Twitter ✓Does sharding
✓Persistent connections ✓Disable nodes on failure https://github.com/twitter/twemproxy

Twemproxy twem1: listen: 127.0.0.1:22122 hash: fnv1a_64
hash_tag: "{}" distribution: ketama auto_eject_hosts: false timeout: 400 redis: true servers: -‐ 10.10.10.40:6379:1 master1 -‐ 10.10.10.43:6379:1 master2

Twemproxy agent ✓NodeJS daemon ✓Monitors master status via Sentinel ✓Changes
Twemproxy config on failover https://github.com/Stono/redis-twemproxy-agent

Twemproxy agent node cli.js -‐h 10.10.10.40 -‐p 26379 \
-‐f /etc/twemproxy/conf/magento.conf \ -‐c /etc/init.d/twemproxy restart \ -‐l /var/log/twemproxy/twemproxy_sentinel.log

Redis Cluster

Redis Cluster ✓In v3 ✓Sharding ✓Replication ✓Autofailover ✓Auto shard redirection
✓No need for Sentinel

port 7000 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout
5000 appendonly yes port 7001 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout 5000 appendonly yes port 7002 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout 5000 appendonly yes port 7003 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout 5000 appendonly yes port 7004 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout 5000 appendonly yes port 7005 cluster-‐enabled yes cluster-‐config-‐file nodes.conf cluster-‐node-‐timeout 5000 appendonly yes Cluster node config

$> ./redis-‐trib.rb create -‐-‐replicas 1 127.0.0.1:7000 \ 127.0.0.1:7001 127.0.0.1:7002
127.0.0.1:7003 127.0.0.1:7004 \ 127.0.0.1:7005 >>> Creating cluster Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7003: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7005: OK >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 Creating the cluster

$> redis-‐cli -‐c -‐p 7000 127.0.0.1:7000> set key value
-‐> Redirected to slot [12539] located at 127.0.0.1:7002 OK 127.0.0.1:7000> get key "value" 127.0.0.1:7000> Using the cluster $> redis-‐cli -‐c -‐p 7005 127.0.0.1:7005> get key -‐> Redirected to slot [12539] located at 127.0.0.1:7002 "value" 127.0.0.1:7005> get key "value"

Magento v2?

My Redis multi-server patch in production?

Redis v3 with built-in clustering?

Replacing Credis with Predis inside Cm_Cache_Backend_redis

HHVM as the default PHP runtime?

Percona Server as the default MySQL server?

Varnish v4 support?

Use Consul for service discovery?

And that is how me make sure you sleep at
night

But don’t ask us to write Magento code for you

We have partners for that

But we can make your Magento go fast

https://joind.in/talk/view/11867 Feedback

Making Magento go fast

Making Magento go fast

More Decks by Thijs Feryn

Other Decks in Technology

Featured

Transcript