Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making Magento go fast

Ca901ddcea38854b9783781c91fc87c9?s=47 Thijs Feryn
November 12, 2014

Making Magento go fast

My Magento talk at PHP World 2014 in Washington DC

Ca901ddcea38854b9783781c91fc87c9?s=128

Thijs Feryn

November 12, 2014
Tweet

Transcript

  1. Making Magento go fast By Thijs Feryn

  2. my name Thijs Hi, is

  3. And I’m @ThijsFeryn on Twitter

  4. None
  5. I’m an at Evangelist

  6. I’m at a board member

  7. https://joind.in/talk/view/11867 Feedback

  8. None
  9. Magento is kinda slow …

  10. But tremendously flexible

  11. In the beginning we were somewhat clueless

  12. But we learned a lot!

  13. It did take some time though

  14. Operations approach

  15. The first attempts

  16. First attempts ✓Install powerful machines ✓Lots of RAM & CPU

    ✓Loadbalancing on the webservers ✓Master/master replication on the MySQL servers ✓High memory limit ✓APC byte code caching
  17. Looking for Magento knowledge

  18. First Magento skills ✓Activate caching in the admin panel ✓Flat

    catalogs ✓The Magento compiler ✓JS & CSS minification
  19. The Magento compiler is a lie!

  20. Use a real byte code cache instead

  21. Things started to speed up But it wasn’t fantastic

  22. First conclusions

  23. First conclusions ✓It uses a lot of RAM & CPU

    ✓And a whole lot of I/O ✓The MySQL server gets hammered
  24. We  learned  how  to  cache

  25. By default on disk in in /var/cache

  26. ./mage-­‐-­‐c   ./mage-­‐-­‐a   ./mage-­‐-­‐4   ./mage-­‐-­‐7   ./mage-­‐-­‐e  

    ./mage-­‐-­‐5   ./mage-­‐-­‐b   ./mage-­‐-­‐9   ./mage-­‐-­‐0   ./mage-­‐-­‐f   ./mage-­‐-­‐3   ./mage-­‐-­‐6   ./mage-­‐-­‐8   ./mage-­‐-­‐d   ./mage-­‐-­‐1
  27. ./mage-­‐-­‐8/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_STORE_BE_EN_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_STORE_LU_EN_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_NL_NL_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_LU_EN_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_LU_FR_CONFIG_CACHE  

    ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_FR_EN_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_STORE_LU_FR_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_BE_FR_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_STORE_ADMIN_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_STORE_BE_FR_CONFIG_CACHE   ./mage-­‐-­‐8/mage-­‐-­‐-­‐MAGE_DB_PDO_MYSQL_DDL_sales_flat_shipment_grid_3   ./mage-­‐-­‐d/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_APP_E4D52B98688947405EDE639E947EE03D   ./mage-­‐-­‐d/mage-­‐-­‐-­‐MAGE_APP_B1FB6E8F13287C01E5C05063633DDA4C   ./mage-­‐-­‐d/mage-­‐-­‐-­‐MAGE_DB_PDO_MYSQL_DDL_log_url_info_1   ./mage-­‐-­‐d/mage-­‐-­‐-­‐MAGE_APP_E4D52B98688947405EDE639E947EE03D   ./mage-­‐-­‐d/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_APP_B1FB6E8F13287C01E5C05063633DDA4C   ./mage-­‐-­‐d/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_DB_PDO_MYSQL_DDL_log_url_info_1   ./mage-­‐-­‐1/mage-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_currencysymbol_   ./mage-­‐-­‐1/mage-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_nametocurrency_   ./mage-­‐-­‐1/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_currencynumber_   ./mage-­‐-­‐1/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_currencysymbol_   ./mage-­‐-­‐1/mage-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_currencynumber_   ./mage-­‐-­‐1/mage-­‐-­‐-­‐internal-­‐metadatas-­‐-­‐-­‐MAGE_Zend_LocaleC_nl_NL_nametocurrency_
  28. APC

  29. In app/etc/local.xml <global>          <cache>    

                 <backend>apc</backend>                  <prefix>MAGE_</prefix>          </cache>   </global>
  30. Memcached

  31. <global>    <cache>        <memcached>      

       <servers>              <server>                <host><![CDATA[127.0.0.1]]></host>                <port><![CDATA[11211]]></port>                <persistent><![CDATA[1]]></persistent>              </server>          </servers>      </memcached>    </cache>   </global> In app/etc/local.xml Multiple servers
  32. Still there are files in /var/cache

  33. 2-level cache ✓Fast backend first (adapter) ✓Then the slow backend

    (disk)
  34. /var/cache in tmpfs mount  -­‐t  tmpfs  -­‐o  size=20m  tmpfs  /var/www/var/cache

  35. Bigger machines

  36. is slow

  37. Especially the older versions

  38. Magento doesn’t officially work on the latest versions

  39. But we made it work on 5.4. (with a patch)

    http://www.magentocommerce.com/knowledge-base/ entry/php54-patch
  40. Without byte code caching, you’re in trouble

  41. We forgot that byte code caches aren’t efficient on FastCGI

  42. 19280  33686  1.3    0.4  322052  110408  ?  S  14:07

     0:01  /usr/local/php-­‐5.4/bin/php-­‐cgi   18834  33696  0.0    0.0  267780  13608    ?  S  14:07  0:00  /usr/local/php-­‐5.4/bin/php-­‐cgi   19280  33698  0.9    0.4  321808  110948  ?  S  14:07  0:01  /usr/local/php-­‐5.4/bin/php-­‐cgi   19057  33700  0.0    0.0  268124  13416    ?  S  14:07  0:00  /usr/local/php-­‐5.4/bin/php-­‐cgi   18767  33710  0.0    0.0  269728  14156    ?  S  14:08  0:00  /usr/local/php-­‐5.4/bin/php-­‐cgi Separate processes, no joint cache
  43. And we forgot that APC is slowly dying

  44. -FPM

  45. root  26067  0.0  0.0  342892  6248  ?  S  13:43  0:00

     php-­‐fpm:  master  process  (/etc/php5/fpm/php-­‐fpm.conf)   www-­‐data  26068    0.2    1.1  396952  94684  ?  S  13:43  0:03  php-­‐fpm:  pool  www   www-­‐data  26069    0.0    0.6  363472  56456  ?  S  13:43  0:01  php-­‐fpm:  pool  www   www-­‐data  26087    0.0    0.6  362708  50188  ?  S  13:44  0:00  php-­‐fpm:  pool  www Master proces that stores the byte code cache
  46. Opcache is awesome … and a lot faster

  47. Everything in memory!!!

  48. We looked at HHVM

  49. But the database is also important

  50. innodb_buffer_pool_size  =  1024M   Give MySQL more RAM

  51. Turn off query caching

  52. innodb_flush_log_at_trx_commit

  53. Which one will we choose?

  54. <global>      <resources>          <default_setup>  

               <connection>                  <host><![CDATA[Master-­‐host]]></host>                  <username><![CDATA[user]]></username>                  <password><![CDATA[pass]]></password>                  <dbname><![CDATA[magentodb]]></dbname>                  <active>1</active>              </connection>          </default_setup>          <default_read>              <connection>                  <use/>                  <host><![CDATA[Slave-­‐host]]></host>                  <username><![CDATA[user]]></username>                  <password><![CDATA[pass]]></password>                  <dbname><![CDATA[magento]]></dbname>                  <type>pdo_mysql</type>                  <model>mysql4</model>                  <initStatements>SET  NAMES  utf8</initStatements>                  <active>1</active>              </connection>          </default_read>        </resources>   </global> R/W   splitting
  55. And all of a sudden, we discovered Redis!

  56. is GREAT! ✓Built-in replication ✓Data types (hash, list, …) ✓Save

    on disk ✓Session clustering ✓Cm_Cache_Backend_Redis by default in Magento 1.8 CE & 1.13 EE ✓No 2-level cache ✓Multiple databases ✓Authentication
  57. Cm_Cache_Backend_Redis https://github.com/colinmollenhour/ Cm_Cache_Backend_Redis

  58. <global>      <cache>          <backend>Cm_Cache_Backend_Redis</backend>  

           <backend_options>              <server>127.0.0.1</server>              <port>6379</port>              <persistent></persistent>              <database>1</database>              <password></password>              <force_standalone>0</force_standalone>              <connect_retries>1</connect_retries>              <read_timeout>10</read_timeout>              <automatic_cleaning_factor>0</automatic_cleaning_factor>              <compress_data>1</compress_data>              <compress_tags>1</compress_tags>              <compress_threshold>20480</compress_threshold>              <compression_lib>gzip</compression_lib>          </backend_options>      </cache>   </global> In app/etc/local.xml
  59. Cm_RedisSession https://github.com/colinmollenhour/Cm_RedisSession

  60. In app/etc/modules/Cm_RedisSession.xml <config>      <modules>        

     <Cm_RedisSession>              <active>true</active>              <codePool>community</codePool>          </Cm_RedisSession>      </modules>   </config>
  61. <global>      <session_save>db</session_save>      <redis_session>      

       <host>127.0.0.1</host>          <port>6379</port>          <password></password>          <timeout>2.5</timeout>          <persistent></persistent>          <db>1</db>          <compression_threshold>2048</compression_threshold>          <compression_lib>gzip</compression_lib>          <log_level>1</log_level>          <max_concurrency>6</max_concurrency>          <break_after_frontend>5</break_after_frontend>          <break_after_adminhtml>30</break_after_adminhtml>          <bot_lifetime>7200</bot_lifetime>      </redis_session>   </global> In app/etc/local.xml
  62. Varnish

  63. Varnish ✓Only works on product catalog & CMS pages ✓Doesn’t

    work on checkout ✓Great for static content ✓No SSL support ✓Best way to cache
  64. Hit rate kinda sucked

  65. Turpentine https://github.com/nexcess/magento-turpentine

  66. Turpentine ✓Community module for Varnish in Magento ✓ESI or AJAX

    for session data ✓Configurable in the admin ✓VCL is pushed via Turpentine to the Varnish over Telnet ✓Takes care of purging ✓SSL termination via Nginx
  67. Lesti_FPC When Varnish is not an option https://github.com/GordonLesti/Lesti_Fpc

  68. <global>      <fpc>          <lifetime>86400</lifetime>  

      <debug>false</debug>          <backend>Cm_Cache_Backend_Redis</backend>          <backend_options>              <server>127.0.0.1</server>              <port>6379</port>              <persistent>cache-­‐fpc</persistent>              <database>1</database>              <password></password>              <force_standalone>1</force_standalone>              <connect_retries>1</connect_retries>              <lifetimelimit>86400</lifetimelimit>              <read_timeout>10</read_timeout>              <compress_data>1</compress_data>              <compress_tags>1</compress_tags>              <compress_data>gzip</compress_data>          </backend_options>      </fpc>   </global> In app/etc/local.xml Also  works   with  other   backends
  69. Turn your log tables into blackhole tables You’re not using

    them when you have Varnish/ Lesti_FPC
  70. ALTER  TABLE  `log_customer`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_quote`  ENGINE=BLACKHOLE;  

    ALTER  TABLE  `log_summary`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_summary_type`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_url`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_url_info`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_visitor`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_visitor_info`  ENGINE=BLACKHOLE;   ALTER  TABLE  `log_visitor_online`  ENGINE=BLACKHOLE;
  71. Search

  72. Full-text search on the database is stupid … and slow

  73. You can take down certain Magento shops by doing excessive

    search
  74. You could use

  75. But I prefer https://github.com/jreinke/magento-Elasticsearch

  76. Host static files separately (~CDN) http://www.magentocommerce.com/magento-connect/onepica- imagecdn-1.html

  77. Horizontal scalability

  78. Loadbalanced ElasticSearch Redis Loadbalanced MySQL MySQL Loadbalanced frontend Nginx Nginx

    Loadbalanced Varnish Varnish Loadbalanced backend Nginx Nginx Client Admin PHP-FPM PHP-FPM PHP-FPM PHP-FPM SSL endpoint ElasticSearch
  79. Problem!

  80. Cm_Cache_Backend_Redis can’t handle multiple instances

  81. Got that covered!

  82. Redis “clustering” ✓Standalone Redis instance per webserver ✓Twemproxy & Twemproxy

    agent ✓Redis-cluster (in v3, which is not yet stable) ✓Redis Sentinel & keepalived ✓Redis Sentinel patch for Cm_Cache_Backend_Redis
  83. Sentinel

  84. Sentinel ✓Discovery service for Redis servers ✓Part of the Redis

    core ✓Monitoring, notification & autofailover ✓Knows which are masters & slaves ✓Avoid problems when the master is down ✓Avoids problems when to master is up again ✓First connect to Sentinel, then to Redis
  85. Loadbalanced Redis master Client Loadbalanced Sentinel Sentinel Redis slave Redis

    slave
  86. port  26379   sentinel  monitor  mymaster  127.0.0.1  6379  2  

    sentinel  down-­‐after-­‐milliseconds  mymaster  5000   sentinel  parallel-­‐syncs  mymaster  2   sentinel  client-­‐reconfig-­‐script  mymaster  ./sentinel.sh Sentinel config
  87. $>  redis-­‐cli  -­‐p  26379   127.0.0.1:26379>  sentinel  master  mymaster  

     1)  "name"    2)  "mymaster"    3)  "ip"    4)  "127.0.0.1"    5)  "port"    6)  "6379"    7)  "runid"    8)  "e3a3fbab9687857c74266b360a85317aea2a213c"    9)  "flags"   10)  "master"   11)  "pending-­‐commands"   12)  "0"   13)  "last-­‐ping-­‐sent"   14)  "0"   15)  "last-­‐ok-­‐ping-­‐reply"   16)  "268"   17)  "last-­‐ping-­‐reply"   Discover master
  88. $>  redis-­‐cli  -­‐p  26379   127.0.0.1:26379>  sentinel  slave  mymaster  

    1)    1)  "name"          2)  "127.0.0.1:6380"          3)  "ip"          4)  "127.0.0.1"          5)  "port"          6)  "6380"          7)  "runid"          8)  "40a20d214c30b5a63a78882ee68d51f2b8506cc0"          9)  "flags"        10)  "slave"        11)  "pending-­‐commands"        12)  "0"        13)  "last-­‐ping-­‐sent"        14)  "0"        15)  "last-­‐ok-­‐ping-­‐reply"        16)  "158"        17)  "last-­‐ping-­‐reply"        18)  "158"   Discover slaves Supports multiple slaves
  89. $>  redis-­‐cli  -­‐p  26379   127.0.0.1:26379>  SENTINEL  get-­‐master-­‐addr-­‐by-­‐name  mymaster  

    1)  "127.0.0.1"   2)  "6379" Get master host & port
  90. Keepalived

  91. Keepalived ✓Active-passive setups ✓Manages the virtual IP ✓Master gets virtual

    IP ✓Does failover ✓Uses check scripts
  92. Keepalived vrrp_script  chk_redis_master  {          script  "/vagrant/keepalived.sh"

             interval  2   }   vrrp_instance  redis  {          state  MASTER          interface  eth1          virtual_router_id  51          priority  150          advert_int  1          authentication  {                  auth_type  PASS                  auth_pass  thepasswordispassword          }          virtual_ipaddress  {                  10.10.10.42          }          track_script  {                  chk_redis_master          }           } vrrp_script  chk_redis_master  {          script  "/vagrant/keepalived.sh"          interval  2   }   vrrp_instance  redis  {          state  MASTER          interface  eth1          virtual_router_id  51          priority  100          advert_int  1          authentication  {                  auth_type  PASS                  auth_pass  thepasswordispassword          }          virtual_ipaddress  {                  10.10.10.42          }          track_script  {                  chk_redis_master          }           } Master Slave
  93. Keepalived #!/bin/bash   result=$(redis-­‐cli  -­‐p  26379  SENTINEL  get-­‐master-­‐addr-­‐by-­‐name  mymaster  |

     head  -­‐n1  2>&1)   ip=`/sbin/ifconfig  eth1  |  grep  'inet  addr:'  |  cut  -­‐d:  -­‐f2  |  awk  '{  print  $1}'`   if  [  "$result"  ==  "$ip"  ]  ;   then      echo  "I'm  master"      exit  0   else      echo  "I'm  slave"      exit  1   fi check script
  94. Cm_Cache_Backend_redis patch

  95. https://github.com/ colinmollenhour/credis https://github.com/ThijsFeryn/ Cm_Cache_Backend_Redis/tree/ multiserver Multi-­‐server  support Library  contains  

    my  patches Work  in   progress Sentinel   support
  96. Replacing Credis with Predis inside Cm_Cache_Backend_redis Would make more sense

  97. Requires merging @vjom’s https:// github.com/ vmattila/predis/ tree/feature- sentinel feature branch

  98. Twemproxy & Twemproxy agent https://github.com/Stono/redis-twemproxy-agent https://github.com/twitter/twemproxy

  99. Twemproxy ✓Redis & Memcached Proxy ✓Written by Twitter ✓Does sharding

    ✓Persistent connections ✓Disable nodes on failure https://github.com/twitter/twemproxy
  100. Twemproxy twem1:      listen:  127.0.0.1:22122      hash:  fnv1a_64

         hash_tag:  "{}"      distribution:  ketama      auto_eject_hosts:  false      timeout:  400      redis:  true      servers:        -­‐  10.10.10.40:6379:1  master1        -­‐  10.10.10.43:6379:1  master2
  101. Twemproxy agent ✓NodeJS daemon ✓Monitors master status via Sentinel ✓Changes

    Twemproxy config on failover https://github.com/Stono/redis-twemproxy-agent
  102. Twemproxy agent node  cli.js  -­‐h  10.10.10.40  -­‐p  26379  \  

    -­‐f  /etc/twemproxy/conf/magento.conf  \   -­‐c  /etc/init.d/twemproxy  restart  \   -­‐l  /var/log/twemproxy/twemproxy_sentinel.log
  103. Redis Cluster

  104. Redis Cluster ✓In v3 ✓Sharding ✓Replication ✓Autofailover ✓Auto shard redirection

    ✓No need for Sentinel
  105. port  7000   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout

     5000   appendonly  yes port  7001   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout  5000   appendonly  yes port  7002   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout  5000   appendonly  yes port  7003   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout  5000   appendonly  yes port  7004   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout  5000   appendonly  yes port  7005   cluster-­‐enabled  yes   cluster-­‐config-­‐file  nodes.conf   cluster-­‐node-­‐timeout  5000   appendonly  yes Cluster node config
  106. $>  ./redis-­‐trib.rb  create  -­‐-­‐replicas  1  127.0.0.1:7000  \   127.0.0.1:7001  127.0.0.1:7002

     127.0.0.1:7003  127.0.0.1:7004  \   127.0.0.1:7005   >>>  Creating  cluster   Connecting  to  node  127.0.0.1:7000:  OK   Connecting  to  node  127.0.0.1:7001:  OK   Connecting  to  node  127.0.0.1:7002:  OK   Connecting  to  node  127.0.0.1:7003:  OK   Connecting  to  node  127.0.0.1:7004:  OK   Connecting  to  node  127.0.0.1:7005:  OK   >>>  Performing  hash  slots  allocation  on  6  nodes...   Using  3  masters:   127.0.0.1:7000   127.0.0.1:7001   127.0.0.1:7002   Creating the cluster
  107. $>  redis-­‐cli  -­‐c  -­‐p  7000   127.0.0.1:7000>  set  key  value

      -­‐>  Redirected  to  slot  [12539]  located  at   127.0.0.1:7002   OK   127.0.0.1:7000>  get  key   "value"   127.0.0.1:7000> Using the cluster $>  redis-­‐cli  -­‐c  -­‐p  7005   127.0.0.1:7005>  get  key   -­‐>  Redirected  to  slot  [12539]  located  at   127.0.0.1:7002   "value"   127.0.0.1:7005>  get  key   "value"
  108. None
  109. Magento v2?

  110. My Redis multi-server patch in production?

  111. Redis v3 with built-in clustering?

  112. Replacing Credis with Predis inside Cm_Cache_Backend_redis

  113. HHVM as the default PHP runtime?

  114. Percona Server as the default MySQL server?

  115. Varnish v4 support?

  116. Use Consul for service discovery?

  117. And that is how me make sure you sleep at

    night
  118. But don’t ask us to write Magento code for you

  119. We have partners for that

  120. But we can make your Magento go fast

  121. None
  122. https://joind.in/talk/view/11867 Feedback