Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling in Mind - Case study of Drupal Core

Jimmy Huang
October 17, 2014

Scaling in Mind - Case study of Drupal Core

Jimmy Huang

October 17, 2014
Tweet

More Decks by Jimmy Huang

Other Decks in Technology

Transcript

  1. Scalability in Mind 當老軟體 Drupal 遇上大架構 2014-10-18 PHPConf Jimmy Huang

    黃雋
  2. Drupal and me 媽,我在這裡 Jimmy Major Versions

  3. Why not Drupal?

  4. It’s just a CMS for damned cat not for me

    Image from: https://flic.kr/p/hv9xDs
  5. Learning Curve Image from http://www.codem0nk3y.com/2012/04/what-bugs-me-about-modx-and-why/cms-learning-curve/

  6. Slower... than my own fastest code Image from: https://flic.kr/p/9CWhYu

  7. Too may reason to say no... Not OOP No ORM

    Made by PHP Hard to make theme Hard to staging, continues deploying
  8. 沒有 愛

  9. 1. Flexibility For Drupal Beginner

  10. Drupal can be a: • Personal blog • Company official

    site • Community forum • Online commerce shopping mall • Company intranet portal • Heavy media site • Video portal • Mobile backend CMS
  11. None
  12. None
  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. CMS? Development oriented CMS • Not (only) a framework •

    config many things in UI • Abstract in data layer • Need 3-party modules
  20. Content Type Sample for phpconf 2014

  21. Sample for phpconf 2014

  22. View 1

  23. View 1

  24. View 2

  25. View 2

  26. View 3

  27. View 3

  28. Query Generator in clicks same query, different layout

  29. Modules will working together hook API design

  30. Modules will working together

  31. /** * Implementation of hook_captcha(). */ function AMAZINGMY_captcha_captcha($op, $captcha_type='') {

    switch ($op) { case 'list': return array('AMAZINGMY CAPTCHA'); case 'generate': if ($captcha_type == 'AMAZINGMY CAPTCHA') { $captcha = array(); $captcha['solution'] = 'AMAZINGMY'; $captcha['form']['captcha_response'] = array( '#type' => 'textfield', '#title' => t('Enter "Amazing"'), '#required' => TRUE, ); return $captcha; module_invoke('AMAZINGMY_captcha', 'captcha', 'generate', $captcha_type_challenge);
  32. horizontal 2. Scalability

  33. Image from https://groups.drupal.org/node/24412 scale out scale out Hosting Architecture for

    pure dynamic site
  34. Prepare to scale • Reverse proxy or Hardware load balance?

    • Where are your Sessions? • File storage? • Separated read/write query?
  35. Reverse Proxy ip from - $_SERVER[‘HTTP_X_FORWARDED_FOR‘] https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/ip_address/7

  36. Reverse Proxy Reverse Proxy Remote addr will get proxy IP

    Real ip need forward from Proxy
  37. Reverse Proxy Example setting of Nginx Location { proxy_set_header Host

    $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }
  38. Session Storage • Plugable • Centralized • Fast

  39. Session Storage before 2008, Drupal 6 save session to DB

    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/_drupal_bootstrap/6
  40. Session Storage after 2011, Drupal 7, have plugable Session config

    in core https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_bootstrap/7
  41. Session Storage after 2014, Drupal 8 include better handler from

    Symfony 2 Drupal 8 API: http://goo.gl/VVQ2Ua
  42. Session Storage PHP 5.4 also have better SessionHandler class http://php.net/manual/en/class.sessionhandler.php

  43. File Storage • After upload can other instance saw files?

  44. File Storage Drupal 6 – only 1 hook nothing to

    help scaling
  45. File Storage Drupal 7 – complete file handling api (hook_file_*)

  46. File Storage • After upload, send to AWS S3 or

    FTP? – Yes! by hook_file_copy • Before display, alter URL for CDN support? – Yes! by hook_file_url_alter • When load file, streaming by other host? – Yes! by hook_file_load
  47. File Storage function hook_file_url_alter(&$uri) { $cdn1 = 'http://cdn1.example.com'; $cdn2 =

    'http://cdn2.example.com'; $cdn_extensions = array('css', 'js'); if($this_file_extension in $cdn_extensions){ $uri = $cdn1 . '/' . $path; } else{ $uri = $cdn2 . '/' . $path; } }
  48. File Storage Third-party module - Storage API • Save to

    FTP / HTTP • Save to Database • Save to S3 • Save to Rackspace
  49. Database Scaling MongoDB? PostgreSQL?

  50. Database Scaling Drupal 6 - happy querying, tragedy scaling function

    statistics_get($nid) { if ($nid > 0) { // Retrieve an array with both totalcount and daycount. $statistics = db_fetch_array(db_query('SELECT totalcount, daycount, timestamp FROM {node_counter} WHERE nid = %d', $nid)); } return $statistics; }
  51. Database Scaling Drupal 7 – DB abstract layer $statistics =

    db_select('node_counter', 'n') ->fields('n', array( 'totalcount', 'daycount', 'timestamp')) ->condition('nid', $nid,'=') ->execute() ->fetchAssoc(); • Support another Database (not MySQL only) • Separate R/W query easily
  52. Database Scaling random slave every time DB bootstrap # default

    master (read / write query) $databases['default']['default'] = $info_array; # multiple slave (read only query) $databases['default']['slave'][] = $info_array; $databases['default']['slave'][] = $info_array;
  53. Database Scaling page specific query to slave by 1 click

  54. Database Scaling

  55. 3. why Scalability matter?

  56. 我不胖,只是腫了一點

  57. Not Fastest solution But Available solution

  58. Why a CMS designed like this? • Pro – Quick

    and easy to enter Drupal (even not Engineer) – Can stack special requirement into Drupal – When more function or more user, scale out • Cons – Not so easy (if you would like to develop with D) – Not so flexible vs framework (because it isn’t) – Definitely can scale if well planned, but always not
  59. 沒有深深愛過 怎知好與壞? 我知道你胖,但還是愛你

  60. you may interested in: • 2.4 million page views per

    day in Drupal http://sf2010.drupal.org/conference/sessions/24-million-page-views-day-6 0-m-month-one-server.html • Auto Scale Drupal setup in AWS http://www.slideshare.net/burgerboydaddy/scaling-drupal-horizontally-and- in-cloud • Drupal vs Django http://birdhouse.org/blog/2009/11/11/drupal-or-django/ • Drupal with nodejs https://www.drupal.org/project/nodejs • Drupal with Docker https://github.com/ricardoamaro/docker-drupal • Drupal with MongoDB https://www.drupal.org/project/mongodb
  61. Thank You! 每週三晚上 8:00 Hangout 網路聚 1. DrupalTaiwan.org 2. goo.gl/PxuhqQ

    3. FB/groups/drupaltaiwan/ DrupalTaiwan Facebook 社團 You can also find Drupaler here: