Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling in Mind - Case study of Drupal Core

Jimmy Huang
October 17, 2014

Scaling in Mind - Case study of Drupal Core

Jimmy Huang

October 17, 2014
Tweet

More Decks by Jimmy Huang

Other Decks in Technology

Transcript

  1. Scalability in Mind
    當老軟體 Drupal 遇上大架構
    2014-10-18 PHPConf
    Jimmy Huang 黃雋

    View Slide

  2. Drupal and me
    媽,我在這裡
    Jimmy Major Versions

    View Slide

  3. Why not Drupal?

    View Slide

  4. It’s just a CMS
    for damned cat not for me
    Image from: https://flic.kr/p/hv9xDs

    View Slide

  5. Learning Curve
    Image from http://www.codem0nk3y.com/2012/04/what-bugs-me-about-modx-and-why/cms-learning-curve/

    View Slide

  6. Slower...
    than my own fastest code
    Image from: https://flic.kr/p/9CWhYu

    View Slide

  7. Too may reason to say no...
    Not OOP
    No ORM
    Made by PHP
    Hard to make theme
    Hard to staging, continues deploying

    View Slide

  8. 沒有

    View Slide

  9. 1. Flexibility
    For Drupal Beginner

    View Slide

  10. Drupal can be a:
    ● Personal blog
    ● Company official site
    ● Community forum
    ● Online commerce shopping mall
    ● Company intranet portal
    ● Heavy media site
    ● Video portal
    ● Mobile backend CMS

    View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. CMS?
    Development oriented CMS
    ● Not (only) a framework
    ● config many things in UI
    ● Abstract in data layer
    ● Need 3-party modules

    View Slide

  20. Content Type
    Sample for phpconf 2014

    View Slide

  21. Sample for phpconf 2014

    View Slide

  22. View 1

    View Slide

  23. View 1

    View Slide

  24. View 2

    View Slide

  25. View 2

    View Slide

  26. View 3

    View Slide

  27. View 3

    View Slide

  28. Query Generator in clicks
    same query, different layout

    View Slide

  29. Modules will working together
    hook API design

    View Slide

  30. Modules will working together

    View Slide

  31. /**
    * Implementation of hook_captcha().
    */
    function AMAZINGMY_captcha_captcha($op, $captcha_type='') {
    switch ($op) {
    case 'list':
    return array('AMAZINGMY CAPTCHA');
    case 'generate':
    if ($captcha_type == 'AMAZINGMY CAPTCHA') {
    $captcha = array();
    $captcha['solution'] = 'AMAZINGMY';
    $captcha['form']['captcha_response'] = array(
    '#type' => 'textfield',
    '#title' => t('Enter "Amazing"'),
    '#required' => TRUE,
    );
    return $captcha;
    module_invoke('AMAZINGMY_captcha', 'captcha',
    'generate', $captcha_type_challenge);

    View Slide

  32. horizontal
    2. Scalability

    View Slide

  33. Image from https://groups.drupal.org/node/24412
    scale out
    scale out
    Hosting Architecture
    for pure dynamic site

    View Slide

  34. Prepare to scale
    ● Reverse proxy or Hardware load balance?
    ● Where are your Sessions?
    ● File storage?
    ● Separated read/write query?

    View Slide

  35. Reverse Proxy
    ip from - $_SERVER[‘HTTP_X_FORWARDED_FOR‘]
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/ip_address/7

    View Slide

  36. Reverse Proxy
    Reverse Proxy
    Remote addr will get proxy IP
    Real ip need forward from Proxy

    View Slide

  37. Reverse Proxy
    Example setting of Nginx
    Location {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    View Slide

  38. Session Storage
    ● Plugable
    ● Centralized
    ● Fast

    View Slide

  39. Session Storage
    before 2008, Drupal 6 save session to DB
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/_drupal_bootstrap/6

    View Slide

  40. Session Storage
    after 2011, Drupal 7, have plugable Session config in core
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_bootstrap/7

    View Slide

  41. Session Storage
    after 2014, Drupal 8 include better handler from Symfony 2
    Drupal 8 API: http://goo.gl/VVQ2Ua

    View Slide

  42. Session Storage
    PHP 5.4 also have better SessionHandler class
    http://php.net/manual/en/class.sessionhandler.php

    View Slide

  43. File Storage
    ● After upload
    can other instance saw files?

    View Slide

  44. File Storage
    Drupal 6 – only 1 hook nothing to help scaling

    View Slide

  45. File Storage
    Drupal 7 – complete file handling api (hook_file_*)

    View Slide

  46. File Storage
    ● After upload, send to AWS S3 or FTP?
    – Yes! by hook_file_copy
    ● Before display, alter URL for CDN support?
    – Yes! by hook_file_url_alter
    ● When load file, streaming by other host?
    – Yes! by hook_file_load

    View Slide

  47. File Storage
    function hook_file_url_alter(&$uri) {
    $cdn1 = 'http://cdn1.example.com';
    $cdn2 = 'http://cdn2.example.com';
    $cdn_extensions = array('css', 'js');
    if($this_file_extension in $cdn_extensions){
    $uri = $cdn1 . '/' . $path;
    }
    else{
    $uri = $cdn2 . '/' . $path;
    }
    }

    View Slide

  48. File Storage
    Third-party module - Storage API
    ● Save to FTP / HTTP
    ● Save to Database
    ● Save to S3
    ● Save to Rackspace

    View Slide

  49. Database Scaling
    MongoDB? PostgreSQL?

    View Slide

  50. Database Scaling
    Drupal 6 - happy querying, tragedy scaling
    function statistics_get($nid) {
    if ($nid > 0) {
    // Retrieve an array with both totalcount and
    daycount.
    $statistics = db_fetch_array(db_query('SELECT
    totalcount, daycount, timestamp FROM {node_counter}
    WHERE nid = %d', $nid));
    }
    return $statistics;
    }

    View Slide

  51. Database Scaling
    Drupal 7 – DB abstract layer
    $statistics = db_select('node_counter', 'n')
    ->fields('n', array(
    'totalcount',
    'daycount',
    'timestamp'))
    ->condition('nid', $nid,'=')
    ->execute()
    ->fetchAssoc();
    ● Support another Database (not MySQL only)
    ● Separate R/W query easily

    View Slide

  52. Database Scaling
    random slave every time DB bootstrap
    # default master (read / write query)
    $databases['default']['default'] = $info_array;
    # multiple slave (read only query)
    $databases['default']['slave'][] = $info_array;
    $databases['default']['slave'][] = $info_array;

    View Slide

  53. Database Scaling
    page specific query to slave by 1 click

    View Slide

  54. Database Scaling

    View Slide

  55. 3. why Scalability matter?

    View Slide

  56. 我不胖,只是腫了一點

    View Slide

  57. Not Fastest solution
    But Available solution

    View Slide

  58. Why a CMS designed like this?
    ● Pro
    – Quick and easy to enter Drupal (even not Engineer)
    – Can stack special requirement into Drupal
    – When more function or more user, scale out
    ● Cons
    – Not so easy (if you would like to develop with D)
    – Not so flexible vs framework (because it isn’t)
    – Definitely can scale if well planned, but always not

    View Slide

  59. 沒有深深愛過
    怎知好與壞?
    我知道你胖,但還是愛你

    View Slide

  60. you may interested in:
    ● 2.4 million page views per day in Drupal
    http://sf2010.drupal.org/conference/sessions/24-million-page-views-day-6
    0-m-month-one-server.html
    ● Auto Scale Drupal setup in AWS
    http://www.slideshare.net/burgerboydaddy/scaling-drupal-horizontally-and-
    in-cloud
    ● Drupal vs Django
    http://birdhouse.org/blog/2009/11/11/drupal-or-django/
    ● Drupal with nodejs
    https://www.drupal.org/project/nodejs
    ● Drupal with Docker
    https://github.com/ricardoamaro/docker-drupal
    ● Drupal with MongoDB
    https://www.drupal.org/project/mongodb

    View Slide

  61. Thank You!
    每週三晚上 8:00 Hangout 網路聚
    1. DrupalTaiwan.org
    2. goo.gl/PxuhqQ
    3. FB/groups/drupaltaiwan/
    DrupalTaiwan Facebook 社團
    You can also find Drupaler here:

    View Slide