Scaling in Mind - Case study of Drupal Core

Transcript

1. Scalability in Mind
   當老軟體 Drupal 遇上大架構
   2014-10-18 PHPConf
   Jimmy Huang 黃雋
   

   View Slide

2. Drupal and me
   媽，我在這裡
   Jimmy Major Versions
   

   View Slide

3. Why not Drupal?
   

   View Slide

4. It’s just a CMS
   for damned cat not for me
   Image from: https://flic.kr/p/hv9xDs
   

   View Slide

5. Learning Curve
   Image from http://www.codem0nk3y.com/2012/04/what-bugs-me-about-modx-and-why/cms-learning-curve/
   

   View Slide

6. Slower...
   than my own fastest code
   Image from: https://flic.kr/p/9CWhYu
   

   View Slide

7. Too may reason to say no...
   Not OOP
   No ORM
   Made by PHP
   Hard to make theme
   Hard to staging, continues deploying
   

   View Slide

8. 沒有
   愛
   

   View Slide

9. 1. Flexibility
   For Drupal Beginner
   

   View Slide

10. Drupal can be a:
    ● Personal blog
    ● Company official site
    ● Community forum
    ● Online commerce shopping mall
    ● Company intranet portal
    ● Heavy media site
    ● Video portal
    ● Mobile backend CMS
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. CMS?
    Development oriented CMS
    ● Not (only) a framework
    ● config many things in UI
    ● Abstract in data layer
    ● Need 3-party modules
    

    View Slide

20. Content Type
    Sample for phpconf 2014
    

    View Slide

21. Sample for phpconf 2014
    

    View Slide

22. View 1
    

    View Slide

23. View 1
    

    View Slide

24. View 2
    

    View Slide

25. View 2
    

    View Slide

26. View 3
    

    View Slide

27. View 3
    

    View Slide

28. Query Generator in clicks
    same query, different layout
    

    View Slide

29. Modules will working together
    hook API design
    

    View Slide

30. Modules will working together
    

    View Slide

31. /**
    * Implementation of hook_captcha().
    */
    function AMAZINGMY_captcha_captcha($op, $captcha_type='') {
    switch ($op) {
    case 'list':
    return array('AMAZINGMY CAPTCHA');
    case 'generate':
    if ($captcha_type == 'AMAZINGMY CAPTCHA') {
    $captcha = array();
    $captcha['solution'] = 'AMAZINGMY';
    $captcha['form']['captcha_response'] = array(
    '#type' => 'textfield',
    '#title' => t('Enter "Amazing"'),
    '#required' => TRUE,
    );
    return $captcha;
    module_invoke('AMAZINGMY_captcha', 'captcha',
    'generate', $captcha_type_challenge);
    

    View Slide

32. horizontal
    2. Scalability
    

    View Slide

33. Image from https://groups.drupal.org/node/24412
    scale out
    scale out
    Hosting Architecture
    for pure dynamic site
    

    View Slide

34. Prepare to scale
    ● Reverse proxy or Hardware load balance?
    ● Where are your Sessions?
    ● File storage?
    ● Separated read/write query?
    

    View Slide

35. Reverse Proxy
    ip from - $_SERVER[‘HTTP_X_FORWARDED_FOR‘]
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/ip_address/7
    

    View Slide

36. Reverse Proxy
    Reverse Proxy
    Remote addr will get proxy IP
    Real ip need forward from Proxy
    

    View Slide

37. Reverse Proxy
    Example setting of Nginx
    Location {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    

    View Slide

38. Session Storage
    ● Plugable
    ● Centralized
    ● Fast
    

    View Slide

39. Session Storage
    before 2008, Drupal 6 save session to DB
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/_drupal_bootstrap/6
    

    View Slide

40. Session Storage
    after 2011, Drupal 7, have plugable Session config in core
    https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_bootstrap/7
    

    View Slide

41. Session Storage
    after 2014, Drupal 8 include better handler from Symfony 2
    Drupal 8 API: http://goo.gl/VVQ2Ua
    

    View Slide

42. Session Storage
    PHP 5.4 also have better SessionHandler class
    http://php.net/manual/en/class.sessionhandler.php
    

    View Slide

43. File Storage
    ● After upload
    can other instance saw files?
    

    View Slide

44. File Storage
    Drupal 6 – only 1 hook nothing to help scaling
    

    View Slide

45. File Storage
    Drupal 7 – complete file handling api (hook_file_*)
    

    View Slide

46. File Storage
    ● After upload, send to AWS S3 or FTP?
    – Yes! by hook_file_copy
    ● Before display, alter URL for CDN support?
    – Yes! by hook_file_url_alter
    ● When load file, streaming by other host?
    – Yes! by hook_file_load
    

    View Slide

47. File Storage
    function hook_file_url_alter(&$uri) {
    $cdn1 = 'http://cdn1.example.com';
    $cdn2 = 'http://cdn2.example.com';
    $cdn_extensions = array('css', 'js');
    if($this_file_extension in $cdn_extensions){
    $uri = $cdn1 . '/' . $path;
    }
    else{
    $uri = $cdn2 . '/' . $path;
    }
    }
    

    View Slide

48. File Storage
    Third-party module - Storage API
    ● Save to FTP / HTTP
    ● Save to Database
    ● Save to S3
    ● Save to Rackspace
    

    View Slide

49. Database Scaling
    MongoDB? PostgreSQL?
    

    View Slide

50. Database Scaling
    Drupal 6 - happy querying, tragedy scaling
    function statistics_get($nid) {
    if ($nid > 0) {
    // Retrieve an array with both totalcount and
    daycount.
    $statistics = db_fetch_array(db_query('SELECT
    totalcount, daycount, timestamp FROM {node_counter}
    WHERE nid = %d', $nid));
    }
    return $statistics;
    }
    

    View Slide

51. Database Scaling
    Drupal 7 – DB abstract layer
    $statistics = db_select('node_counter', 'n')
    ->fields('n', array(
    'totalcount',
    'daycount',
    'timestamp'))
    ->condition('nid', $nid,'=')
    ->execute()
    ->fetchAssoc();
    ● Support another Database (not MySQL only)
    ● Separate R/W query easily
    

    View Slide

52. Database Scaling
    random slave every time DB bootstrap
    # default master (read / write query)
    $databases['default']['default'] = $info_array;
    # multiple slave (read only query)
    $databases['default']['slave'][] = $info_array;
    $databases['default']['slave'][] = $info_array;
    

    View Slide

53. Database Scaling
    page specific query to slave by 1 click
    

    View Slide

54. Database Scaling
    

    View Slide

55. 3. why Scalability matter?
    

    View Slide

56. 我不胖，只是腫了一點
    

    View Slide

57. Not Fastest solution
    But Available solution
    

    View Slide

58. Why a CMS designed like this?
    ● Pro
    – Quick and easy to enter Drupal (even not Engineer)
    – Can stack special requirement into Drupal
    – When more function or more user, scale out
    ● Cons
    – Not so easy (if you would like to develop with D)
    – Not so flexible vs framework (because it isn’t)
    – Definitely can scale if well planned, but always not
    

    View Slide

59. 沒有深深愛過
    怎知好與壞？
    我知道你胖，但還是愛你
    

    View Slide

60. you may interested in:
    ● 2.4 million page views per day in Drupal
    http://sf2010.drupal.org/conference/sessions/24-million-page-views-day-6
    0-m-month-one-server.html
    ● Auto Scale Drupal setup in AWS
    http://www.slideshare.net/burgerboydaddy/scaling-drupal-horizontally-and-
    in-cloud
    ● Drupal vs Django
    http://birdhouse.org/blog/2009/11/11/drupal-or-django/
    ● Drupal with nodejs
    https://www.drupal.org/project/nodejs
    ● Drupal with Docker
    https://github.com/ricardoamaro/docker-drupal
    ● Drupal with MongoDB
    https://www.drupal.org/project/mongodb
    

    View Slide

61. Thank You!
    每週三晚上 8:00 Hangout 網路聚
    1. DrupalTaiwan.org
    2. goo.gl/PxuhqQ
    3. FB/groups/drupaltaiwan/
    DrupalTaiwan Facebook 社團
    You can also find Drupaler here:
    

    View Slide