Scaling in Mind - Case study of Drupal Core

Slide 1

Slide 1 text

Scalability in Mind 當老軟體 Drupal 遇上大架構 2014-10-18 PHPConf Jimmy Huang 黃雋

Slide 2

Slide 2 text

Drupal and me 媽，我在這裡 Jimmy Major Versions

Slide 3

Slide 3 text

Why not Drupal?

Slide 4

Slide 4 text

It’s just a CMS for damned cat not for me Image from: https://flic.kr/p/hv9xDs

Slide 5

Slide 5 text

Learning Curve Image from http://www.codem0nk3y.com/2012/04/what-bugs-me-about-modx-and-why/cms-learning-curve/

Slide 6

Slide 6 text

Slower... than my own fastest code Image from: https://flic.kr/p/9CWhYu

Slide 7

Slide 7 text

Too may reason to say no... Not OOP No ORM Made by PHP Hard to make theme Hard to staging, continues deploying

Slide 8

Slide 8 text

沒有愛

Slide 9

Slide 9 text

1. Flexibility For Drupal Beginner

Slide 10

Slide 10 text

Drupal can be a: ● Personal blog ● Company official site ● Community forum ● Online commerce shopping mall ● Company intranet portal ● Heavy media site ● Video portal ● Mobile backend CMS

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

CMS? Development oriented CMS ● Not (only) a framework ● config many things in UI ● Abstract in data layer ● Need 3-party modules

Slide 20

Slide 20 text

Content Type Sample for phpconf 2014

Slide 21

Slide 21 text

Sample for phpconf 2014

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Query Generator in clicks same query, different layout

Slide 29

Slide 29 text

Modules will working together hook API design

Slide 30

Slide 30 text

Modules will working together

Slide 31

Slide 31 text

/** * Implementation of hook_captcha(). */ function AMAZINGMY_captcha_captcha($op, $captcha_type='') { switch ($op) { case 'list': return array('AMAZINGMY CAPTCHA'); case 'generate': if ($captcha_type == 'AMAZINGMY CAPTCHA') { $captcha = array(); $captcha['solution'] = 'AMAZINGMY'; $captcha['form']['captcha_response'] = array( '#type' => 'textfield', '#title' => t('Enter "Amazing"'), '#required' => TRUE, ); return $captcha; module_invoke('AMAZINGMY_captcha', 'captcha', 'generate', $captcha_type_challenge);

Slide 32

Slide 32 text

horizontal 2. Scalability

Slide 33

Slide 33 text

Image from https://groups.drupal.org/node/24412 scale out scale out Hosting Architecture for pure dynamic site

Slide 34

Slide 34 text

Prepare to scale ● Reverse proxy or Hardware load balance? ● Where are your Sessions? ● File storage? ● Separated read/write query?

Slide 35

Slide 35 text

Reverse Proxy ip from - $_SERVER[‘HTTP_X_FORWARDED_FOR‘] https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/ip_address/7

Slide 36

Slide 36 text

Reverse Proxy Reverse Proxy Remote addr will get proxy IP Real ip need forward from Proxy

Slide 37

Slide 37 text

Reverse Proxy Example setting of Nginx Location { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }

Slide 38

Slide 38 text

Session Storage ● Plugable ● Centralized ● Fast

Slide 39

Slide 39 text

Session Storage before 2008, Drupal 6 save session to DB https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/_drupal_bootstrap/6

Slide 40

Slide 40 text

Session Storage after 2011, Drupal 7, have plugable Session config in core https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_bootstrap/7

Slide 41

Slide 41 text

Session Storage after 2014, Drupal 8 include better handler from Symfony 2 Drupal 8 API: http://goo.gl/VVQ2Ua

Slide 42

Slide 42 text

Session Storage PHP 5.4 also have better SessionHandler class http://php.net/manual/en/class.sessionhandler.php

Slide 43

Slide 43 text

File Storage ● After upload can other instance saw files?

Slide 44

Slide 44 text

File Storage Drupal 6 – only 1 hook nothing to help scaling

Slide 45

Slide 45 text

File Storage Drupal 7 – complete file handling api (hook_file_*)

Slide 46

Slide 46 text

File Storage ● After upload, send to AWS S3 or FTP? – Yes! by hook_file_copy ● Before display, alter URL for CDN support? – Yes! by hook_file_url_alter ● When load file, streaming by other host? – Yes! by hook_file_load

Slide 47

Slide 47 text

File Storage function hook_file_url_alter(&$uri) { $cdn1 = 'http://cdn1.example.com'; $cdn2 = 'http://cdn2.example.com'; $cdn_extensions = array('css', 'js'); if($this_file_extension in $cdn_extensions){ $uri = $cdn1 . '/' . $path; } else{ $uri = $cdn2 . '/' . $path; } }

Slide 48

Slide 48 text

File Storage Third-party module - Storage API ● Save to FTP / HTTP ● Save to Database ● Save to S3 ● Save to Rackspace

Slide 49

Slide 49 text

Database Scaling MongoDB? PostgreSQL?

Slide 50

Slide 50 text

Database Scaling Drupal 6 - happy querying, tragedy scaling function statistics_get($nid) { if ($nid > 0) { // Retrieve an array with both totalcount and daycount. $statistics = db_fetch_array(db_query('SELECT totalcount, daycount, timestamp FROM {node_counter} WHERE nid = %d', $nid)); } return $statistics; }

Slide 51

Slide 51 text

Database Scaling Drupal 7 – DB abstract layer $statistics = db_select('node_counter', 'n') ->fields('n', array( 'totalcount', 'daycount', 'timestamp')) ->condition('nid', $nid,'=') ->execute() ->fetchAssoc(); ● Support another Database (not MySQL only) ● Separate R/W query easily

Slide 52

Slide 52 text

Database Scaling random slave every time DB bootstrap # default master (read / write query) $databases['default']['default'] = $info_array; # multiple slave (read only query) $databases['default']['slave'][] = $info_array; $databases['default']['slave'][] = $info_array;

Slide 53

Slide 53 text

Database Scaling page specific query to slave by 1 click

Slide 54

Slide 54 text

Database Scaling

Slide 55

Slide 55 text

3. why Scalability matter?

Slide 56

Slide 56 text

我不胖，只是腫了一點

Slide 57

Slide 57 text

Not Fastest solution But Available solution

Slide 58

Slide 58 text

Why a CMS designed like this? ● Pro – Quick and easy to enter Drupal (even not Engineer) – Can stack special requirement into Drupal – When more function or more user, scale out ● Cons – Not so easy (if you would like to develop with D) – Not so flexible vs framework (because it isn’t) – Definitely can scale if well planned, but always not

Slide 59

Slide 59 text

沒有深深愛過怎知好與壞？我知道你胖，但還是愛你

Slide 60

Slide 60 text

you may interested in: ● 2.4 million page views per day in Drupal http://sf2010.drupal.org/conference/sessions/24-million-page-views-day-6 0-m-month-one-server.html ● Auto Scale Drupal setup in AWS http://www.slideshare.net/burgerboydaddy/scaling-drupal-horizontally-and- in-cloud ● Drupal vs Django http://birdhouse.org/blog/2009/11/11/drupal-or-django/ ● Drupal with nodejs https://www.drupal.org/project/nodejs ● Drupal with Docker https://github.com/ricardoamaro/docker-drupal ● Drupal with MongoDB https://www.drupal.org/project/mongodb

Slide 61

Slide 61 text

Thank You! 每週三晚上 8:00 Hangout 網路聚 1. DrupalTaiwan.org 2. goo.gl/PxuhqQ 3. FB/groups/drupaltaiwan/ DrupalTaiwan Facebook 社團 You can also find Drupaler here: