Death is not the End

Imagine you developed your first custom e-Commerce site from scratch in 2003, and having maintained it and successfully grown it for the subsequent 14 years, your client now wants a completely redesigned front end. What do you do?

This is a story of how the state of web development in 2003 influenced its design, how to cost-effectively evolve a website through 14 years of technological change and in the process make it more maintainable.

It's a journey from Build Your Own Framework to - drum roll - Build Your Own Framework, but with a twist.

Transcript

1. DEATH IS NOT THE END Giving new life to old web applications @ninthspace chris@junctionbox.com
2. WHY AM I HERE?
3. WHY AM I HERE? ๏ Complex e-Commerce site built 2003 ๏ Went live early 2005 ๏ Further developed and maintained ๏ Now needs a new front-end
4. WHAT I'LL COVER ๏ Demo of current site ๏ Architecture of current site ๏ Why the change? ๏ Essential changes required ๏ Other architectural improvements and additions
5. DEMO OF CURRENT SITE
6. ARCHITECTURE OF CURRENT SITE Let's go back to 2003
7. STATE OF FRAMEWORKS IN 2003 ๏ Let's use a Framework ➤ FuelPHP ➤ Phalcon ➤ Laravel ➤ Zend ➤ CodeIgniter ➤ Yii ➤ Kohana ➤ CakePHP ➤ Symfony ➤ WordPress ➤ Drupal
8. STATE OF FRAMEWORKS IN 2003 ๏ Let's use a Framework ➤ FuelPHP ➤ Phalcon ➤ Laravel ➤ Zend ➤ CodeIgniter ➤ Yii ➤ Kohana ➤ CakePHP ➤ Symfony ➤ WordPress ➤ Drupal
9. SO.. I DECIDED TO BUILD A HIGHLY COMPLEX E-COMMERCE SYSTEM USING A CONTENT MANAGEMENT SYSTEM
10. SO.. I DECIDED TO BUILD A HIGHLY COMPLEX E-COMMERCE SYSTEM USING A CONTENT MANAGEMENT SYSTEM NO, NOT REALLY
11. I BUILT MY OWN
12. INSPIRED BY SUN MICROSYSTEMS BOOK: DESIGNING ENTERPRISE APPLICATIONS WITH THE J2EE PLATFORM, SECOND EDITION (2002) SECTION 4.4
13. OUTLINE
14. OUTLINE Receive Request PHP Class Work out what PHP Class and Method needs to handle it Generate response to browser PHP Class PHP Class PHP Class PHP Class PHP Class PHP Class
15. ORIGINAL HANDLING OF REQUEST index.php Determine which PHP Class and Method to call Authorise Invoke Class with Method Lookup and prepare View according to Result
16. OUTLINE
17. CLASSES
18. CLASSES
19. KEY ASPECTS OF THIS ARCHITECTURE ๏ Developed with PHP 4.3 ๏ Now running PHP 5.6 ๏ Uses PEAR packages ๏ Uses Prototypejs ๏ Supports IE 6 ๏ Help files generated with nanoc (https://nanoc.ws/) ๏ Only I know how it works ๏ Could do with some architectural refactoring
20. IMPORTANT CHANGES TO DATE ๏ Main philosophy: minimal changes, assuring security ๏ Upgrade to PHP 5.4 ➤ Lots of backward-incompatible changes, particularly removal of call-time pass by reference ๏ MySQL upgrades ๏ Deployment to different hosting providers ➤ Currently with Rackspace ➤ Custom Fedora build, using Sprinkle ➤ https://github.com/sprinkle-tool/sprinkle
21. http://pear.php.net/
22. http://prototypejs.org/
23. WHY THE CHANGE? Let's go forward to 2031
24. WHY THE CHANGE? ๏ Redesign of front-end ๏ End of life for PHP 5.6 ๏ Maybe PEAR is dead? ๏ Everyone uses jQuery ๏ No-one uses IE 6 ๏ Maintainable for the next 14 years (or more) ๏ Maintainable by other people (including help files) ๏ Take advantage of modern architectural techniques ๏ Need to retain code that is proven (e.g. back-end and business logic)
25. STRATEGY ๏ Front End ➤ Redevelop in its entirety ➤ Embrace modern architecture ➤ Make it accessible ๏ Back End ➤ Keep as much as possible ➤ Take advantage of architectural changes introduced
26. CHANGE 0: PHP 7 UPGRADE
27. PHP 7 UPGRADE ๏ Get enough working to be confident it "sorta works" ๏ PHP 7 Compatibility Checker ➤ https://github.com/sstalle/php7cc ๏ Handle any residual issues during redevelopment of front end ๏ Review PEAR modules and their compatibility with PHP 7
28. CHANGE 1: DITCH PEAR Is PEAR dead?
29. PEAR ๏ Evaluate all PEAR libraries ๏ Which work with PHP 7? ๏ Replace or upgrade ๏ Move to Composer
30. PEAR LIBRARIES ๏ Some development versions exist which work with PHP 7 ๏ All available in Composer ๏ Some others don't work with PHP 7 and therefore need replacing ๏ Some say they work with PHP 7 but don't
31. REPLACE PEAR WITH COMPOSER https://getcomposer.org/
33. MOVING TO COMPOSER (1/2) ๏ Big advantage is that one can install and upgrade packages more easily ๏ Autoloader brings in all packages ๏ But my code has this:
34. MOVING TO COMPOSER (2/2) ๏ Namespace all the things! ๏ Build an Application class which is responsible for booting the application including the autoloader
35. NAMESPACE ALL THE THINGS
36. NAMESPACE ALL THE THINGS ๏ Previous serialized objects won't work, unless: O:13:"TGS_Dimension":3:{s:11:"base_factor";i:1;s:6:"amount";s: 4:"4.00";s:2:"id";s:1:"2";}
37. ORIGINAL HANDLING OF REQUEST index.php Determine which PHP Class and Method to call Authorise Invoke Class with Method Lookup and prepare View according to Result
38. NEW HANDLING OF REQUEST index.php require config/bootstrap.php require vendor/autoload.php Application::instance()->boot(); Handle request Prepare and return view
39. INDEX.PHP
40. NEW HANDLING OF REQUEST index.php require config/bootstrap.php require vendor/autoload.php Application::instance()->boot(); Handle request Prepare and return view
41. BOOTSTRAP.PHP
42. VENDOR/AUTOLOAD.PHP
43. NEW HANDLING OF REQUEST index.php require config/bootstrap.php require vendor/autoload.php Application::instance()->boot(); Handle request Prepare and return view
44. COMPOSER CAN LOAD MY STUFF TOO
45. CODE WALKTHROUGH
46. NEW HANDLING OF REQUEST index.php require config/bootstrap.php require vendor/autoload.php Application::instance()->boot(); Handle request Prepare and return view
47. NEW HANDLING OF REQUEST index.php require config/bootstrap.php require vendor/autoload.php Application::instance()->boot(); Handle request Use old way of handling requests if above fails Prepare and return view
48. CHANGE 2: APPLICATION What's this Application::boot() thing?
49. “ This is where I rummage around on the interwebs in an attempt to work out what the hell this is going to do.. -Me, sometime March 2017
50. LET'S LOOK AT LARAVEL
51. INSPIRATION FROM LARAVEL ๏ Need something to handle routes ➤ Routes turn HTTP requests into actionable methods ➤ A route invokes a Method in a Controller ๏ So.. ➤ We need a Router ➤ We need Controllers ➤ And of course something to generate the View that's sent back to the browser
52. INSPIRATION FROM LARAVEL: CONTROLLERS ๏ GET /login ➤ LoginController!::showLoginForm()
53. INSPIRATION FROM LARAVEL: ROUTES ๏ Create some route thing, then: ➤ $route!->get('/login', 'LoginController!::showLoginForm') ➤ $route!->post('/login', 'LoginController!::login') ➤ .. etc
54. “ This is where I rummage some more on the interwebs.. -Me, sometime later in March 2017
55. THE LEAGUE OF EXTRAORDINARY PACKAGES https://thephpleague.com/
56. ROUTER ๏ http://route.thephpleague.com/
57. WHAT?
59. WHAT?
61. CONTROLLER ๏ Just a class with some methods ๏ Each method is responsible for handling a specific request, typically: index Lists a resource create Shows a form for creating a new resource store Store a newly created resource in storage edit Show the form for editing the specified resource update Update the specified resource in storage destroy Remove the specified resource from storage
62. VIEW ๏ Switch to a proper templating language ➤ Use Blade (from Laravel)? ➤ Twig
63. https://twig.symfony.com/
64. CODE WALKTHROUGH
65. HOW IS THIS ALL CONNECTED?
66. BUT.. ๏ How do I handle other things, like: ➤ The old back-end still needs to work ➤ Session initiation ➤ Logging user actions ➤ Recording referrers ➤ Emails ➤ Database queries ➤ CSS and JavaScript ➤ Ajax ➤ Form validation ➤ Deployment ๏ New things I want to add, like: ➤ CSRF token verification ➤ Bug logging ➤ Database migrations ➤ Taking the website down when I deploy it ➤ Command-line debugging ➤ Queues ➤ Decoupling components via Events ➤ WordPress for help files ➤ Request normalisation ➤ Cache busting
67. THESE THINGS CAN BE DONE OUTSIDE OF THE APPLICATION: ๏ How do I handle other things, like: ➤ The old back-end still needs to work ➤ Session initiation ➤ Logging user actions ➤ Recording referrers ➤ Emails ➤ Database queries ➤ CSS and JavaScript ➤ Ajax ➤ Form validation ➤ Deployment ๏ New things I want to add, like: ➤ CSRF token verification ➤ Bug logging ➤ Database migrations ➤ Taking the website down when I deploy it ➤ Command-line debugging ➤ Queues ➤ Decoupling components via Events ➤ WordPress for help files ➤ Request normalisation ➤ Cache busting
68. INSPIRATION FROM LARAVEL: MIDDLEWARE function (ServerRequestInterface $request, ResponseInterface $response, callable $next)
69. OUTLINE OF MIDDLEWARE CLASS
70. ROUTING LIVES IN MIDDLEWARE
71. RELAYPHP.COM
73. CHANGE 3:
 BUILDING A NEW FRAMEWORK
74. WHICH OF THESE WOULD YOU LIKE TO KNOW ABOUT? ๏ How do I handle other things, like: ➤ The old back-end still needs to work ➤ Session initiation ➤ Logging user actions ➤ Recording referrers ➤ Emails ➤ Database queries ➤ CSS and JavaScript ➤ Ajax ➤ Form validation ➤ Deployment ๏ New things I want to add, like: ➤ CSRF token verification ➤ Bug logging ➤ Database migrations ➤ Taking the website down when I deploy it ➤ Command-line debugging ➤ Queues ➤ Decoupling components via Events ➤ WordPress for help files ➤ Request normalisation ➤ Cache busting
75. THE OLD BACK-END STILL NEEDS TO WORK ๏ If the request isn't handled by the Application then just forward the request onto the original back-end code
76. SESSION INITIATION ๏ Do this as part of the Application::boot() activity
77. LOGGING USER ACTIONS ๏ Middleware that records the action the user requested prior to it being actioned ๏ Logging uses Eloquent (which itself comes from Laravel) ๏ Logging can be extended to log anything via an Event handler
78. RECORDING REFERRERS ๏ How did a user find the site? ๏ Keep this information in a cookie to store in a user's account if and when they register ๏ Do all this in Middleware
79. EMAILS ๏ Build a mailing package on top of Swift Mailer ๏ Further enhanced to allow: ➤ Deferred emailing (because sending an email takes time) ➤ HTML and Text version ➤ Attachments ➤ CSS ➤ Markdown messages
80. DATABASE QUERIES ๏ No change to existing database queries, which use PEAR::DB ๏ DB has been superseded by MDB2, but MDB2 hasn't been updated since 2012, whereas DB was last updated November 2015 ๏ It's likely that the database functionality will need to be replaced at some point because of this ๏ Therefore new database functionality uses Eloquent (extracted from Laravel) ➤ composer require illuminate/database
81. CSS AND JAVASCRIPT ๏ Existing site just loads in plain CSS and JavaScript (Prototypejs) files ๏ New front-end now uses: ➤ Bootstrap 4 beta https://getbootstrap.com/ ➤ jQuery https://jquery.com/ ➤ Stylus http://stylus-lang.com/ ➤ ES6 JavaScript (with eslint) https://eslint.org/ ➤ Compiled, minified and versioned with webpack 2
 https://webpack.js.org/ ➤ Automatic cache busting with manifest files
82. AJAX ๏ Moved from cpaint to jQuery's Ajax functions ๏ For additional interactivity: ➤ http://intercoolerjs.org/ ➤ Build extra functionality into Twig to use this
83. FORM VALIDATION ๏ Existing site uses custom validation process ๏ New front-end now uses: ➤ https://github.com/siriusphp/validation ➤ 45 built-in validation rules ➤ Extensible
84. DEPLOYMENT ๏ Existing site uses rsync ๏ New architecture deploys directly from a git repository ๏ I was using https://www.deployhq.com/ (then https:// envoyer.io/) ๏ Now using GitFTP-Deploy ➤ https://eastwest.se/apps/gitftpdeploy
85. CSRF TOKEN VERIFICATION ๏ Required for all requests that change state to prevent CSRF ๏ Session has a token ๏ Token is submitted with every Ajax request ๏ Middleware checks these match ๏ Middleware (will) also checks Origin and Referer
86. BUG LOGGING ๏ Add bugsnag to automatically record errors that occur on production site ๏ https://www.bugsnag.com/
87. DATABASE MIGRATIONS ๏ Everything used to be done manually via mySQL and PHP scripts ๏ Now using Phinx (https://phinx.org/) ๏ Migrations need to be done during deployment, so, we need something to handle.. commands, e.g. ➤ Deploy code, then ➤ Run migrations
88. TAKING THE WEBSITE DOWN WHEN I DEPLOY IT ๏ More middleware, inspired by Laravel ๏ Taking the site down writes a message to a file ๏ Middleware checks if the file exists and then displays it as message ๏ Bringing the site back up deletes the file
89. COMMAND LINE TOOLS (1/2) ๏ Need something to handle: ➤ Database migrations ➤ Taking the website in to and out of maintenance mode ➤ Debugging? ๏ https://github.com/vanilla/garden-cli ๏ Register commands inside Application ๏ Application directs accordingly
90. COMMAND LINE TOOLS (2/2) ๏ Debugging ➤ Again inspired by Laravel ➤ Psysh ➤ http://psysh.org/
91. QUEUES ๏ Used to offload long-running tasks outwith of the user's request ๏ For example, sending emails ๏ Restructured version of https://github.com/seatgeek/djjob ๏ Needs a command line tool to initiate the processing of the queue ๏ Needs something to monitor the queue
92. DECOUPLING VIA EVENTS ๏ For example, when a Sale is received: ➤ Allocate stock ➤ Send email to customer ➤ Send email to administrator ➤ Update sales statistics ๏ Fire an Event "SaleWasReceived" ๏ Other parts of application Listen for this Event ๏ Be careful if there are dependencies
93. WORDPRESS FOR HELP FILES ๏ Existing site uses nanoc ➤ Ruby script to process markdown files into HTML ๏ New front-end uses WordPress ➤ Separate subdomain for a WordPress install ➤ Administrators edit content in WordPress ➤ https://github.com/corcel/corcel ➤ Front-end reads the WordPress content and displays it within a page template
94. REQUEST NORMALISATION ๏ Middleware to: ➤ Remove trailing slash ➤ Override pseudo-methods ➤ Trim strings
95. CACHE BUSTING ๏ Webpack's output can name files depending on their content, e.g.: ➤ app.ee70ab8163e78b262d84.js ๏ Webpack can produce a manifest file that contains mapping of original files to their hashed name ๏ Therefore need a Twig function to seamlessly translate between the two
96. THE END @ninthspace chris@junctionbox.com