DevOps for TYPO3 Teams and Projects

DevOps for TYPO3 Teams and Projects

Slides from my talk at TYPO3 Developer Days 2014 (21/06/2014, Eindhoven, Netherlands)

Fdeabaecd044c39cb7d75b104b0aff7c?s=128

Fedir RYKHTIK

June 21, 2014
Tweet

Transcript

  1. DevOps for TYPO3 Teams and Projects TYPO3 Developer Days 2014

  2. Hi. I’m Fedir. I make sites and apps. @FedirFr

  3. • DevOps • Team • Project • Tools • Exchanging

    This workshop
  4. What is DevOps ? Let's take a survey

  5. Popular ideas about DevOps meaning

  6. What is DevOps DevOps = Dev(elopment) + Op(eration)s

  7. Software Development activities analysis coding documenting testing debugging code maintaining

  8. Information Technology Operations activities management envisioning planning design implementation construction

    deployment distribution verification installation instantiation execution maintenance
  9. Dev... DevPM DevArchitect DevDev DevInt DevSysAdmin IntAdmin DevQA IntQA

  10. DevOps is not all about automation DevOps is the confluence

    of development and operations.
  11. Developers Developers are living in an abstract world of Design

    Patterns and perfect IDE. App Classes are well ranged into MVC containers. Code is very structured.
  12. SysAdmins SysAdmins touches reality. They need to think more about

    limits of current servers, about versions and compatibility of heterogeneous software. They don’t knows how Your App is working. For somebody it’s a cloud, for somebody it’s complex interconnected server infrastructure.
  13. And ... Integrators ! DevOps terminology was created for companies

    with simple traditional structure. TYPO3 has Integrators ! Guys, who installs and configures TYPO3 on real servers, who adapts the system and extensions to current infrastructure (often they communicates with Admins).
  14. Communication model

  15. DevOps stages during project life In reality there is even

    more communication vectors Infrastructure : admin, developer, integrator CMS install : admin, integrator CMS tuning : developer, integrator Theming : integrator Functional coding Content : editor, integrator Quality control : QA manager, editor, developer, integrator, admin Deployment : admin, integrator Iterative development : admin, developer, integrator Support : admin, developer, integrator
  16. Communication & methodology We should communicate effectively We should work

    like a team
  17. Different Development Workflows Models

  18. Different Projects Development Workflows

  19. Different Projects Development Workflows

  20. Different Projects Development Workflows

  21. What is the structure of Your team ? People

  22. Communication = cooperation

  23. Developing tools to help the project team • Runtime control

    and services management • Automated functional tests • Automated visual tests • Deployment hooks
  24. Dedicated interfaces • QA / PM - Web control panel

    with tests summary and the access to statistics logs • SysAdmin - CLI (scriptable), automatically controllable
  25. Self-tested & self-controllable systems Delivered product (site, extension) delivers not

    only desired functionality, but QA and SA interfaces • Tests ◦ Unit ◦ Visual ◦ Functional ◦ Performance • Deployment hooks
  26. When we should look for DevOps ? Projects

  27. Iterative agile development V.1 V.2 V.3 When the concept of

    sprints is applied with granular delivery
  28. Single shot project Simple task Do and forget Never meet

    customer again But what about code and procedures capitalisation ?
  29. LTS Projects TYPO3 4.5 LTS TYPO3 6.2 LTS ... =>

    MyProject LTS
  30. Multiple servers infrastructure When Your app needs more the web

    server and web database • Search engines • Proxies • Caches • Services
  31. DevOps and TYPO3 Closer look

  32. What is special in TYPO3 • TYPO3 CMS is used

    for large projects, usually developer centred • TYPO3 has lot’s of data in the database • TYPO3 has multiple cache systems • TYPO3 uses external applications • TYPO3 supports automating testing
  33. TYPO3 project team specific roles • Extension Developer • Site

    integrator • Content manager • Server Administrator • QA Tester As TYPO3 model uses roles separation, between developer and integrator, an additional level of exchange should be provided
  34. Testing If my TYPO3 site works well ?

  35. FE testing tools • dalek.js • galen framework • casper.js

    • W3C CSS Validator (local service) • W3C Markup Validator (local service) • W3C Link checker (local service)
  36. FE testing aspects • Multichannel projections • Using the full

    spectrum of FE roles • Checking functionalities by module • Automatic checking for HTML and CSS validity • Accessibility
  37. The next step : BE Users & Groups test Does

    Your user could make all available operations Does Your user could make forbidden operations Errors control Activity statistics
  38. TYPO3 requirements OK • Checking, if Your server is TYPO3

    ready ◦ TYPO3 Probe
  39. TYPO3 installation control • Controlling Your installation ◦ additional_reports

  40. Performance tests • WebPageTest ◦ You could install it on

    Your server • JMeter • gor • gatling • Tsung
  41. Extensions testing 1 extension = 1 test package Easy syntax

    Automatic tests generation, based on content data and page structure Security
  42. Install the system, keep it up to date, upgrade it

    Migrations Reality
  43. Migrations is not only about the content

  44. How to ship the site and transfer • Scriptable environment

    to automate instances ◦ Ansible ◦ Puppet ◦ Chef • TYPO3 Flow base + extending by Your app • git + hooks ◦ files ◦ dB ◦ content projections
  45. Application redeployment • Flush all if You can and redeploy

    (backups) or ... • Merge it ◦ Files content ◦ Databases ◦ Indexing ◦ Cache
  46. Content components and projections • Resource-costly projections of the content

    also could be synchronised
  47. Content merge • SearchEngine API • Using caching framework •

    EXT:t3xutils
  48. Content automated merge • UUID • Database fixtures • EventSourcing

    implementation TYPO3 Neos goes right way !
  49. Projections : should it be versioned ? • Physical limits

    of repositories • Time of synchronisation • Granularity • Ability to be merged • Relation to versioned content
  50. • Update scenarios ◦ Security ▪ Critical update detection -

    Real-time application - Automatical tests scenarios - Verification ASAP ◦ Features added ▪ Notification - Update - Control • EXT:coreapi Infrastructure updates
  51. Core Initiatives Let’s make TYPO3 better together

  52. Default TYPO3 CMS package testing • Core functionality tests •

    Default template tests ◦ Visual tests inclusion ◦ Functional tests inclusion • Performance tests ◦ Single page load ◦ Multiple page • Platform configuration tests (pre-deploy)
  53. Big architectural adjustments • UUID • Event Sourcing • Database

    fixtures
  54. Thank You for Your attention Questions / Answers

  55. Ressources http://en.wikipedia.org/wiki/DevOps http://en.wikipedia.org/wiki/Software_development http://en.wikipedia.org/wiki/Computer_programming http://en.wikipedia.org/wiki/Information_technology_operations https://devcentral.f5.com/articles/devops-is-not-all-about-automation http://www.slideshare.net/kfish/deploying-typo3-neos-websites-using- surf http://www.slideshare.net/ufried/devops-for-developers-28043923 http://www.servermanaged.it/devops/sysadmin-is-changing-cloud-

    devops/ http://www.webpagetest.org https://github.com/WPO-Foundation/webpagetest/ https://github.com/excilys/gatling http://dalekjs.com/ http://galenframework.com/ http://typo3.org/extensions/repository/view/additional_reports https://github.com/7elix/TYPO3-Probe http://fr.slideshare.net/etobide/20130704-t3dd13-deployment https://github.com/TYPO3-coreapi/ext-coreapi http://etobi.de/blog/2013/07/deployment-workshop-on-typo3-devdays- t3dd13-part-110/ https://github.com/etobi/Typo3ExtensionUtils http://jigsaw.w3.org/css-validator/DOWNLOAD.html http://validator.w3.org/source/#deb http://tsung.erlang-projects.org/ http://search.cpan.org/dist/W3C-LinkChecker/ http://java-source.net/open-source/web-testing-tools http://phpsec.org/library/ http://wapiti.sourceforge.net/ http://wiki.typo3.org/Caching_framework