DevOps for TYPO3 Teams and Projects

Slides from my talk at TYPO3 Developer Days 2014 (21/06/2014, Eindhoven, Netherlands)

Transcript

1. DevOps for TYPO3 Teams and Projects TYPO3 Developer Days 2014
2. Hi. I’m Fedir. I make sites and apps. @FedirFr
3. • DevOps • Team • Project • Tools • Exchanging This workshop
4. What is DevOps ? Let's take a survey
5. Popular ideas about DevOps meaning
6. What is DevOps DevOps = Dev(elopment) + Op(eration)s
7. Software Development activities analysis coding documenting testing debugging code maintaining
8. Information Technology Operations activities management envisioning planning design implementation construction deployment distribution verification installation instantiation execution maintenance
9. Dev... DevPM DevArchitect DevDev DevInt DevSysAdmin IntAdmin DevQA IntQA
10. DevOps is not all about automation DevOps is the confluence of development and operations.
11. Developers Developers are living in an abstract world of Design Patterns and perfect IDE. App Classes are well ranged into MVC containers. Code is very structured.
12. SysAdmins SysAdmins touches reality. They need to think more about limits of current servers, about versions and compatibility of heterogeneous software. They don’t knows how Your App is working. For somebody it’s a cloud, for somebody it’s complex interconnected server infrastructure.
13. And ... Integrators ! DevOps terminology was created for companies with simple traditional structure. TYPO3 has Integrators ! Guys, who installs and configures TYPO3 on real servers, who adapts the system and extensions to current infrastructure (often they communicates with Admins).
14. Communication model
15. DevOps stages during project life In reality there is even more communication vectors Infrastructure : admin, developer, integrator CMS install : admin, integrator CMS tuning : developer, integrator Theming : integrator Functional coding Content : editor, integrator Quality control : QA manager, editor, developer, integrator, admin Deployment : admin, integrator Iterative development : admin, developer, integrator Support : admin, developer, integrator
16. Communication & methodology We should communicate effectively We should work like a team
17. Different Development Workflows Models
18. Different Projects Development Workflows
19. Different Projects Development Workflows
20. Different Projects Development Workflows
21. What is the structure of Your team ? People
22. Communication = cooperation
23. Developing tools to help the project team • Runtime control and services management • Automated functional tests • Automated visual tests • Deployment hooks
24. Dedicated interfaces • QA / PM - Web control panel with tests summary and the access to statistics logs • SysAdmin - CLI (scriptable), automatically controllable
25. Self-tested & self-controllable systems Delivered product (site, extension) delivers not only desired functionality, but QA and SA interfaces • Tests ◦ Unit ◦ Visual ◦ Functional ◦ Performance • Deployment hooks
26. When we should look for DevOps ? Projects
27. Iterative agile development V.1 V.2 V.3 When the concept of sprints is applied with granular delivery
28. Single shot project Simple task Do and forget Never meet customer again But what about code and procedures capitalisation ?
29. LTS Projects TYPO3 4.5 LTS TYPO3 6.2 LTS ... => MyProject LTS
30. Multiple servers infrastructure When Your app needs more the web server and web database • Search engines • Proxies • Caches • Services
31. DevOps and TYPO3 Closer look
32. What is special in TYPO3 • TYPO3 CMS is used for large projects, usually developer centred • TYPO3 has lot’s of data in the database • TYPO3 has multiple cache systems • TYPO3 uses external applications • TYPO3 supports automating testing
33. TYPO3 project team specific roles • Extension Developer • Site integrator • Content manager • Server Administrator • QA Tester As TYPO3 model uses roles separation, between developer and integrator, an additional level of exchange should be provided
34. Testing If my TYPO3 site works well ?
35. FE testing tools • dalek.js • galen framework • casper.js • W3C CSS Validator (local service) • W3C Markup Validator (local service) • W3C Link checker (local service)
36. FE testing aspects • Multichannel projections • Using the full spectrum of FE roles • Checking functionalities by module • Automatic checking for HTML and CSS validity • Accessibility
37. The next step : BE Users & Groups test Does Your user could make all available operations Does Your user could make forbidden operations Errors control Activity statistics
38. TYPO3 requirements OK • Checking, if Your server is TYPO3 ready ◦ TYPO3 Probe
39. TYPO3 installation control • Controlling Your installation ◦ additional_reports
40. Performance tests • WebPageTest ◦ You could install it on Your server • JMeter • gor • gatling • Tsung
41. Extensions testing 1 extension = 1 test package Easy syntax Automatic tests generation, based on content data and page structure Security
42. Install the system, keep it up to date, upgrade it Migrations Reality
43. Migrations is not only about the content
44. How to ship the site and transfer • Scriptable environment to automate instances ◦ Ansible ◦ Puppet ◦ Chef • TYPO3 Flow base + extending by Your app • git + hooks ◦ files ◦ dB ◦ content projections
45. Application redeployment • Flush all if You can and redeploy (backups) or ... • Merge it ◦ Files content ◦ Databases ◦ Indexing ◦ Cache
46. Content components and projections • Resource-costly projections of the content also could be synchronised
47. Content merge • SearchEngine API • Using caching framework • EXT:t3xutils
48. Content automated merge • UUID • Database fixtures • EventSourcing implementation TYPO3 Neos goes right way !
49. Projections : should it be versioned ? • Physical limits of repositories • Time of synchronisation • Granularity • Ability to be merged • Relation to versioned content
50. • Update scenarios ◦ Security ▪ Critical update detection - Real-time application - Automatical tests scenarios - Verification ASAP ◦ Features added ▪ Notification - Update - Control • EXT:coreapi Infrastructure updates
51. Core Initiatives Let’s make TYPO3 better together
52. Default TYPO3 CMS package testing • Core functionality tests • Default template tests ◦ Visual tests inclusion ◦ Functional tests inclusion • Performance tests ◦ Single page load ◦ Multiple page • Platform configuration tests (pre-deploy)
53. Big architectural adjustments • UUID • Event Sourcing • Database fixtures
54. Thank You for Your attention Questions / Answers
55. Ressources http://en.wikipedia.org/wiki/DevOps http://en.wikipedia.org/wiki/Software_development http://en.wikipedia.org/wiki/Computer_programming http://en.wikipedia.org/wiki/Information_technology_operations https://devcentral.f5.com/articles/devops-is-not-all-about-automation http://www.slideshare.net/kfish/deploying-typo3-neos-websites-using- surf http://www.slideshare.net/ufried/devops-for-developers-28043923 http://www.servermanaged.it/devops/sysadmin-is-changing-cloud- devops/ http://www.webpagetest.org https://github.com/WPO-Foundation/webpagetest/ https://github.com/excilys/gatling http://dalekjs.com/ http://galenframework.com/ http://typo3.org/extensions/repository/view/additional_reports https://github.com/7elix/TYPO3-Probe http://fr.slideshare.net/etobide/20130704-t3dd13-deployment https://github.com/TYPO3-coreapi/ext-coreapi http://etobi.de/blog/2013/07/deployment-workshop-on-typo3-devdays- t3dd13-part-110/ https://github.com/etobi/Typo3ExtensionUtils http://jigsaw.w3.org/css-validator/DOWNLOAD.html http://validator.w3.org/source/#deb http://tsung.erlang-projects.org/ http://search.cpan.org/dist/W3C-LinkChecker/ http://java-source.net/open-source/web-testing-tools http://phpsec.org/library/ http://wapiti.sourceforge.net/ http://wiki.typo3.org/Caching_framework