Automated CI with AEM Cloud service

With the introduction of RDEs in AEMaaCS earlier this year, we now have the potential to deploy code much faster to a cloud environment

We will showcase how we can automate actions with RDE environments from CI/CD pipelines and how to leverage them to run multiple types of validations like integration test, ui tests or lighthouse performance tests.

At the end we will share a customer perspective of RDE integration with the GitLab CI system.

Deploying features to a short-lived and disposable AEMaaCS instance prior to code merge not only shortens your feedback loop, but also makes overall delivery less error prone, as the entire test suite (including functional and UI tests) can be triggered much earlier in the process increasing the confidence of the code pushed without having to wait for the Production Pipeline to fail.

Transcript

1. EUROPE'S LEADING AEM DEVELOPER CONFERENCE 25th – 27th SEPTEMBER 2023 Automated CI with AEM Cloud service Andres Bott, Jakub Wądołowski
2. Automated CI with AEM Cloud service 2 Agenda: - Intro - About CI/CD systems and AEMCS - How to use RDEs for Continuous Integration - Customer perspective Andres Bott Adobe Jakub Wądołowski Diva-e
3. 3 How did we get here
4. Getting in touch with Diva-e 4 Early 2022 Jakub and I got connected Conversation on different subjects related to testing Pain point: Cloud Manager feedback on prod deployments is too late. Work on POC how Diva-e could run tests close to CM. RDEs became a thing around the same time
5. 5 From developer to production
6. Local testing 6 From developer to production Developer tests locally (SDK) - mvn clean verify - Manual testing
7. 7 From developer to production Developer tests locally (SDK) Functional tests on deployments - Code is already merged and pushed to Cloud Manager Local testing System testing
8. Local testing 8 System testing CI testing* From developer to production Developer tests locally Functional tests on deployments Missing part: quick code iterations on branches or PRs * Not part of AEMCS offering
9. Ci system testing 9 When projects grow: Build step in CI + Functional tests / Integration tests 1. Deploy code on an environment. 2. Blackbox automated verification. - Mostly doing Http requests to the system.
10. 10 Where do we deploy our changes?
11. Deployment Targets 11 Do nothing: - Accept the risk of less verification. - Slower feedback loop. - Harder to debug issues.
12. Deployment Targets 12 Do nothing Replicate cloud service with SDK(s): - Start author + publish SDK and test against them. (e.g. on a VM) - (alternative) Containerize SDKs, deploy on own K8S, etc. - System is not comparable to CS: Mongo, replication, etc. - Expensive maintenance work.
13. Deployment Targets 13 Do nothing Replicate cloud service with SDK(s) Use AEMCS Dev/sandbox environments: - Pipelines are slow (30-60 min) - API limitations - Code needs to be pushed to CM
14. Deployment Targets 14 Do nothing Replicate cloud service with SDK(s) Use AEMCS Dev/sandbox environments RDEs to the rescue: - Fast deployments (3-4min) - Isolated reproduceable environment - Very similar to production
15. 15 CI with RDEs
16. Demo 16 Sample: https://github.com/andresbott/aem-guides- wknd/blob/main/.circleci/config.yml - Sample using CircleCi - Other systems are easy as well
17. Reproducible results 17 Start from a clean state: aio aem:rde:reset RDE reset at the start - Usually take ~10-15 min - At the end of the pipeline do a "fire and forget" reset - We can check if RDE is already clean
18. Admin like user 18 Tests require an "admin" like user: - Needed to interact with AEM features - Options: - Content package - OSGI config with repoinit
19. Admin like user 19 Tests require an "admin" like user: - OSGI config with repoinit aio aem:rde:install -t osgi-config \ org.apache.sling.jcr.repoinit.RepositoryInitializer~testadmin-user-add.cfg.json delete user test-admin create user test-admin with password <pass> add test-admin to group administrators
20. RDE install 20 - Install project: aio aem:rde:install -t content-package <package.zip> aio aem:rde:install -t dispatcher-config <dispatcher-config.zip> - AEM might restart sling bundles, - the CI system needs to handle the restart. - wait until publish returns status code 200 - wait until publish page contains <title>= "…"
21. Tests 21 Let's look into the tests - Java ITs: Custom functional - Product tests (Exploratory using Adobe internal platform) - Lighthouse performance validation - Others: - UI tests, cypress, selenium - … ideas ?
22. 22 From the Customer perspective
23. Platform overview 23
24. 24 Setup complexity § E-commerce platform § Dozens of services § AEMaaCS exposes headless content API § Interconnections & dependencies
25. 25 What’s wrong with local AEM setup? § Cloud vs local SDK discrepancies § Missing surrounding services § Behavior differences
26. 26 Review apps § New merge request == new RDE § Streamlined testing § Change isolation § Hassle-free setup
27. Review apps 27
28. 28 Gitlab MR
29. 29 Gitlab pipeline
30. 30 Caveats § Pre-provisioned RDEs § 3 Gitlab environments per MR § RDE Author § RDE Publish § Web frontend § RDE state/allocation management
31. 31 Gitlab environments
32. 32 .gitlab-ci.yml snippet
33. 33 Q&A
34. 34 Appendix
35. 35 CircleCi: overall pipeline
36. 36 CircleCi - RDE reset: setup local user
37. 37 CircleCi – Build: maven project
38. 38 CircleCi – RDE Install
39. 39 CircleCi – RDE ready
40. 40 CircleCi – Custom functional: using mvn