$30 off During Our Annual Pro Sale. View Details »

Develop it, Test it, Ship it, Measure it - Building an end-to-end CI
 solution for Neos projects

April 30, 2022

Develop it, Test it, Ship it, Measure it - Building an end-to-end CI
 solution for Neos projects

The development of a complex Neos project is not done by single developers. A team of specialists will be involved in different parts of your project workflow. And not all of them are developers.

How can you provide a great development (and operational) experience that automates as much as possible? And how can end-to-end tests or measurements for core web vitals be integrated into the development pipeline? Are containers the answer? And what can a CI tool do for you?

In this talk we will have a 360° look at the DevOps lifecycle and build the perfect project pipeline using tools like GitLab CI, Kubernetes, Helm, Playwright and Lighthouse CI.


April 30, 2022

More Decks by hlubek

Other Decks in Programming


  1. Develop it Test it Ship it Measure it Christopher Hlubek

    (@hlubek) Building an end-to-end CI 
 solution for Neos projects.
  2. 💖 Automation

  3. 😡 Automation It's always the little things...

  4. Why automate?

  5. Care about 
 Developer experience Creator

  6. Agile >> Fast feedback loops

  7. Capture knowledge

  8. Continuous Integration Check that changes don't 
 break anything

  9. Run a build Compile code Build frontend assets Continuous Integration

  10. Static checks Linter Typechecker TypeScript Continuous Integration

  11. Verify code With tests Unit Tests Functional Tests or Fusion

    Snapshot Tests [https://github.com/suf fl e/Suf fl e.Snapshot] Continuous Integration
  12. Make Deployments Easy and safe So you can do it

  13. Build artifacts For delivery

  14. Get early Feedback

  15. Review deployments Preview deployments

  16. Verify Correctness

  17. Do manual tests On disposable instances

  18. Run end-to-end Tests on environments

  19. Measure performance Web vitals Metrics

  20. How we do it

  21. Monorepo

  22. 1 Repository Many components Monorepo

  23. Monorepo JS app Native app Neos Microservice Documentation

  24. GitLab CI

  25. Project = Repository GitLab CI

  26. .gitlab-ci.yml CI Con fi guration in Git Repo GitLab CI

    Yay, YAML!
  27. Pipeline A CI run with multiple Jobs in Stages GitLab

  28. Job Runs in a container GitLab CI Artifacts Rules Cache

  29. GitLab CI Build assets composer Test php:unit php:functional Deploy Docker-build

    app js:unit integration Acceptance acceptance Stage by stage
  30. GitLab CI assets composer php:unit php:functional app js:unit integration acceptance

    app: stage: docker-build needs: - composer - assets With graph 
  31. Deployments / Environments

  32. Review Auto-deployed per 
 merge request Reset on each deploy

    Dynamic URL Deployments / Environments https:/ /my-project-task-update.cluster.dev
  33. Integration Auto-deployed for 
 main branch Reset on each deploy

    Fixed URL Deployments / Environments https:/ /my-project-integration.cluster.dev
  34. Staging Auto-deployed for tags Data / DB is persistent Fixed

    URL Deployments / Environments https:/ /my-project-staging.cluster.dev
  35. Integration vs. Staging Deployments / Environments Most current state Always

    fresh Reference system Preview release Test migrations Use prod content / data
  36. Production Auto-deploy for 
 version tags 
 3.7.0 Manual deploy

 release candidates 
 3.7.2-rc Deployments / Environments
  37. Kubernetes For CI jobs and dev deployments.

  38. Worker 1 Kubernetes Worker 2 Worker 3 Worker 4 Controlplane

    1 Controlplane 2 Controlplane 3 Physical view
  39. Kubernetes Namespace A Logical view Namespace B Deployment Service Ingress

    Pod Container Pod Container
  40. Kubernetes Namespace dev-my-project CI and dev deployments Namespace gitlab-ci Deployment

    gitlab-runner Pod job-123 Pod job-124 Deployment integration Ingress integration.domain.dev Deployment review-foo Ingress review-foo.domain.dev
  41. Access from 
 GitLab CI Service account of namespace

    be connected via certi fi cate Kubernetes Will be replaced by 
 GitLab Kubernetes Agent
  42. Manifests = Lots of YAML Kubernetes Declarative Resources Metadata How

    to manage?
  43. Helm

  44. Templates YAML Kubernetes Helm

  45. Charts Bundled in monorepo Templates 
 for resources Customize through

    values Helm
  46. Releases Metadata for release Can be managed via Helm CLI

    Easy to use via CI Helm
  47. Unit Test Snapshots & assertions Test different values Verify YAML

    is correct Helm https://github.com/quintush/helm-unittest
  48. Values Example default: image: # A default image tag for

    the project images (excluding Nginx) tag: 'latest' postgres: # A locally deployed PostgreSQL database (without persistence) enabled: false image: repository: "postgres" tag: "13" pullPolicy: Always env: POSTGRES_USER: neos POSTGRES_PASSWORD: only-for-development service: type: ClusterIP port: 5432 neos: replicaCount: 1 revisionHistoryLimit: 0 strategyType: Recreate # Neos Deploymnent uses 2 containers for PHP FPM and a Nginx webserver php: image: repository: 'registry.networkteam.com/chlubek/neos-ci-end-to-end/neos' # tag: 'latest' pullPolicy: Always env: FPM_FASTCGI_ADDRESS: localhost:9000 FLOW_HTTP_TRUSTED_PROXIES: "*" resources: {} nginx: image: repository: 'registry.networkteam.com/networkteam/docker/nginx' tag: '1.3' pullPolicy: Always env: FPM_FASTCGI_ADDRESS: localhost:9000 resources: {} # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: Helm
  49. Overriding from CI deploy:integration: extends: .helm-deploy-job # Use Helm image

    and run chart in ci/helm with overrides from env vars image: $HELM_IMAGE stage: deploy rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH environment: name: integration url: https://my-project-integration.cluster.dev variables: RELEASE_NAME: integration script: - | cat <<EOF > overrides.yaml neos: php: env: NEOS_CREATE_ADMIN_USER_NAME: admin NEOS_CREATE_ADMIN_USER_PASSWORD: password NEOS_IMPORT_SITE_PACKAGE_KEY: 'Neos.Demo' ingress: hostnames: - my-project-$RELEASE_NAME.cluster.dev tls: - secretName: wildcard-cluster-dev hosts: - my-project-$RELEASE_NAME.cluster-dev default: image: tag: $CI_REGISTRY_TAG annotations: app.gitlab.com/env: $CI_ENVIRONMENT_SLUG app.gitlab.com/app: $CI_PROJECT_PATH_SLUG postgres: enabled: true EOF # Make sure the previous release is completely removed - helm delete $RELEASE_NAME > /dev/null 2>&1 || echo "No previous release found" # Install the chart and wait until everything is ready, # otherwise print logs of failed pods and fail the build - | (helm install --wait -f overrides.yaml $RELEASE_NAME . && echo "Done.") \ || (inspect-failed-release $RELEASE_NAME && false) Helm
  50. Kaniko https://github.com/GoogleContainerTools/kaniko

  51. Unprivileged Image builds No Docker-in-Docker Runs in a container Shared

    build cache Kaniko
  52. Custom Kaniko Image Build script Registry auth Push to Harbor

 (for tags) Kaniko Built using low-level tools 
 (Skopeo, Umoci)
  53. Bonus 
 Reproducible Builds Exact digest (hash) with same file

    content Kaniko
  54. None
  55. Playwright

  56. Code tests In TypeScript Playwright import { test as base,

    expect, Page, Locator } from '@playwright/test'; // ... test.describe('ContactForm', () => { test('success', async ({ contactForm }) => { await contactForm.fillField('Name', 'Christopher'); await contactForm.fillField('Email', 'j.doe@example.com'); await contactForm.fillField('Message', 'Hi there, this is a test!'); await contactForm.getSubmitButton().click(); await expect(contactForm.message).toHaveText('Thank you!'); }); test.describe('validation', () => { test('required field', async ({ contactForm }) => { await contactForm.fillField('Name', 'Christopher'); await contactForm.fillField('Email', 'j.doe@example.com'); await contactForm.getSubmitButton().click(); await expect(contactForm.errorFor("Message")) .toHaveText('This property is required'); }); test('email field', async ({ contactForm }) => { await contactForm.fillField('Email', 'invalid'); await contactForm.getSubmitButton().click(); await expect(contactForm.errorFor("Email")) .toHaveText('Please specify a valid email address'); }); ContactForm.spec.ts
  57. Use Page Object Model 
 To abstract selectors Playwright Especially

    for forms! import { expect, Locator } from "@playwright/test"; export class FormObject { protected form: Locator; constructor(form: Locator) { this.form = form; } async isVisible() { await expect(this.form).toBeVisible(); } getField(label: string) { return this.form.locator( `.form-group:has(label:has-text("${label}")) .form-control` ); } errorFor(label: string) { return this.form.locator( `.form-group:has(label:has-text("${label}")) .errors` ); } async fillField(label: string, value: string) { const field = this.getField(label); await expect(field).toBeVisible(); await field.fill(value); } getSubmitButton() { return this.form.locator('button[type="submit"]'); } }
  58. Use integration / review For acceptance tests Playwright

  59. Config Example import { PlaywrightTestConfig } from "@playwright/test"; const config:

    PlaywrightTestConfig = { testMatch: "DistributionPackages/**/*.spec.ts", use: { baseURL: process.env.BASE_URL || "http://localhost:8081", trace: "on-first-retry", }, // Enable retries for CI and to capture videos, but not for local development retries: process.env.CI ? 2 : undefined, // Enable list reporter (defaults to dot) and junit for CI reporter: process.env.CI ? [["list"], ["junit", { outputFile: "test-results/report.xml" }]] : "list", }; export default config; Playwright
  60. Test 
 Presentational components Via Monocle Playwright

  61. Neos Fixtures Playwright Site dump with 
 fixed content Provide

    Rest API to 
 set up fixture data
  62. Reliable Tests Playwright Tests can be retried Auto-wait with timeout

  63. Traces For debugging Playwright Saved as artifact Can be viewed

  64. ... and more Playwright Extend Playwright fixtures (type-safe) Intercept and

    assert XHR requests Easy to run headless in CI via container
  65. Lighthouse CI LHCI for short.

  66. LHCI Lighthouse CI Standalone server Build results can be reported

    to LHCI Maintains history to compare reports
  67. Deploy the Server Lighthouse CI Deployed in dev cluster Integrated

    into GitLab via webhook
  68. Measure Lighthouse CI Run for deployed environments Job template is

    added via include 🚫 Do not run on shared instances / containers
  69. Compare Lighthouse CI

  70. What it looks like

  71. Demo project https://github.com/networkteam/neos-ci-end-to-end

  72. None
  73. None
  74. None
  75. None
  76. None
  77. None
  78. None
  79. None
  80. None
  81. None
  82. Seriously YAML is okay You need to know the rules.

 There are strange ones.
  83. What we learned

  84. None
  85. Reliability is key

  86. Pipelines 
 Must not be slow

  87. Share common CI configuration But beware having 
 too much

  88. Cache what you can

  89. Building Docker images Is Hard™

  90. Fin Let's get in contact about Neos and CI 

    Christopher Hlubek (@hlubek) https://github.com/networkteam/neos-ci-end-to-end