Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building "Build Configs"

Building "Build Configs"

Oliver Davies

January 23, 2024
Tweet

More Decks by Oliver Davies

Other Decks in Technology

Transcript

  1. Building "Build
    Configs"
    Oliver Davies (@opdavies)
    https://opdavi.es/bcm

    View full-size slide

  2. What is "Build Configs"?
    • Command-line tool.
    • Inspired by Workspace, name from the TheAltF4Stream.
    • Built with Symfony.
    • Creates and manages build configuration files.
    • Customisable per-project.
    • Drupal, PHP library, Fractal (TypeScript).
    • "Sprint zero in a box".
    @opdavies

    View full-size slide

  3. What Problem Does it Solve?
    • I work on multiple similar projects.
    • Different configuration values - e.g. web vs. docroot.
    • Different versions of PHP, node, etc.
    • Different Docker Compose (fpm vs. apache images).
    • Each project was separate.
    • Difficult to add new features and fix bugs across all projects.
    • Inconsistencies across projects.
    • Out of the box solutions didn't seem like the best fit.
    @opdavies

    View full-size slide

  4. How Does it Work?
    @opdavies

    View full-size slide

  5. What Files Does it Generate?
    • Dockerfile, Docker Compose, Nix Flake, php.ini, NGINX default.conf.
    • run file.
    • PHPUnit, PHPCS, PHPStan.
    • GitHub Actions workflow.
    • Git hooks.
    @opdavies

    View full-size slide

  6. Example
    build.yaml:
    name: my-example-project
    type: drupal
    language: php
    php:
    version: 8.1-fpm-bullseye
    Dockerfile:
    FROM php:8.1-fpm-bullseye AS base
    @opdavies

    View full-size slide

  7. Configuring a Project
    php:
    version: 8.1-fpm-bullseye
    # Which PHPCS standards should be used and on which paths?
    phpcs:
    paths: [web/modules/custom]
    standards: [Drupal, DrupalPractice]
    # What level should PHPStan run and on what paths?
    phpstan:
    level: max
    paths: [web/modules/custom]
    @opdavies

    View full-size slide

  8. docker-compose:
    # Which Docker Compose services do we need?
    services:
    - database
    - php
    - web
    dockerfile:
    stages:
    build:
    # What commands do we need to run?
    commands:
    - composer validate --strict
    - composer install
    @opdavies

    View full-size slide

  9. web:
    type: nginx # nginx, apache, caddy
    database:
    type: mariadb # mariadb, mysql
    version: 10
    # Where is Drupal located?
    drupal:
    docroot: web # web, docroot, null
    experimental:
    createGitHubActionsConfiguration: true
    runGitHooksBeforePush: true
    useNewDatabaseCredentials: true
    @opdavies

    View full-size slide

  10. Overriding Values
    php:
    version: 8.1-fpm-bullseye
    # Disable PHPCS, PHPStan and PHPUnit.
    phpcs: false
    phpstan: false
    phpunit: false
    # Ignore more directories from Git.
    git:
    ignore:
    - /bin/
    - /libraries/
    - /web/profiles/contrib/
    @opdavies

    View full-size slide

  11. dockerfile:
    stages:
    build:
    # What additional directories do we need?
    extra_directories:
    - config
    - patches
    - scripts
    commands:
    - composer validate --strict
    - composer install
    # What additional PHP extensions do we need?
    extensions:
    install: [bcmath]
    @opdavies

    View full-size slide

  12. Dockerfile.twig
    1 FROM php:{{ php.version }} AS base
    2
    3 COPY --from=composer:2 /usr/bin/composer /usr/bin/composer
    4 RUN which composer && composer -V
    5
    6 ARG DOCKER_UID=1000
    7 ENV DOCKER_UID="${DOCKER_UID}"
    8
    9 WORKDIR {{ project_root }}
    10
    11 RUN adduser --disabled-password --uid "${DOCKER_UID}" app \
    12 && chown app:app -R {{ project_root }}
    @opdavies

    View full-size slide

  13. Dockerfile.twig
    1 {% if dockerfile.stages.build.extensions.install %}
    2 RUN docker-php-ext-install
    3 {{ dockerfile.stages.build.extensions.install|join(' ') }}
    4 {% endif %}
    5
    6 COPY --chown=app:app phpunit.xml* ./
    7
    8 {% if dockerfile.stages.build.extra_files %}
    9 COPY --chown=app:app {{ dockerfile.stages.build.extra_files|join(" ") }} ./
    10 {% endif %}
    11
    12 {% for directory in dockerfile.stages.build.extra_directories %}
    13 COPY --chown=app:app {{ directory }} {{ directory }}
    14 {% endfor %}
    @opdavies

    View full-size slide

  14. docker-compose.yaml.twig
    1 services:
    2 {% if "web" in dockerCompose.services %}
    3 web:
    4 <<: [*default-proxy, *default-app]
    5 build:
    6 context: .
    7 target: web
    8 depends_on:
    9 - php
    10 profiles: [web]
    11 {% endif %}
    @opdavies

    View full-size slide

  15. phpstan.neon.dist.twig
    1 parameters:
    2 level: {{ php.phpstan.level }}
    3 excludePaths:
    4 - *Test.php
    5 - *TestBase.php
    6 paths:
    7 {% for path in php.phpstan.paths -%}
    8 - {{ path }}
    9 {%- endfor %}
    10
    11 {% if php.phpstan.baseline %}
    12 includes:
    13 - phpstan-baseline.neon
    14 {% endif %}
    @opdavies

    View full-size slide

  16. phpunit.xml.dist.twig
    1 2 beStrictAboutChangesToGlobalState="true"
    3 beStrictAboutOutputDuringTests="false"
    4 beStrictAboutTestsThatDoNotTestAnything="true"
    5 bootstrap="{{ drupal.docroot }}/core/tests/bootstrap.php"
    6 cacheResult="false"
    7 colors="true"
    8 failOnWarning="true"
    9 printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter"
    10 >
    @opdavies

    View full-size slide

  17. phpunit.xml.dist.twig
    1
    2
    3 ./{{ drupal.docroot }}/modules/custom/**/tests/**/Functional
    4
    5
    6 ./{{ drupal.docroot }}/modules/custom/**/tests/**/Kernel
    7
    8
    9 ./{{ drupal.docroot }}/modules/custom/**/tests/**/Unit
    10
    11
    @opdavies

    View full-size slide

  18. Build Configs internals
    @opdavies

    View full-size slide

  19. src/
    Action/
    CreateFinalConfigurationData.php
    CreateListOfFilesToGenerate.php
    GenerateConfigurationFiles.php
    ValidateConfigurationData.php
    Command/
    GenerateCommand.php
    InitCommand.php
    DataTransferObject/
    ConfigDto.php
    TemplateFile.php
    Enum/
    Language.php
    ProjectType.php
    WebServer.php
    @opdavies

    View full-size slide

  20. 1 protected function configure(): void
    2 $this
    3 ->addOption(
    4 name: 'config-file',
    5 shortcut: ['c'],
    6 mode: InputOption::VALUE_REQUIRED,
    7 description: 'The path to the project\'s build.yaml file',
    8 default: 'build.yaml',
    9 )
    10 ->addOption(
    11 name: 'output-dir',
    12 shortcut: ['o'],
    13 mode: InputOption::VALUE_REQUIRED,
    14 description: 'The directory to create files in',
    15 default: '.',
    16 );
    17 }
    @opdavies

    View full-size slide

  21. 1 protected function execute(InputInterface $input, OutputInterface $output): int
    2 {
    3 $io = new SymfonyStyle($input, $output);
    4
    5 $configFile = $input->getOption(name: 'config-file');
    6 $outputDir = $input->getOption(name: 'output-dir');
    7 }
    @opdavies

    View full-size slide

  22. 1 protected function execute(InputInterface $input, OutputInterface $output): int
    2 {
    3 // ...
    4
    5 $pipelines = [
    6 new CreateFinalConfigurationData(),
    7
    8 new ValidateConfigurationData(),
    9
    10 new CreateListOfFilesToGenerate(),
    11
    12 new GenerateConfigurationFiles(
    13 $this->filesystem,
    14 $this->twig,
    15 $outputDir,
    16 ),
    17 ];
    18 }
    @opdavies

    View full-size slide

  23. 1 protected function execute(InputInterface $input, OutputInterface $output): int
    2 {
    3 // ...
    4
    5 /**
    6 * @var Collection $generatedFiles
    7 * @var ConfigDto $configurationData
    8 */
    9 [$configurationData, $generatedFiles] = (new Pipeline())
    10 ->send($configFile)
    11 ->through($pipelines)
    12 ->thenReturn();
    13
    14 $io->info("Building configuration for {$configurationData->name}.");
    15
    16 $io->write('Generated files:');
    17 $io->listing(static::getListOfFiles(filesToGenerate: $generatedFiles)->toArray());
    18
    19 return Command::SUCCESS;
    20 }
    @opdavies

    View full-size slide

  24. 1 // CreateFinalConfigurationData.php
    2
    3 public function handle(string $configFile, \Closure $next) {
    4 {
    5 $configurationData = Yaml::parseFile(filename: $configFile);
    6
    7 $configurationData = array_replace_recursive(
    8 Yaml::parseFile(filename: __DIR__ . '/../../resources/build.defaults.yaml'),
    9 $configurationData,
    10 );
    11
    12 // ...
    13
    14 return $next($configurationData);
    15 }
    @opdavies

    View full-size slide

  25. 1 // ValidateConfigurationData.php
    2
    3 public function handle(array $configurationData, \Closure $next)
    4 {
    5 // Convert the input to a configuration data object.
    6 $normalizer = new ObjectNormalizer(null, new CamelCaseToSnakeCaseNameConverter());
    7 $serializer = new Serializer([$normalizer], [new JsonEncoder()]);
    8
    9 $configurationDataDto = $serializer->deserialize(
    10 json_encode($configurationData),
    11 ConfigDto::class,
    12 'json',
    13 );
    14
    15 // ...
    16 }
    @opdavies

    View full-size slide

  26. 1 // ValidateConfigurationData.php
    2
    3 public function handle(array $configurationData, \Closure $next)
    4 {
    5 // ...
    6
    7 $validator = Validation::createValidatorBuilder()
    8 ->enableAnnotationMapping()
    9 ->getValidator();
    10 $violations = $validator->validate($configurationDataDto);
    11
    12 if (0 < $violations->count()) {
    13 throw new \RuntimeException('Configuration is invalid.');
    14 }
    15
    16 return $next([$configurationData, $configurationDataDto]);
    17 }
    @opdavies

    View full-size slide

  27. 1 // ConfigDto.php
    2
    3 #[Assert\Collection(
    4 allowExtraFields: false,
    5 fields: ['docroot' => new Assert\Choice([null, 'web', 'docroot'])],
    6 )]
    7 public array $drupal;
    8
    9 #[Assert\Collection([
    10 'ignore' => new Assert\Optional([
    11 new Assert\All([
    12 new Assert\Type('string'),
    13 ]),
    14 ]),
    15 ])]
    16 public array $git;
    17
    @opdavies

    View full-size slide

  28. 18 #[Assert\Choice(choices: ['javascript', 'php', 'typescript'])]
    19 public string $language;
    20
    21 #[Assert\NotBlank]
    22 #[Assert\Type('string')]
    23 public string $name;
    24
    25 #[Assert\Type('string')]
    26 public string $projectRoot;
    27
    28 #[Assert\Choice(choices: [
    29 'drupal',
    30 'fractal',
    31 'php-library',
    32 'symfony',
    33 ])]
    34 public string $type;
    @opdavies

    View full-size slide

  29. 1 // CreateListOfFilesToGenerate.php
    2
    3 public function handle(array $configurationDataAndDto, \Closure $next)
    4 {
    5 /**
    6 * @var ConfigDto $configDto,
    7 * @var array $configurationData
    8 */
    9 [$configurationData, $configDto] = $configurationDataAndDto;
    10
    11 /** @var Collection */
    12 $filesToGenerate = collect();
    13
    14 // ...
    15 }
    @opdavies

    View full-size slide

  30. 1 // CreateListOfFilesToGenerate.php
    2
    3 public function handle(array $configurationDataAndDto, \Closure $next)
    4 {
    5 // ...
    6
    7 if (!isset($configDto->php['phpunit']) || $configDto->php['phpunit'] !== false) {
    8
    9 $filesToGenerate->push(
    10 new TemplateFile(
    11 data: 'drupal/phpunit.xml.dist',
    12 name: 'phpunit.xml.dist',
    13 )
    14 );
    15 }
    16
    17 // ...
    18
    19 return $next([$configurationData, $configDto, $filesToGenerate]);
    20 }
    @opdavies

    View full-size slide

  31. 1 // GenerateConfigurationFiles.php
    2
    3 public function handle(array $filesToGenerateAndConfigurationData, \Closure $next)
    4 {
    5 // ...
    6
    7 $filesToGenerate->each(function(TemplateFile $templateFile) use ($configurationData): void {
    8 if ($templateFile->path !== null) {
    9 if (!$this->filesystem->exists($templateFile->path)) {
    10 $this->filesystem->mkdir("{$this->outputDir}/{$templateFile->path}");
    11 }
    12 }
    13
    14 $sourceFile = "{$templateFile->data}.twig";
    15
    16 $outputFile = collect([$this->outputDir, $templateFile->path, $templateFile->name])
    17 ->filter()->implode('/');
    18
    19 $this->filesystem->dumpFile($outputFile, $this->twig->render($sourceFile, $configurationData));
    20 });
    21
    22 return $next([$configurationDataDto, $filesToGenerate]);
    23 }
    @opdavies

    View full-size slide

  32. Demo
    @opdavies

    View full-size slide

  33. Result
    • Easier and faster to create and onboard projects.
    • One canonical source of truth.
    • Easy to add new features and fixes for all projects.

    Automation is easier due to consistency (e.g. Docker Compose service
    names).
    @opdavies

    View full-size slide

  34. Thanks!
    References:
    • https://opdavi.es/build-configs
    • https://github.com/opdavies/docker-example-drupal
    • https://github.com/opdavies/docker-example-drupal-commerce-kickstart
    • https://github.com/opdavies/docker-example-drupal-localgov
    Me:
    • https://www.oliverdavies.uk
    @opdavies

    View full-size slide