Building "Build Configs"

https://www.oliverdavies.uk/talks/building-build-configs

Transcript

1. Building "Build Configs" Oliver Davies (@opdavies) https://opdavi.es/bcm
2. What is "Build Configs"? • Command-line tool. • Inspired by Workspace, name from the TheAltF4Stream. • Built with Symfony. • Creates and manages build configuration files. • Customisable per-project. • Drupal, PHP library, Fractal (TypeScript). • "Sprint zero in a box". @opdavies
3. What Problem Does it Solve? • I work on multiple similar projects. • Different configuration values - e.g. web vs. docroot. • Different versions of PHP, node, etc. • Different Docker Compose (fpm vs. apache images). • Each project was separate. • Difficult to add new features and fix bugs across all projects. • Inconsistencies across projects. • Out of the box solutions didn't seem like the best fit. @opdavies
4. How Does it Work? @opdavies
5. What Files Does it Generate? • Dockerfile, Docker Compose, Nix Flake, php.ini, NGINX default.conf. • run file. • PHPUnit, PHPCS, PHPStan. • GitHub Actions workflow. • Git hooks. @opdavies
6. Example build.yaml: name: my-example-project type: drupal language: php php: version: 8.1-fpm-bullseye Dockerfile: FROM php:8.1-fpm-bullseye AS base @opdavies
7. Configuring a Project php: version: 8.1-fpm-bullseye # Which PHPCS standards should be used and on which paths? phpcs: paths: [web/modules/custom] standards: [Drupal, DrupalPractice] # What level should PHPStan run and on what paths? phpstan: level: max paths: [web/modules/custom] @opdavies
8. docker-compose: # Which Docker Compose services do we need? services: - database - php - web dockerfile: stages: build: # What commands do we need to run? commands: - composer validate --strict - composer install @opdavies
9. web: type: nginx # nginx, apache, caddy database: type: mariadb # mariadb, mysql version: 10 # Where is Drupal located? drupal: docroot: web # web, docroot, null experimental: createGitHubActionsConfiguration: true runGitHooksBeforePush: true useNewDatabaseCredentials: true @opdavies
10. Overriding Values php: version: 8.1-fpm-bullseye # Disable PHPCS, PHPStan and PHPUnit. phpcs: false phpstan: false phpunit: false # Ignore more directories from Git. git: ignore: - /bin/ - /libraries/ - /web/profiles/contrib/ @opdavies
11. dockerfile: stages: build: # What additional directories do we need? extra_directories: - config - patches - scripts commands: - composer validate --strict - composer install # What additional PHP extensions do we need? extensions: install: [bcmath] @opdavies
12. Dockerfile.twig 1 FROM php:{{ php.version }} AS base 2 3 COPY --from=composer:2 /usr/bin/composer /usr/bin/composer 4 RUN which composer && composer -V 5 6 ARG DOCKER_UID=1000 7 ENV DOCKER_UID="${DOCKER_UID}" 8 9 WORKDIR {{ project_root }} 10 11 RUN adduser --disabled-password --uid "${DOCKER_UID}" app \ 12 && chown app:app -R {{ project_root }} @opdavies
13. Dockerfile.twig 1 {% if dockerfile.stages.build.extensions.install %} 2 RUN docker-php-ext-install 3 {{ dockerfile.stages.build.extensions.install|join(' ') }} 4 {% endif %} 5 6 COPY --chown=app:app phpunit.xml* ./ 7 8 {% if dockerfile.stages.build.extra_files %} 9 COPY --chown=app:app {{ dockerfile.stages.build.extra_files|join(" ") }} ./ 10 {% endif %} 11 12 {% for directory in dockerfile.stages.build.extra_directories %} 13 COPY --chown=app:app {{ directory }} {{ directory }} 14 {% endfor %} @opdavies
14. docker-compose.yaml.twig 1 services: 2 {% if "web" in dockerCompose.services %} 3 web: 4 <<: [*default-proxy, *default-app] 5 build: 6 context: . 7 target: web 8 depends_on: 9 - php 10 profiles: [web] 11 {% endif %} @opdavies
15. phpstan.neon.dist.twig 1 parameters: 2 level: {{ php.phpstan.level }} 3 excludePaths: 4 - *Test.php 5 - *TestBase.php 6 paths: 7 {% for path in php.phpstan.paths -%} 8 - {{ path }} 9 {%- endfor %} 10 11 {% if php.phpstan.baseline %} 12 includes: 13 - phpstan-baseline.neon 14 {% endif %} @opdavies
16. phpunit.xml.dist.twig 1 <phpunit 2 beStrictAboutChangesToGlobalState="true" 3 beStrictAboutOutputDuringTests="false" 4 beStrictAboutTestsThatDoNotTestAnything="true" 5 bootstrap="{{ drupal.docroot }}/core/tests/bootstrap.php" 6 cacheResult="false" 7 colors="true" 8 failOnWarning="true" 9 printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter" 10 > @opdavies
17. phpunit.xml.dist.twig 1 <testsuites> 2 <testsuite name="functional"> 3 <directory>./{{ drupal.docroot }}/modules/custom/**/tests/**/Functional</directory> 4 </testsuite> 5 <testsuite name="kernel"> 6 <directory>./{{ drupal.docroot }}/modules/custom/**/tests/**/Kernel</directory> 7 </testsuite> 8 <testsuite name="unit"> 9 <directory>./{{ drupal.docroot }}/modules/custom/**/tests/**/Unit</directory> 10 </testsuite> 11 </testsuites> @opdavies
18. Build Configs internals @opdavies
19. src/ Action/ CreateFinalConfigurationData.php CreateListOfFilesToGenerate.php GenerateConfigurationFiles.php ValidateConfigurationData.php Command/ GenerateCommand.php InitCommand.php DataTransferObject/ ConfigDto.php TemplateFile.php Enum/ Language.php ProjectType.php WebServer.php @opdavies
20. 1 protected function configure(): void 2 $this 3 ->addOption( 4 name: 'config-file', 5 shortcut: ['c'], 6 mode: InputOption::VALUE_REQUIRED, 7 description: 'The path to the project\'s build.yaml file', 8 default: 'build.yaml', 9 ) 10 ->addOption( 11 name: 'output-dir', 12 shortcut: ['o'], 13 mode: InputOption::VALUE_REQUIRED, 14 description: 'The directory to create files in', 15 default: '.', 16 ); 17 } @opdavies
21. 1 protected function execute(InputInterface $input, OutputInterface $output): int 2 { 3 $io = new SymfonyStyle($input, $output); 4 5 $configFile = $input->getOption(name: 'config-file'); 6 $outputDir = $input->getOption(name: 'output-dir'); 7 } @opdavies
22. 1 protected function execute(InputInterface $input, OutputInterface $output): int 2 { 3 // ... 4 5 $pipelines = [ 6 new CreateFinalConfigurationData(), 7 8 new ValidateConfigurationData(), 9 10 new CreateListOfFilesToGenerate(), 11 12 new GenerateConfigurationFiles( 13 $this->filesystem, 14 $this->twig, 15 $outputDir, 16 ), 17 ]; 18 } @opdavies
23. 1 protected function execute(InputInterface $input, OutputInterface $output): int 2 { 3 // ... 4 5 /** 6 * @var Collection<int,TemplateFile> $generatedFiles 7 * @var ConfigDto $configurationData 8 */ 9 [$configurationData, $generatedFiles] = (new Pipeline()) 10 ->send($configFile) 11 ->through($pipelines) 12 ->thenReturn(); 13 14 $io->info("Building configuration for {$configurationData->name}."); 15 16 $io->write('Generated files:'); 17 $io->listing(static::getListOfFiles(filesToGenerate: $generatedFiles)->toArray()); 18 19 return Command::SUCCESS; 20 } @opdavies
24. 1 // CreateFinalConfigurationData.php 2 3 public function handle(string $configFile, \Closure $next) { 4 { 5 $configurationData = Yaml::parseFile(filename: $configFile); 6 7 $configurationData = array_replace_recursive( 8 Yaml::parseFile(filename: __DIR__ . '/../../resources/build.defaults.yaml'), 9 $configurationData, 10 ); 11 12 // ... 13 14 return $next($configurationData); 15 } @opdavies
25. 1 // ValidateConfigurationData.php 2 3 public function handle(array $configurationData, \Closure $next) 4 { 5 // Convert the input to a configuration data object. 6 $normalizer = new ObjectNormalizer(null, new CamelCaseToSnakeCaseNameConverter()); 7 $serializer = new Serializer([$normalizer], [new JsonEncoder()]); 8 9 $configurationDataDto = $serializer->deserialize( 10 json_encode($configurationData), 11 ConfigDto::class, 12 'json', 13 ); 14 15 // ... 16 } @opdavies
26. 1 // ValidateConfigurationData.php 2 3 public function handle(array $configurationData, \Closure $next) 4 { 5 // ... 6 7 $validator = Validation::createValidatorBuilder() 8 ->enableAnnotationMapping() 9 ->getValidator(); 10 $violations = $validator->validate($configurationDataDto); 11 12 if (0 < $violations->count()) { 13 throw new \RuntimeException('Configuration is invalid.'); 14 } 15 16 return $next([$configurationData, $configurationDataDto]); 17 } @opdavies
27. 1 // ConfigDto.php 2 3 #[Assert\Collection( 4 allowExtraFields: false, 5 fields: ['docroot' => new Assert\Choice([null, 'web', 'docroot'])], 6 )] 7 public array $drupal; 8 9 #[Assert\Collection([ 10 'ignore' => new Assert\Optional([ 11 new Assert\All([ 12 new Assert\Type('string'), 13 ]), 14 ]), 15 ])] 16 public array $git; 17 @opdavies
28. 18 #[Assert\Choice(choices: ['javascript', 'php', 'typescript'])] 19 public string $language; 20 21 #[Assert\NotBlank] 22 #[Assert\Type('string')] 23 public string $name; 24 25 #[Assert\Type('string')] 26 public string $projectRoot; 27 28 #[Assert\Choice(choices: [ 29 'drupal', 30 'fractal', 31 'php-library', 32 'symfony', 33 ])] 34 public string $type; @opdavies
29. 1 // CreateListOfFilesToGenerate.php 2 3 public function handle(array $configurationDataAndDto, \Closure $next) 4 { 5 /** 6 * @var ConfigDto $configDto, 7 * @var array<string,mixed> $configurationData 8 */ 9 [$configurationData, $configDto] = $configurationDataAndDto; 10 11 /** @var Collection<int, TemplateFile> */ 12 $filesToGenerate = collect(); 13 14 // ... 15 } @opdavies
30. 1 // CreateListOfFilesToGenerate.php 2 3 public function handle(array $configurationDataAndDto, \Closure $next) 4 { 5 // ... 6 7 if (!isset($configDto->php['phpunit']) || $configDto->php['phpunit'] !== false) { 8 9 $filesToGenerate->push( 10 new TemplateFile( 11 data: 'drupal/phpunit.xml.dist', 12 name: 'phpunit.xml.dist', 13 ) 14 ); 15 } 16 17 // ... 18 19 return $next([$configurationData, $configDto, $filesToGenerate]); 20 } @opdavies
31. 1 // GenerateConfigurationFiles.php 2 3 public function handle(array $filesToGenerateAndConfigurationData, \Closure $next) 4 { 5 // ... 6 7 $filesToGenerate->each(function(TemplateFile $templateFile) use ($configurationData): void { 8 if ($templateFile->path !== null) { 9 if (!$this->filesystem->exists($templateFile->path)) { 10 $this->filesystem->mkdir("{$this->outputDir}/{$templateFile->path}"); 11 } 12 } 13 14 $sourceFile = "{$templateFile->data}.twig"; 15 16 $outputFile = collect([$this->outputDir, $templateFile->path, $templateFile->name]) 17 ->filter()->implode('/'); 18 19 $this->filesystem->dumpFile($outputFile, $this->twig->render($sourceFile, $configurationData)); 20 }); 21 22 return $next([$configurationDataDto, $filesToGenerate]); 23 } @opdavies
32. Demo @opdavies
33. Result • Easier and faster to create and onboard projects. • One canonical source of truth. • Easy to add new features and fixes for all projects. • Automation is easier due to consistency (e.g. Docker Compose service names). @opdavies
34. Thanks! References: • https://opdavi.es/build-configs • https://github.com/opdavies/docker-example-drupal • https://github.com/opdavies/docker-example-drupal-commerce-kickstart • https://github.com/opdavies/docker-example-drupal-localgov Me: • https://www.oliverdavies.uk @opdavies