Embedded Composer (sflive portland 2013)

Composer is a wonderful way to manage a project. But what happens when you need your application to be extensible at runtime? Enter Embedded Composer. Embedding Composer will ensure that the dependencies already included by your application are taken into account when adding additional dependencies at runtime. While this can be very useful for any application that may be installed globally it is critical for any application that may be distributed as a phar.

Transcript

1. Embedded Composer
2. simensen @beausimensen   beau.io  Beau Simensen
3. dflydev 
6. Sculpin Static Site Generator sculpin.io sculpin  @getsculpin 
7. Sculpin Static Site Generator • Symfony Console application • Non-HTTP Symfony Kernel based application • React HTTP Server • Embedded Composer My Playground
8. Sculpin Static Site Generator • Sculpin needed to support plugins and extensions on a site-by-site basis • Sculpin needed to be able to be installed globally Features That Led to Embedded Composer
9. • Create a composer.json for an individual site • Require the sculpin/sculpin package • Require additional packages (plugins, themes, etc.) • Run composer install • Profit Typical Composer Managed Installation Sculpin Static Site Generator
10. Sculpin Static Site Generator { "name": "acme/static-public-website", "description": "Acme's Public Website", "require": { "sculpin/sculpin": "2.*@dev", "components/bootstrap": "~2.3.1", "components/jquery": "~1.9.1", "components/highlightjs": "~7.3.0", "components/font-awesome": "~3.0.2", "dflydev/embedded-composer": "@dev", "composer/composer": "@dev", "symfony/console": "@dev", "symfony/debug": "@dev", "symfony/http-kernel": "@dev" }, "config": { "component-dir": "source/components" } } > composer install > vendor/bin/sculpin generate --watch --server
11. Sculpin Static Site Generator Results in Installing over 45 packages current status: downloading the internet — @igorw it's kind of like "I wanted a toothbrush, I got a vacuum cleaner from the future that can travel through space" well, not exactly, because that would be awesome. :) — @igorw
12. Sculpin Static Site Generator Global Installation Not just an alternative to downloading the internet, in some cases it may be more user friendly. Potential downside being that it is installed globally. In order to support plugins and extensions on a site-by-site basis and in order to support global installation, Sculpin would need to be runtime extensible.
13. Sculpin Static Site Generator Runtime Extensible Taking an application with known installed dependencies and extending it with additional dependencies at runtime.
14. • The application will likely have dependencies • The project may have its own dependencies • The project or its dependencies may depend on the application (especially likely for plugins) • The dependencies for the application are already installed Considerations for Runtime Extensibility Sculpin Static Site Generator
15. A Little About How Composer Works
16. Repositories Repositories represent a collection of packages Composer
17. Well Known Types of Repositories • VCS (Git, SVN, GitHub) • Composer (Packagist, Satis) • Package • PEAR Composer
18. Lesser Known Types of Repositories • Local Repository • Platform Repository • Additional Installed Repository Composer
19. Local Repository A Repository containing all of the packages installed for the root package. vendor/composer/installed.json Composer
20. { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Local Repository twig/twig: 1.12.3 vendor/composer/installed.json Composer
21. Not exactly... Lock file can change and be kept under version control. Local Repository only changes on Composer Installer actions and lives under vendor. Lock file is compared against the Local Repository to determine which dependencies are out of date. Composer So It is like a Lock File?
22. Platform Repository A Repository containing virtual packages representing platform specific packages. Think things like PHP and PHP extensions. Things that are either installed or they are not. Composer
23. php: 5.3.15 ext-gd: 0 ext-intl: 1.1.0 lib-curl: 7.22.0 ... Platform Repository $versionParser = new VersionParser(); try { $prettyVersion = PHP_VERSION; $version = $versionParser->normalize($prettyVersion); } catch (\UnexpectedValueException $e) { $prettyVersion = preg_replace('#^([^~+-]+).*$#', '$1', PHP_VERSION); $version = $versionParser->normalize($prettyVersion); } $php = new CompletePackage('php', $version, $prettyVersion); $php->setDescription('The PHP interpreter'); $platformRepository->addPackage($php); Composer
24. Platform Packages Platform Packages are how you can require: php:>5.3.2 The nature of Platform Packages is that they are immutable. For example, you cannot change the version of PHP that is installed at runtime. They are used by the Solver to satisfy dependencies but they can be neither installed nor updated. You either have them or you don’t. Composer
25. Additional Installed Repository An optional Repository that can be used to inform Composer about additional packages that should be considered as already installed. Composer
26. acme/fabricated: 1.0.0 Additional Installed Repository $additionalInstalledRepository = new InstalledArrayRepository( array( new CompletePackage('acme/fabricated', '1.0.0', '1.0.0'); ) ); $installer = Installer::create($io, $composer); $installer->setAdditionalInstalledRepository( $additionalInstalledRepository ); Composer
27. Additional Installed Packages They are similar to Platform Packages in that they are used by the Solver to satisfy dependencies but they can neither be installed nor updated. Unlike Platform Packages these packages can come from anywhere. This concept was added to Composer specifically for the purpose of Embedded Composer. Composer
28. Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install Composer
29. { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository acme/myapp: 1.0.2 irken/zim: 1.0.7 twig/twig: 1.12.3 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > composer install Composer
30. Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > myapp composer:install twig/twig: 1.12.3 Composer
31. Runtime Extensibility Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 > myapp composer:install twig/twig: 1.12.3 Composer
32. What else Do We Need to Know? Knowing that the packages are essential to ensure that Composer’s solver will know what is already installed. This is not enough. We also need to be able to locate and load the classes that are already installed. Composer
33. Vendor Directories and Class Loaders Composer will install dependencies and a class loader for the application. Composer will install dependencies and a class loader for the project. Composer
34. Easy Way to Manage These Pieces? Composer
35. https://github.com/dflydev/dflydev-embedded-composer  https://packagist.org/packages/dflydev/embedded-composer Embedded Composer
36. What is It? A set of packages and conventions to support the concept of embedding Composer into an application. Embedded Composer
37. Who Needs It? Useful for any application that wants to interact with Composer to manage its dependencies. Critical for an application that can be extended at runtime and also be installed globally. Primary focus has been on the latter. :) Embedded Composer
38. Where Did It Come From? Originally a part of Sculpin. Spun out as an independent project for the benefit of mankind. Embedded Composer
39. • dflydev/embedded-composer • dflydev/embedded-composer-core • dflydev/embedded-composer-console • dflydev/embedded-composer-bundle The Packages Embedded Composer
40. Core dflydev/embedded-composer-core Embedded Composer
41. Application vs. Project Concept In discussing Embedded Composer it is helpful to keep the concepts of Applications and Projects in mind. The Application is the code being run whether installed globally or installed as a dependency. The Project is an isolated “instance” of the application. Embedded Composer
42. Internal vs. External Concept Embedded Composer uses the terms “internal” and “external” to add context to its properties. Internal is used for the embedded Application and its dependencies if the Application is installed globally. External always refers to the Project. Embedded Composer
43. Internal-External Duality Embedded Composer has the capability for an Application to run both as a global installation as well as installed as a dependency. It is all about the class loader the application finds first and whether or not it is considered an internal or external class loader. Embedded Composer
44. Bootstraps Embedded Composer Getting the application’s class loader
45. Embedded Composer #!/usr/bin/env php <?php // // bin/myapp // function includeIfExists($file) { if (file_exists($file)) { return include $file; } } if ( // Check where autoload would be if this is a global installation // of myapp. (based on actual file) (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) && // Check where autoload would be if this is myapp included // as a dependency. (!$classLoader = includeIfExists(__DIR__.'/../../../autoload.php')) ) { echo 'You must set up the project dependencies, run the following commands:'.PHP_EOL. 'curl -sS https://getcomposer.org/installer | php'.PHP_EOL. 'php composer.phar install'.PHP_EOL; exit(1); } include('myapp.php'); Application Bootstrap Can be used both as a global installation and installed as a dependency.
46. Embedded Composer $classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php'); $classLoader = includeIfExists(__DIR__.'/../../../autoload.php'); Global Installation Typical Composer Installation Internal repository is used Internal repository is NOT used
47. Embedded Composer Phar Bootstrap Because if it isn’t where you expect it... #!/usr/bin/env php <?php // // bin/myapp-phar-stub // function includeIfExists($file) { if (file_exists($file)) { return include $file; } } if (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) { echo 'There is something terribly wrong with your archive'.PHP_EOL. 'Try downloading again?'.PHP_EOL; exit(1); } include('myapp.php');
48. Builder Embedded Composer Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder
49. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput; $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build();
50. Embedded Composer $projectDir = $input->getParameterOption('--project-dir') ?: null; Project Directory not ready for primetime :(
51. So What Can We Do Now? Embedded Composer Dflydev\EmbeddedComposer\Core\EmbeddedComposerInterface
52. Embedded Composer $embeddedComposer->processAdditionalAutoloads(); Enable Additional Autoloaders ... if the internal repository is being used (is always safe to call this)
53. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput; $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build(); $embeddedComposer->processAdditionalAutoloads();
54. Embedded Composer public function processAdditionalAutoloads() { if ($this->hasInternalRepository) { $externalAutoload = $this->externalVendorDirectory.'/autoload.php'; if (file_exists($externalAutoload)) { require_once $externalAutoload; } } }
55. Create an Embedded Environment Aware Instance of Composer Embedded Composer // requires an IOInterface $composer = $embeddedComposer->createComposer($io);
56. // requires an IOInterface $installer = $embeddedComposer->createInstaller($io); Create an Embedded Environment Aware Composer Installer Embedded Composer
57. Embedded Composer protected function execute(InputInterface $input, OutputInterface $output) { if (!$this->getApplication() instanceof EmbeddedComposerAwareInterface) { throw new \RuntimeException( 'Application must be instance of EmbeddedComposerAwareInterface' ); } $embeddedComposer = $this->getApplication()->getEmbeddedComposer(); $io = new ConsoleIO($input, $output, $this->getApplication()->getHelperSet()); $installer = $embeddedComposer->createInstaller($io); $installer ->setDryRun($input->getOption('dry-run')) ->setVerbose($input->getOption('verbose')) ->setPreferSource($input->getOption('prefer-source')) ->setDevMode($input->getOption('dev')) ->setRunScripts(!$input->getOption('no-scripts')); return $installer->run() ? 0 : 1; }
58. Find Packages in the Embedded Composer Environment Embedded Composer $package = $embeddedComposer->find('acme/myapp');
59. Access the Embedded Package Repositories Embedded Composer $externalRepository = $embeddedComposer->getExternalRepository(); $internalRepository = $embeddedComposer->getInternalRepository(); // Composite Repository representing the External and Internal // repositories. $repository = $embeddedComposer->getRepository();
60. Console dflydev/embedded-composer-console Embedded Composer
61. Bundle dflydev/embedded-composer-bundle (work in progress) Embedded Composer
62. Embedded Composer class AppKernel extends Kernel { public function registerBundles() { $bundles = array( /* ... */ new Dflydev\EmbeddedComposer\Bundle\DflydevEmbeddedComposerBundle(), ); } } { "require": { "dflydev/embedded-composer-bundle": "1.*@dev" } } > app/console --list composer composer:install Install dependencies composer:update Update dependencies
63. Embedded Composer > app/console composer:install [RuntimeException] Application must be instance of EmbeddedComposerAwareInterface D’oh
64. Issues and Workarounds Embedded Composer
65. Safe Mode If you embed Composer you need to account for dependencies not being installed. Embedded Composer
66. Embedded Composer public function doRun(InputInterface $input, OutputInterface $output) { if ($input->hasParameterOption('--safe')) { // For safe mode we should enable the Composer // commands manually. $this->add(new InstallCommand); $this->add(new UpdateCommand); } else { $this->registerCommands(); } parent::doRun($input, $output); }
67. The Root Package Installed Issue Composer does not currently add the root package to the local repository acme/myapp is not added to installed.json (There is a PR for this) Embedded Composer
68. Where is acme/myapp? Embedded Composer Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 twig/twig: 1.12.3
69. Embedded Composer vendor/dflydev/embedded-composer/.root_package.json (it is just an installed repository with one package) { "name": "acme/myapp", "require": { "twig/twig": "~1.2" }, "scripts": { "post-autoload-dump": "Dflydev\\EmbeddedComposer\\Core\\Script::postAutoloadDump" } } > composer install > composer update
70. Embedded Composer Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 acme/myapp: 1.0.2 twig/twig: 1.12.3 Local Repository Composite Repository Root Package Repository twig/twig: 1.12.3 acme/myapp: 1.0.2 Nothing to see here
71. Lock File Thrashing Switching between embedded Composer and external Composer will make a project’s lock file have an identity crisis. Embedded Composer
72. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing dependencies - Installing symfony/console (dev-master 45c1953) Cloning 45c1953fee4034f8fc92bc2029b124ed9ffb34db - Installing psr/log (1.0.0) Loading from cache - Installing twig/twig (v1.13.0) Loading from cache [...] - Installing composer/composer (dev-master 978ba29) Cloning 978ba292a67590fa1a05aff8e819ed5798a529c8 - Installing components/font-awesome (3.0.2) Loading from cache - Installing components/normalize.css (2.1.1) Loading from cache [...] Writing lock file Generating autoload files Compiling component files Embedded Composer Downloads the Internet
73. altern8@datsik:~/workspaces/websites/beau.io$ sculpin update Loading composer repositories with package information Updating dependencies - Removing sculpin/sculpin (dev-master) - Removing twig/twig (v1.13.0) - Removing symfony/yaml (v2.2.1) - Removing symfony/http-kernel (dev-master) - Removing psr/log (1.0.0) - Removing symfony/http-foundation (v2.2.1) - Removing symfony/filesystem (v2.2.1) - Removing symfony/dependency-injection (v2.2.1) - Removing symfony/debug (dev-master) - Removing symfony/config (v2.2.1) - Removing symfony/class-loader (v2.2.1) [...] - Removing dflydev/dot-access-configuration (v1.0.0) - Removing dflydev/dot-access-data (v1.0.0) - Removing dflydev/placeholder-resolver (v1.0.2) - Removing dflydev/apache-mime-types (v1.0.1) - Removing webignition/internet-media-type (0.4.1.1) - Removing webignition/quoted-string (0.1) - Removing webignition/string-parser (0.2.1) - Removing dflydev/ant-path-matcher (v1.0.3) - Removing dflydev/canal (v1.0.0) Writing lock file Generating autoload files Compiling component files Embedded Composer Removes 41 Packages
74. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing dependencies from lock file Your requirements could not be resolved to an installable set of packages. Problem 1 - Installation request for kriswallsmith/assetic v1.1.0 -> satisfiable by kriswallsmith/assetic[v1.1.0]. - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. Problem 2 - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. - robloach/component-installer 0.0.11 requires kriswallsmith/assetic 1.* -> satisfiable by kriswallsmith/assetic[v1.1.0]. - Installation request for robloach/component-installer 0.0.11 -> satisfiable by robloach/component-installer[0.0.11]. Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details. Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems. Embedded Composer
75. Embedded Composer $embeddedComposer = $embeddedComposerBuilder ->setComposerFilename('myapp.json') ->setVendorDirectory('.myapp') ->build();
76. Embedded Composer Your Options Require consistency (don’t mix and match global + typical) Don’t check in lock files (might be OK in some cases?) Segregate your application configuration (possibly the best option)
77. Hints and Suggestions Embedded Composer
78. Building a Phar What needs to be included? Embedded Composer
79. Embedded Composer { "alias": "myapp.phar", "chmod": "0755", "compactors": [ "Herrera\\Box\\Compactor\\Json", "Herrera\\Box\\Compactor\\Php" ], "files": [ "LICENSE", "vendor/composer/installed.json", "vendor/dflydev/embedded-composer/.root_package.json" ], "finder": [ { "name": [ "*.php", "*.xml", "*.xsd", "*.yaml", "*.json" ], "notName": [ "composer.json" ], "exclude": [ "phpunit", "Tests", "tests" ], "in": ["bin", "src", "vendor"] } ], "git-version": "git_version", "main": "bin/myapp-phar-stub", "output": "myapp.phar", "stub": true } Box box-project.org > box build > ./myapp.phar --version
80. Application Version Embedded Composer
81. Embedded Composer class MyApp { const GIT_VERSION = '@git_version@'; } class MyApp { const GIT_VERSION = '27fa12c'; } Git Version becomes (Box makes this easy) { "git-version": "git_version" } with
82. Embedded Composer $version = $embeddedComposer->findPackage('acme/myapp')->getPrettyVersion(); if ($version !== MyApp::GIT_VERSION && MyApp::GIT_VERSION !== '@'.'git_version'.'@') { $version .= ' ('.MyApp::GIT_VERSION.')'; } Pretty Version (Embedded Composer makes this easy) 2.0.x-dev 2.0.x-dev (27fa12c) 2.0.0-alpha3
83. Why Would I Want to Embed Composer? Embedded Composer
84. CLI Embedded Composer Embedded Composer has focussed mostly on the CLI use case largely due to Sculpin.
85. • phpunit • phpspec • phrozn • phing • drush • behat • mink • composer Embedded Composer CLI App Candidates
86. GUI Embedded Composer Most of the Embedded Composer concepts can apply to GUI as well. The internal concept may not make sense but can be safely ignored.
87. See Embedded Composer in Action Embedded Composer (or “another excuse to talk about Sculpin. ”)
88. • Sculpin needed to support plugins and extensions on a site-by-site basis • Sculpin needed to be able to be installed globally Previously Stated Sculpin Requirements Embedded Composer
89. Embedded Composer > mkdir /tmp/sculpin-demo > cd /tmp/sculpin-demo > curl -sS https://sculpin.io/installer | php > php sculpin.phar --version Sculpin version 2.0.x-dev (f11db95) - app/dev/debug
90. Embedded Composer > git clone https://github.com/simensen/beau.io.git > cd beau.io > php ../sculpin.phar install Note what gets installed here: font-awesome, jquery, modernizr, normalize.css (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
91. Embedded Composer > git clone https://github.com/sculpin/sculpin-blog-skeleton.git > cd sculpin-blog-skeleton > php ../sculpin.phar install Note what gets installed here: jquery, bootstrap, highlight (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
92. The End Result Embedded Composer • Sculpin supports plugins and extensions on a site-by-site basis • Sculpin is able to be installed globally
93. Embedded Composer https://github.com/dflydev/dflydev-embedded-composer  https://packagist.org/packages/dflydev/embedded-composer #composer #sculpin #silex-php #dflydev Questions? @beausimensen  https://joind.in/8667 sculpin.io
94. Embedded Composer