Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Embedded Composer (sflive portland 2013)

Embedded Composer (sflive portland 2013)

Composer is a wonderful way to manage a project. But what happens when you need your application to be extensible at runtime? Enter Embedded Composer. Embedding Composer will ensure that the dependencies already included by your application are taken into account when adding additional dependencies at runtime. While this can be very useful for any application that may be installed globally it is critical for any application that may be distributed as a phar.

Beau Simensen

May 23, 2013
Tweet

More Decks by Beau Simensen

Other Decks in Programming

Transcript

  1. Sculpin Static Site Generator • Symfony Console application • Non-HTTP

    Symfony Kernel based application • React HTTP Server • Embedded Composer My Playground
  2. Sculpin Static Site Generator • Sculpin needed to support plugins

    and extensions on a site-by-site basis • Sculpin needed to be able to be installed globally Features That Led to Embedded Composer
  3. • Create a composer.json for an individual site • Require

    the sculpin/sculpin package • Require additional packages (plugins, themes, etc.) • Run composer install • Profit Typical Composer Managed Installation Sculpin Static Site Generator
  4. Sculpin Static Site Generator { "name": "acme/static-public-website", "description": "Acme's Public

    Website", "require": { "sculpin/sculpin": "2.*@dev", "components/bootstrap": "~2.3.1", "components/jquery": "~1.9.1", "components/highlightjs": "~7.3.0", "components/font-awesome": "~3.0.2", "dflydev/embedded-composer": "@dev", "composer/composer": "@dev", "symfony/console": "@dev", "symfony/debug": "@dev", "symfony/http-kernel": "@dev" }, "config": { "component-dir": "source/components" } } > composer install > vendor/bin/sculpin generate --watch --server
  5. Sculpin Static Site Generator Results in Installing over 45 packages

    current status: downloading the internet — @igorw it's kind of like "I wanted a toothbrush, I got a vacuum cleaner from the future that can travel through space" well, not exactly, because that would be awesome. :) — @igorw
  6. Sculpin Static Site Generator Global Installation Not just an alternative

    to downloading the internet, in some cases it may be more user friendly. Potential downside being that it is installed globally. In order to support plugins and extensions on a site-by-site basis and in order to support global installation, Sculpin would need to be runtime extensible.
  7. Sculpin Static Site Generator Runtime Extensible Taking an application with

    known installed dependencies and extending it with additional dependencies at runtime.
  8. • The application will likely have dependencies • The project

    may have its own dependencies • The project or its dependencies may depend on the application (especially likely for plugins) • The dependencies for the application are already installed Considerations for Runtime Extensibility Sculpin Static Site Generator
  9. Well Known Types of Repositories • VCS (Git, SVN, GitHub)

    • Composer (Packagist, Satis) • Package • PEAR Composer
  10. Lesser Known Types of Repositories • Local Repository • Platform

    Repository • Additional Installed Repository Composer
  11. Local Repository A Repository containing all of the packages installed

    for the root package. vendor/composer/installed.json Composer
  12. { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Local

    Repository twig/twig: 1.12.3 vendor/composer/installed.json Composer
  13. Not exactly... Lock file can change and be kept under

    version control. Local Repository only changes on Composer Installer actions and lives under vendor. Lock file is compared against the Local Repository to determine which dependencies are out of date. Composer So It is like a Lock File?
  14. Platform Repository A Repository containing virtual packages representing platform specific

    packages. Think things like PHP and PHP extensions. Things that are either installed or they are not. Composer
  15. php: 5.3.15 ext-gd: 0 ext-intl: 1.1.0 lib-curl: 7.22.0 ... Platform

    Repository $versionParser = new VersionParser(); try { $prettyVersion = PHP_VERSION; $version = $versionParser->normalize($prettyVersion); } catch (\UnexpectedValueException $e) { $prettyVersion = preg_replace('#^([^~+-]+).*$#', '$1', PHP_VERSION); $version = $versionParser->normalize($prettyVersion); } $php = new CompletePackage('php', $version, $prettyVersion); $php->setDescription('The PHP interpreter'); $platformRepository->addPackage($php); Composer
  16. Platform Packages Platform Packages are how you can require: php:>5.3.2

    The nature of Platform Packages is that they are immutable. For example, you cannot change the version of PHP that is installed at runtime. They are used by the Solver to satisfy dependencies but they can be neither installed nor updated. You either have them or you don’t. Composer
  17. Additional Installed Repository An optional Repository that can be used

    to inform Composer about additional packages that should be considered as already installed. Composer
  18. acme/fabricated: 1.0.0 Additional Installed Repository $additionalInstalledRepository = new InstalledArrayRepository( array(

    new CompletePackage('acme/fabricated', '1.0.0', '1.0.0'); ) ); $installer = Installer::create($io, $composer); $installer->setAdditionalInstalledRepository( $additionalInstalledRepository ); Composer
  19. Additional Installed Packages They are similar to Platform Packages in

    that they are used by the Solver to satisfy dependencies but they can neither be installed nor updated. Unlike Platform Packages these packages can come from anywhere. This concept was added to Composer specifically for the purpose of Embedded Composer. Composer
  20. { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" }

    } Solver ? Local Repository acme/myapp: 1.0.2 irken/zim: 1.0.7 twig/twig: 1.12.3 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > composer install Composer
  21. Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*",

    "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > myapp composer:install twig/twig: 1.12.3 Composer
  22. Runtime Extensibility Additional Installed Repository { "name": "acme/myapp-project", "require": {

    "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 > myapp composer:install twig/twig: 1.12.3 Composer
  23. What else Do We Need to Know? Knowing that the

    packages are essential to ensure that Composer’s solver will know what is already installed. This is not enough. We also need to be able to locate and load the classes that are already installed. Composer
  24. Vendor Directories and Class Loaders Composer will install dependencies and

    a class loader for the application. Composer will install dependencies and a class loader for the project. Composer
  25. What is It? A set of packages and conventions to

    support the concept of embedding Composer into an application. Embedded Composer
  26. Who Needs It? Useful for any application that wants to

    interact with Composer to manage its dependencies. Critical for an application that can be extended at runtime and also be installed globally. Primary focus has been on the latter. :) Embedded Composer
  27. Where Did It Come From? Originally a part of Sculpin.

    Spun out as an independent project for the benefit of mankind. Embedded Composer
  28. Application vs. Project Concept In discussing Embedded Composer it is

    helpful to keep the concepts of Applications and Projects in mind. The Application is the code being run whether installed globally or installed as a dependency. The Project is an isolated “instance” of the application. Embedded Composer
  29. Internal vs. External Concept Embedded Composer uses the terms “internal”

    and “external” to add context to its properties. Internal is used for the embedded Application and its dependencies if the Application is installed globally. External always refers to the Project. Embedded Composer
  30. Internal-External Duality Embedded Composer has the capability for an Application

    to run both as a global installation as well as installed as a dependency. It is all about the class loader the application finds first and whether or not it is considered an internal or external class loader. Embedded Composer
  31. Embedded Composer #!/usr/bin/env php <?php // // bin/myapp // function

    includeIfExists($file) { if (file_exists($file)) { return include $file; } } if ( // Check where autoload would be if this is a global installation // of myapp. (based on actual file) (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) && // Check where autoload would be if this is myapp included // as a dependency. (!$classLoader = includeIfExists(__DIR__.'/../../../autoload.php')) ) { echo 'You must set up the project dependencies, run the following commands:'.PHP_EOL. 'curl -sS https://getcomposer.org/installer | php'.PHP_EOL. 'php composer.phar install'.PHP_EOL; exit(1); } include('myapp.php'); Application Bootstrap Can be used both as a global installation and installed as a dependency.
  32. Embedded Composer Phar Bootstrap Because if it isn’t where you

    expect it... #!/usr/bin/env php <?php // // bin/myapp-phar-stub // function includeIfExists($file) { if (file_exists($file)) { return include $file; } } if (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) { echo 'There is something terribly wrong with your archive'.PHP_EOL. 'Try downloading again?'.PHP_EOL; exit(1); } include('myapp.php');
  33. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput;

    $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build();
  34. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput;

    $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build(); $embeddedComposer->processAdditionalAutoloads();
  35. Embedded Composer public function processAdditionalAutoloads() { if ($this->hasInternalRepository) { $externalAutoload

    = $this->externalVendorDirectory.'/autoload.php'; if (file_exists($externalAutoload)) { require_once $externalAutoload; } } }
  36. Create an Embedded Environment Aware Instance of Composer Embedded Composer

    // requires an IOInterface $composer = $embeddedComposer->createComposer($io);
  37. Embedded Composer protected function execute(InputInterface $input, OutputInterface $output) { if

    (!$this->getApplication() instanceof EmbeddedComposerAwareInterface) { throw new \RuntimeException( 'Application must be instance of EmbeddedComposerAwareInterface' ); } $embeddedComposer = $this->getApplication()->getEmbeddedComposer(); $io = new ConsoleIO($input, $output, $this->getApplication()->getHelperSet()); $installer = $embeddedComposer->createInstaller($io); $installer ->setDryRun($input->getOption('dry-run')) ->setVerbose($input->getOption('verbose')) ->setPreferSource($input->getOption('prefer-source')) ->setDevMode($input->getOption('dev')) ->setRunScripts(!$input->getOption('no-scripts')); return $installer->run() ? 0 : 1; }
  38. Access the Embedded Package Repositories Embedded Composer $externalRepository = $embeddedComposer->getExternalRepository();

    $internalRepository = $embeddedComposer->getInternalRepository(); // Composite Repository representing the External and Internal // repositories. $repository = $embeddedComposer->getRepository();
  39. Embedded Composer class AppKernel extends Kernel { public function registerBundles()

    { $bundles = array( /* ... */ new Dflydev\EmbeddedComposer\Bundle\DflydevEmbeddedComposerBundle(), ); } } { "require": { "dflydev/embedded-composer-bundle": "1.*@dev" } } > app/console --list composer composer:install Install dependencies composer:update Update dependencies
  40. Safe Mode If you embed Composer you need to account

    for dependencies not being installed. Embedded Composer
  41. Embedded Composer public function doRun(InputInterface $input, OutputInterface $output) { if

    ($input->hasParameterOption('--safe')) { // For safe mode we should enable the Composer // commands manually. $this->add(new InstallCommand); $this->add(new UpdateCommand); } else { $this->registerCommands(); } parent::doRun($input, $output); }
  42. The Root Package Installed Issue Composer does not currently add

    the root package to the local repository acme/myapp is not added to installed.json (There is a PR for this) Embedded Composer
  43. Where is acme/myapp? Embedded Composer Additional Installed Repository { "name":

    "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 twig/twig: 1.12.3
  44. Embedded Composer vendor/dflydev/embedded-composer/.root_package.json (it is just an installed repository with

    one package) { "name": "acme/myapp", "require": { "twig/twig": "~1.2" }, "scripts": { "post-autoload-dump": "Dflydev\\EmbeddedComposer\\Core\\Script::postAutoloadDump" } } > composer install > composer update
  45. Embedded Composer Additional Installed Repository { "name": "acme/myapp-project", "require": {

    "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 acme/myapp: 1.0.2 twig/twig: 1.12.3 Local Repository Composite Repository Root Package Repository twig/twig: 1.12.3 acme/myapp: 1.0.2 Nothing to see here
  46. Lock File Thrashing Switching between embedded Composer and external Composer

    will make a project’s lock file have an identity crisis. Embedded Composer
  47. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing

    dependencies - Installing symfony/console (dev-master 45c1953) Cloning 45c1953fee4034f8fc92bc2029b124ed9ffb34db - Installing psr/log (1.0.0) Loading from cache - Installing twig/twig (v1.13.0) Loading from cache [...] - Installing composer/composer (dev-master 978ba29) Cloning 978ba292a67590fa1a05aff8e819ed5798a529c8 - Installing components/font-awesome (3.0.2) Loading from cache - Installing components/normalize.css (2.1.1) Loading from cache [...] Writing lock file Generating autoload files Compiling component files Embedded Composer Downloads the Internet
  48. altern8@datsik:~/workspaces/websites/beau.io$ sculpin update Loading composer repositories with package information Updating

    dependencies - Removing sculpin/sculpin (dev-master) - Removing twig/twig (v1.13.0) - Removing symfony/yaml (v2.2.1) - Removing symfony/http-kernel (dev-master) - Removing psr/log (1.0.0) - Removing symfony/http-foundation (v2.2.1) - Removing symfony/filesystem (v2.2.1) - Removing symfony/dependency-injection (v2.2.1) - Removing symfony/debug (dev-master) - Removing symfony/config (v2.2.1) - Removing symfony/class-loader (v2.2.1) [...] - Removing dflydev/dot-access-configuration (v1.0.0) - Removing dflydev/dot-access-data (v1.0.0) - Removing dflydev/placeholder-resolver (v1.0.2) - Removing dflydev/apache-mime-types (v1.0.1) - Removing webignition/internet-media-type (0.4.1.1) - Removing webignition/quoted-string (0.1) - Removing webignition/string-parser (0.2.1) - Removing dflydev/ant-path-matcher (v1.0.3) - Removing dflydev/canal (v1.0.0) Writing lock file Generating autoload files Compiling component files Embedded Composer Removes 41 Packages
  49. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing

    dependencies from lock file Your requirements could not be resolved to an installable set of packages. Problem 1 - Installation request for kriswallsmith/assetic v1.1.0 -> satisfiable by kriswallsmith/assetic[v1.1.0]. - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. Problem 2 - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. - robloach/component-installer 0.0.11 requires kriswallsmith/assetic 1.* -> satisfiable by kriswallsmith/assetic[v1.1.0]. - Installation request for robloach/component-installer 0.0.11 -> satisfiable by robloach/component-installer[0.0.11]. Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details. Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems. Embedded Composer
  50. Embedded Composer Your Options Require consistency (don’t mix and match

    global + typical) Don’t check in lock files (might be OK in some cases?) Segregate your application configuration (possibly the best option)
  51. Embedded Composer { "alias": "myapp.phar", "chmod": "0755", "compactors": [ "Herrera\\Box\\Compactor\\Json",

    "Herrera\\Box\\Compactor\\Php" ], "files": [ "LICENSE", "vendor/composer/installed.json", "vendor/dflydev/embedded-composer/.root_package.json" ], "finder": [ { "name": [ "*.php", "*.xml", "*.xsd", "*.yaml", "*.json" ], "notName": [ "composer.json" ], "exclude": [ "phpunit", "Tests", "tests" ], "in": ["bin", "src", "vendor"] } ], "git-version": "git_version", "main": "bin/myapp-phar-stub", "output": "myapp.phar", "stub": true } Box box-project.org > box build > ./myapp.phar --version
  52. Embedded Composer class MyApp { const GIT_VERSION = '@git_version@'; }

    class MyApp { const GIT_VERSION = '27fa12c'; } Git Version becomes (Box makes this easy) { "git-version": "git_version" } with
  53. Embedded Composer $version = $embeddedComposer->findPackage('acme/myapp')->getPrettyVersion(); if ($version !== MyApp::GIT_VERSION &&

    MyApp::GIT_VERSION !== '@'.'git_version'.'@') { $version .= ' ('.MyApp::GIT_VERSION.')'; } Pretty Version (Embedded Composer makes this easy) 2.0.x-dev 2.0.x-dev (27fa12c) 2.0.0-alpha3
  54. • phpunit • phpspec • phrozn • phing • drush

    • behat • mink • composer Embedded Composer CLI App Candidates
  55. GUI Embedded Composer Most of the Embedded Composer concepts can

    apply to GUI as well. The internal concept may not make sense but can be safely ignored.
  56. • Sculpin needed to support plugins and extensions on a

    site-by-site basis • Sculpin needed to be able to be installed globally Previously Stated Sculpin Requirements Embedded Composer
  57. Embedded Composer > mkdir /tmp/sculpin-demo > cd /tmp/sculpin-demo > curl

    -sS https://sculpin.io/installer | php > php sculpin.phar --version Sculpin version 2.0.x-dev (f11db95) - app/dev/debug
  58. Embedded Composer > git clone https://github.com/simensen/beau.io.git > cd beau.io >

    php ../sculpin.phar install Note what gets installed here: font-awesome, jquery, modernizr, normalize.css (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
  59. Embedded Composer > git clone https://github.com/sculpin/sculpin-blog-skeleton.git > cd sculpin-blog-skeleton >

    php ../sculpin.phar install Note what gets installed here: jquery, bootstrap, highlight (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
  60. The End Result Embedded Composer • Sculpin supports plugins and

    extensions on a site-by-site basis • Sculpin is able to be installed globally