Pro Yearly is on sale from $80 to $50! »

Embedded Composer (sflive portland 2013)

Embedded Composer (sflive portland 2013)

Composer is a wonderful way to manage a project. But what happens when you need your application to be extensible at runtime? Enter Embedded Composer. Embedding Composer will ensure that the dependencies already included by your application are taken into account when adding additional dependencies at runtime. While this can be very useful for any application that may be installed globally it is critical for any application that may be distributed as a phar.

23d971deeb3975a7d28246192fbbe7b7?s=128

Beau Simensen

May 23, 2013
Tweet

Transcript

  1. Embedded Composer

  2. simensen @beausimensen   beau.io  Beau Simensen

  3. dflydev 

  4. None
  5. None
  6. Sculpin Static Site Generator sculpin.io sculpin  @getsculpin 

  7. Sculpin Static Site Generator • Symfony Console application • Non-HTTP

    Symfony Kernel based application • React HTTP Server • Embedded Composer My Playground
  8. Sculpin Static Site Generator • Sculpin needed to support plugins

    and extensions on a site-by-site basis • Sculpin needed to be able to be installed globally Features That Led to Embedded Composer
  9. • Create a composer.json for an individual site • Require

    the sculpin/sculpin package • Require additional packages (plugins, themes, etc.) • Run composer install • Profit Typical Composer Managed Installation Sculpin Static Site Generator
  10. Sculpin Static Site Generator { "name": "acme/static-public-website", "description": "Acme's Public

    Website", "require": { "sculpin/sculpin": "2.*@dev", "components/bootstrap": "~2.3.1", "components/jquery": "~1.9.1", "components/highlightjs": "~7.3.0", "components/font-awesome": "~3.0.2", "dflydev/embedded-composer": "@dev", "composer/composer": "@dev", "symfony/console": "@dev", "symfony/debug": "@dev", "symfony/http-kernel": "@dev" }, "config": { "component-dir": "source/components" } } > composer install > vendor/bin/sculpin generate --watch --server
  11. Sculpin Static Site Generator Results in Installing over 45 packages

    current status: downloading the internet — @igorw it's kind of like "I wanted a toothbrush, I got a vacuum cleaner from the future that can travel through space" well, not exactly, because that would be awesome. :) — @igorw
  12. Sculpin Static Site Generator Global Installation Not just an alternative

    to downloading the internet, in some cases it may be more user friendly. Potential downside being that it is installed globally. In order to support plugins and extensions on a site-by-site basis and in order to support global installation, Sculpin would need to be runtime extensible.
  13. Sculpin Static Site Generator Runtime Extensible Taking an application with

    known installed dependencies and extending it with additional dependencies at runtime.
  14. • The application will likely have dependencies • The project

    may have its own dependencies • The project or its dependencies may depend on the application (especially likely for plugins) • The dependencies for the application are already installed Considerations for Runtime Extensibility Sculpin Static Site Generator
  15. A Little About How Composer Works

  16. Repositories Repositories represent a collection of packages Composer

  17. Well Known Types of Repositories • VCS (Git, SVN, GitHub)

    • Composer (Packagist, Satis) • Package • PEAR Composer
  18. Lesser Known Types of Repositories • Local Repository • Platform

    Repository • Additional Installed Repository Composer
  19. Local Repository A Repository containing all of the packages installed

    for the root package. vendor/composer/installed.json Composer
  20. { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Local

    Repository twig/twig: 1.12.3 vendor/composer/installed.json Composer
  21. Not exactly... Lock file can change and be kept under

    version control. Local Repository only changes on Composer Installer actions and lives under vendor. Lock file is compared against the Local Repository to determine which dependencies are out of date. Composer So It is like a Lock File?
  22. Platform Repository A Repository containing virtual packages representing platform specific

    packages. Think things like PHP and PHP extensions. Things that are either installed or they are not. Composer
  23. php: 5.3.15 ext-gd: 0 ext-intl: 1.1.0 lib-curl: 7.22.0 ... Platform

    Repository $versionParser = new VersionParser(); try { $prettyVersion = PHP_VERSION; $version = $versionParser->normalize($prettyVersion); } catch (\UnexpectedValueException $e) { $prettyVersion = preg_replace('#^([^~+-]+).*$#', '$1', PHP_VERSION); $version = $versionParser->normalize($prettyVersion); } $php = new CompletePackage('php', $version, $prettyVersion); $php->setDescription('The PHP interpreter'); $platformRepository->addPackage($php); Composer
  24. Platform Packages Platform Packages are how you can require: php:>5.3.2

    The nature of Platform Packages is that they are immutable. For example, you cannot change the version of PHP that is installed at runtime. They are used by the Solver to satisfy dependencies but they can be neither installed nor updated. You either have them or you don’t. Composer
  25. Additional Installed Repository An optional Repository that can be used

    to inform Composer about additional packages that should be considered as already installed. Composer
  26. acme/fabricated: 1.0.0 Additional Installed Repository $additionalInstalledRepository = new InstalledArrayRepository( array(

    new CompletePackage('acme/fabricated', '1.0.0', '1.0.0'); ) ); $installer = Installer::create($io, $composer); $installer->setAdditionalInstalledRepository( $additionalInstalledRepository ); Composer
  27. Additional Installed Packages They are similar to Platform Packages in

    that they are used by the Solver to satisfy dependencies but they can neither be installed nor updated. Unlike Platform Packages these packages can come from anywhere. This concept was added to Composer specifically for the purpose of Embedded Composer. Composer
  28. Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig":

    "~1.10" } } Solver ? > composer install Composer
  29. { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" }

    } Solver ? Local Repository acme/myapp: 1.0.2 irken/zim: 1.0.7 twig/twig: 1.12.3 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > composer install Composer
  30. Additional Installed Repository { "name": "acme/myapp-project", "require": { "acme/myapp": "1.*",

    "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 Local Repository twig/twig: 1.12.3 { "name": "acme/myapp", "require": { "twig/twig": "~1.10" } } Solver ? > composer install > myapp composer:install twig/twig: 1.12.3 Composer
  31. Runtime Extensibility Additional Installed Repository { "name": "acme/myapp-project", "require": {

    "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 > myapp composer:install twig/twig: 1.12.3 Composer
  32. What else Do We Need to Know? Knowing that the

    packages are essential to ensure that Composer’s solver will know what is already installed. This is not enough. We also need to be able to locate and load the classes that are already installed. Composer
  33. Vendor Directories and Class Loaders Composer will install dependencies and

    a class loader for the application. Composer will install dependencies and a class loader for the project. Composer
  34. Easy Way to Manage These Pieces? Composer

  35. https://github.com/dflydev/dflydev-embedded-composer  https://packagist.org/packages/dflydev/embedded-composer Embedded Composer

  36. What is It? A set of packages and conventions to

    support the concept of embedding Composer into an application. Embedded Composer
  37. Who Needs It? Useful for any application that wants to

    interact with Composer to manage its dependencies. Critical for an application that can be extended at runtime and also be installed globally. Primary focus has been on the latter. :) Embedded Composer
  38. Where Did It Come From? Originally a part of Sculpin.

    Spun out as an independent project for the benefit of mankind. Embedded Composer
  39. • dflydev/embedded-composer • dflydev/embedded-composer-core • dflydev/embedded-composer-console • dflydev/embedded-composer-bundle The Packages

    Embedded Composer
  40. Core dflydev/embedded-composer-core Embedded Composer

  41. Application vs. Project Concept In discussing Embedded Composer it is

    helpful to keep the concepts of Applications and Projects in mind. The Application is the code being run whether installed globally or installed as a dependency. The Project is an isolated “instance” of the application. Embedded Composer
  42. Internal vs. External Concept Embedded Composer uses the terms “internal”

    and “external” to add context to its properties. Internal is used for the embedded Application and its dependencies if the Application is installed globally. External always refers to the Project. Embedded Composer
  43. Internal-External Duality Embedded Composer has the capability for an Application

    to run both as a global installation as well as installed as a dependency. It is all about the class loader the application finds first and whether or not it is considered an internal or external class loader. Embedded Composer
  44. Bootstraps Embedded Composer Getting the application’s class loader

  45. Embedded Composer #!/usr/bin/env php <?php // // bin/myapp // function

    includeIfExists($file) { if (file_exists($file)) { return include $file; } } if ( // Check where autoload would be if this is a global installation // of myapp. (based on actual file) (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) && // Check where autoload would be if this is myapp included // as a dependency. (!$classLoader = includeIfExists(__DIR__.'/../../../autoload.php')) ) { echo 'You must set up the project dependencies, run the following commands:'.PHP_EOL. 'curl -sS https://getcomposer.org/installer | php'.PHP_EOL. 'php composer.phar install'.PHP_EOL; exit(1); } include('myapp.php'); Application Bootstrap Can be used both as a global installation and installed as a dependency.
  46. Embedded Composer $classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php'); $classLoader = includeIfExists(__DIR__.'/../../../autoload.php'); Global Installation

    Typical Composer Installation Internal repository is used Internal repository is NOT used
  47. Embedded Composer Phar Bootstrap Because if it isn’t where you

    expect it... #!/usr/bin/env php <?php // // bin/myapp-phar-stub // function includeIfExists($file) { if (file_exists($file)) { return include $file; } } if (!$classLoader = includeIfExists(__DIR__.'/../vendor/autoload.php')) { echo 'There is something terribly wrong with your archive'.PHP_EOL. 'Try downloading again?'.PHP_EOL; exit(1); } include('myapp.php');
  48. Builder Embedded Composer Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder

  49. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput;

    $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build();
  50. Embedded Composer $projectDir = $input->getParameterOption('--project-dir') ?: null; Project Directory not

    ready for primetime :(
  51. So What Can We Do Now? Embedded Composer Dflydev\EmbeddedComposer\Core\EmbeddedComposerInterface

  52. Embedded Composer $embeddedComposer->processAdditionalAutoloads(); Enable Additional Autoloaders ... if the internal

    repository is being used (is always safe to call this)
  53. Embedded Composer // // bin/myapp.php // use Dflydev\EmbeddedComposer\Core\EmbeddedComposerBuilder; use Symfony\Component\Console\Input\ArgvInput;

    $input = new ArgvInput; $projectDir = $input->getParameterOption('--project-dir') ?: null; $embeddedComposerBuilder = new EmbeddedComposerBuilder( $classLoader, $projectDir ); $embeddedComposer = $embeddedComposerBuilder->build(); $embeddedComposer->processAdditionalAutoloads();
  54. Embedded Composer public function processAdditionalAutoloads() { if ($this->hasInternalRepository) { $externalAutoload

    = $this->externalVendorDirectory.'/autoload.php'; if (file_exists($externalAutoload)) { require_once $externalAutoload; } } }
  55. Create an Embedded Environment Aware Instance of Composer Embedded Composer

    // requires an IOInterface $composer = $embeddedComposer->createComposer($io);
  56. // requires an IOInterface $installer = $embeddedComposer->createInstaller($io); Create an Embedded

    Environment Aware Composer Installer Embedded Composer
  57. Embedded Composer protected function execute(InputInterface $input, OutputInterface $output) { if

    (!$this->getApplication() instanceof EmbeddedComposerAwareInterface) { throw new \RuntimeException( 'Application must be instance of EmbeddedComposerAwareInterface' ); } $embeddedComposer = $this->getApplication()->getEmbeddedComposer(); $io = new ConsoleIO($input, $output, $this->getApplication()->getHelperSet()); $installer = $embeddedComposer->createInstaller($io); $installer ->setDryRun($input->getOption('dry-run')) ->setVerbose($input->getOption('verbose')) ->setPreferSource($input->getOption('prefer-source')) ->setDevMode($input->getOption('dev')) ->setRunScripts(!$input->getOption('no-scripts')); return $installer->run() ? 0 : 1; }
  58. Find Packages in the Embedded Composer Environment Embedded Composer $package

    = $embeddedComposer->find('acme/myapp');
  59. Access the Embedded Package Repositories Embedded Composer $externalRepository = $embeddedComposer->getExternalRepository();

    $internalRepository = $embeddedComposer->getInternalRepository(); // Composite Repository representing the External and Internal // repositories. $repository = $embeddedComposer->getRepository();
  60. Console dflydev/embedded-composer-console Embedded Composer

  61. Bundle dflydev/embedded-composer-bundle (work in progress) Embedded Composer

  62. Embedded Composer class AppKernel extends Kernel { public function registerBundles()

    { $bundles = array( /* ... */ new Dflydev\EmbeddedComposer\Bundle\DflydevEmbeddedComposerBundle(), ); } } { "require": { "dflydev/embedded-composer-bundle": "1.*@dev" } } > app/console --list composer composer:install Install dependencies composer:update Update dependencies
  63. Embedded Composer > app/console composer:install [RuntimeException] Application must be instance

    of EmbeddedComposerAwareInterface D’oh
  64. Issues and Workarounds Embedded Composer

  65. Safe Mode If you embed Composer you need to account

    for dependencies not being installed. Embedded Composer
  66. Embedded Composer public function doRun(InputInterface $input, OutputInterface $output) { if

    ($input->hasParameterOption('--safe')) { // For safe mode we should enable the Composer // commands manually. $this->add(new InstallCommand); $this->add(new UpdateCommand); } else { $this->registerCommands(); } parent::doRun($input, $output); }
  67. The Root Package Installed Issue Composer does not currently add

    the root package to the local repository acme/myapp is not added to installed.json (There is a PR for this) Embedded Composer
  68. Where is acme/myapp? Embedded Composer Additional Installed Repository { "name":

    "acme/myapp-project", "require": { "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 twig/twig: 1.12.3
  69. Embedded Composer vendor/dflydev/embedded-composer/.root_package.json (it is just an installed repository with

    one package) { "name": "acme/myapp", "require": { "twig/twig": "~1.2" }, "scripts": { "post-autoload-dump": "Dflydev\\EmbeddedComposer\\Core\\Script::postAutoloadDump" } } > composer install > composer update
  70. Embedded Composer Additional Installed Repository { "name": "acme/myapp-project", "require": {

    "acme/myapp": "1.*", "irken/zim": "1.*" } } Solver ? Local Repository irken/zim: 1.0.7 acme/myapp: 1.0.2 twig/twig: 1.12.3 Local Repository Composite Repository Root Package Repository twig/twig: 1.12.3 acme/myapp: 1.0.2 Nothing to see here
  71. Lock File Thrashing Switching between embedded Composer and external Composer

    will make a project’s lock file have an identity crisis. Embedded Composer
  72. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing

    dependencies - Installing symfony/console (dev-master 45c1953) Cloning 45c1953fee4034f8fc92bc2029b124ed9ffb34db - Installing psr/log (1.0.0) Loading from cache - Installing twig/twig (v1.13.0) Loading from cache [...] - Installing composer/composer (dev-master 978ba29) Cloning 978ba292a67590fa1a05aff8e819ed5798a529c8 - Installing components/font-awesome (3.0.2) Loading from cache - Installing components/normalize.css (2.1.1) Loading from cache [...] Writing lock file Generating autoload files Compiling component files Embedded Composer Downloads the Internet
  73. altern8@datsik:~/workspaces/websites/beau.io$ sculpin update Loading composer repositories with package information Updating

    dependencies - Removing sculpin/sculpin (dev-master) - Removing twig/twig (v1.13.0) - Removing symfony/yaml (v2.2.1) - Removing symfony/http-kernel (dev-master) - Removing psr/log (1.0.0) - Removing symfony/http-foundation (v2.2.1) - Removing symfony/filesystem (v2.2.1) - Removing symfony/dependency-injection (v2.2.1) - Removing symfony/debug (dev-master) - Removing symfony/config (v2.2.1) - Removing symfony/class-loader (v2.2.1) [...] - Removing dflydev/dot-access-configuration (v1.0.0) - Removing dflydev/dot-access-data (v1.0.0) - Removing dflydev/placeholder-resolver (v1.0.2) - Removing dflydev/apache-mime-types (v1.0.1) - Removing webignition/internet-media-type (0.4.1.1) - Removing webignition/quoted-string (0.1) - Removing webignition/string-parser (0.2.1) - Removing dflydev/ant-path-matcher (v1.0.3) - Removing dflydev/canal (v1.0.0) Writing lock file Generating autoload files Compiling component files Embedded Composer Removes 41 Packages
  74. altern8@datsik:~/workspaces/websites/beau.io$ composer install Loading composer repositories with package information Installing

    dependencies from lock file Your requirements could not be resolved to an installable set of packages. Problem 1 - Installation request for kriswallsmith/assetic v1.1.0 -> satisfiable by kriswallsmith/assetic[v1.1.0]. - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. Problem 2 - kriswallsmith/assetic v1.1.0 requires symfony/process >=2.1,<3.0 -> no matching package found. - robloach/component-installer 0.0.11 requires kriswallsmith/assetic 1.* -> satisfiable by kriswallsmith/assetic[v1.1.0]. - Installation request for robloach/component-installer 0.0.11 -> satisfiable by robloach/component-installer[0.0.11]. Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details. Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems. Embedded Composer
  75. Embedded Composer $embeddedComposer = $embeddedComposerBuilder ->setComposerFilename('myapp.json') ->setVendorDirectory('.myapp') ->build();

  76. Embedded Composer Your Options Require consistency (don’t mix and match

    global + typical) Don’t check in lock files (might be OK in some cases?) Segregate your application configuration (possibly the best option)
  77. Hints and Suggestions Embedded Composer

  78. Building a Phar What needs to be included? Embedded Composer

  79. Embedded Composer { "alias": "myapp.phar", "chmod": "0755", "compactors": [ "Herrera\\Box\\Compactor\\Json",

    "Herrera\\Box\\Compactor\\Php" ], "files": [ "LICENSE", "vendor/composer/installed.json", "vendor/dflydev/embedded-composer/.root_package.json" ], "finder": [ { "name": [ "*.php", "*.xml", "*.xsd", "*.yaml", "*.json" ], "notName": [ "composer.json" ], "exclude": [ "phpunit", "Tests", "tests" ], "in": ["bin", "src", "vendor"] } ], "git-version": "git_version", "main": "bin/myapp-phar-stub", "output": "myapp.phar", "stub": true } Box box-project.org > box build > ./myapp.phar --version
  80. Application Version Embedded Composer

  81. Embedded Composer class MyApp { const GIT_VERSION = '@git_version@'; }

    class MyApp { const GIT_VERSION = '27fa12c'; } Git Version becomes (Box makes this easy) { "git-version": "git_version" } with
  82. Embedded Composer $version = $embeddedComposer->findPackage('acme/myapp')->getPrettyVersion(); if ($version !== MyApp::GIT_VERSION &&

    MyApp::GIT_VERSION !== '@'.'git_version'.'@') { $version .= ' ('.MyApp::GIT_VERSION.')'; } Pretty Version (Embedded Composer makes this easy) 2.0.x-dev 2.0.x-dev (27fa12c) 2.0.0-alpha3
  83. Why Would I Want to Embed Composer? Embedded Composer

  84. CLI Embedded Composer Embedded Composer has focussed mostly on the

    CLI use case largely due to Sculpin.
  85. • phpunit • phpspec • phrozn • phing • drush

    • behat • mink • composer Embedded Composer CLI App Candidates
  86. GUI Embedded Composer Most of the Embedded Composer concepts can

    apply to GUI as well. The internal concept may not make sense but can be safely ignored.
  87. See Embedded Composer in Action Embedded Composer (or “another excuse

    to talk about Sculpin. ”)
  88. • Sculpin needed to support plugins and extensions on a

    site-by-site basis • Sculpin needed to be able to be installed globally Previously Stated Sculpin Requirements Embedded Composer
  89. Embedded Composer > mkdir /tmp/sculpin-demo > cd /tmp/sculpin-demo > curl

    -sS https://sculpin.io/installer | php > php sculpin.phar --version Sculpin version 2.0.x-dev (f11db95) - app/dev/debug
  90. Embedded Composer > git clone https://github.com/simensen/beau.io.git > cd beau.io >

    php ../sculpin.phar install Note what gets installed here: font-awesome, jquery, modernizr, normalize.css (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
  91. Embedded Composer > git clone https://github.com/sculpin/sculpin-blog-skeleton.git > cd sculpin-blog-skeleton >

    php ../sculpin.phar install Note what gets installed here: jquery, bootstrap, highlight (component-installer, assetic) > php ../sculpin.phar generate --watch --server Development server is running at http://localhost:8000 Quit the server with CONTROL-C. > cd ..
  92. The End Result Embedded Composer • Sculpin supports plugins and

    extensions on a site-by-site basis • Sculpin is able to be installed globally
  93. Embedded Composer https://github.com/dflydev/dflydev-embedded-composer  https://packagist.org/packages/dflydev/embedded-composer #composer #sculpin #silex-php #dflydev Questions?

    @beausimensen  https://joind.in/8667 sculpin.io
  94. Embedded Composer