Composer, Satis, Toran Proxy & co

Why you can’t rely only on composer (along with packagist & github) to handle your project dependencies

Transcript

1. Composer, Satis, Toran Proxy & co WHY YOU CAN’T RELY ONLY ON COMPOSER (ALONG WITH PACKAGIST & GITHUB) TO HANDLE YOUR PROJECT DEPENDENCIES /overnetcity /bobey March, 2015 http://reputationvip.io 1
2. OK, you use Composer along with packagist… HOW, ACTUALLY? 2
3. 1- Add dependencies // composer.json { "license": "proprietary", "type": "project", "autoload": { "psr-0": { "": "src/", "SymfonyStandard": "app/" } }, "require": { "php": ">=5.3.3", "symfony/symfony": "~2.6", "doctrine/orm": "~2.2,>=2.2.3,<2.5", "doctrine/dbal": "<2.5", "doctrine/doctrine-bundle": "~1.2", "twig/extensions": "~1.0", // ... } } First, you dump a few packagist dependencies in your composer.json files: 3
4. 2- Composer install Then, you throw `composer install` commands on a day-to-day basis. 15:29:24:gouzigouza:~ $ composer install Loading composer repositories with package information Installing dependencies (including require-dev) from lock file - Installing twig/twig (v1.18.0) Downloading: 100% - Installing psr/log (1.0.0) Downloading: 100% - Installing doctrine/lexer (v1.0.1) Downloading: 100% - Installing doctrine/inflector (v1.0.1) Downloading: 100% - Installing doctrine/cache (v1.4.0) Downloading: 85% 4
5. 3 - You WAIT Start waiting Wait more Go take a coffee Drink it Still wait … Then, you wait. A long time! 5
6. What if you want to add private packages? EASY ENOUGH… 6
7. Composer repositories // composer.json { "type": "project", "repositories": [ { "type": "git", "url": "git@gitlab:rvip/awesome-library.git" }, { "type": "git", "url": "git@gitlab:rvip/even-more-awesome-library.git" }, { "type": "git", "url": "git@gitlab:rvip/stunning-tool.git" }, // and 20 more repositories ... ], "require": { // ... } } Composer repositories, that’s what it’s for: 7
8. Repeat the operation in 10’s of projects inside your organization IT BECOMES KIND OF ANNOYING… BUT EVERYTHING STILL WORKS PRETTY WELL THOUGH! 8
9. Sometimes 15:29:24:gouzigouza:~ $ composer install - Installing doctrine/instantiator (1.0.4) Downloading: Connecting... Could not fetch https://api.github.com/repos/doctrine/instantiator/zipball/f976e5de371104877ebc89bd8fecb0019ed9c119, enter your GitHub credentials to go over the API rate limit A token will be created and stored in "/home/vagrant/.composer/auth.json", your password will never be stored To revoke access to this token you can visit https://github.com/settings/applications Username: ... The CI server hits Github’s API rate limit and stays locked in the middle of the unit test suite execution. Happens on your own machine too: 9
10. Once a year 15:29:24:gouzigouza:~ $ git diff diff --git a/composer.json b/composer.json index f766819..0fb85a1 100644 --- a/composer.json +++ b/composer.json @@ -3,15 +3,15 @@ "repositories": [ { "type": "git", - "url": "git@gitlab:rvip/awesome-library.git" + "url": "git@my.new.gitlab.address:rvip/awesome-library.git" }, { "type": "git", - "url": "git@gitlab/even-more-awesome-library.git" + "url": "git@my.new.gitlab.address:rvip/even-more-awesome-library.git" }, // and 20 more repositories ... ], Your gitlab instance hostname change... You need to update tens of lines in tens of projects’ composer.json file: 10
11. - Installing phpunit/php-file-iterator (1.3.4) Downloading: 100% Downloading: 100% Downloading: 100% Of course, right in the middle of the deploy process! Then, one day... Failed to download phpunit/php-file-iterator from dist: The "https://api.github.com/repos/sebastianbergmann/php-file- iterator/zipball/acd690379117b042d1c8af1fafd61bde001bf6bb" file could not be downloaded: failed to open stream: Connection refused Now trying to download from source 11
12. What can you do to fix it? 12
13. Satis Satis [...] is a static composer repository generator. It is a bit like an ultra-lightweight, static file-based version of packagist and can be used to host the metadata of your company's private packages, or your own. “ “ From composer official documentation: 13
14. Satis - Pros & cons Static file-based No official Web UI to add / edit repositories No professional support Open-Source Quite simple to setup php composer.phar create-project composer/satis 14
15. Commercial alternatives - Toran Proxy Toran Proxy is a commercial alternative to Satis offering professional support as well as a web UI to manage everything and a better integration with Composer. It also provides proxying/mirroring for git repos and package zip files which makes installs faster and independent from third party systems. “ “ From composer official documentation: 15
16. Toran proxy - Pros & cons Quite expensive for mid to large team ~1200€/year for 15 developers Not so much enhancements in the past few months Developed and maintained by Jordi Though, helps to fund Composer & Packagist Simple & effective UX Private packages Proxify github & packagist 16
17. Commercial alternatives - GemFury Gemfury is a hosted repository for your public and private packages, where they are safe and within reach. Install them to any machine in minutes without worrying about running and securing your own repository server. “ “ From GemFury website: 17
18. GemFury - Pros & cons Hosted Still quite expensive if you have numerous private projects ~1000€/year for 20 to 50 private packages Doesn’t proxify github / packagist Hosted Price doesn’t depend on the number of developers 18
19. Satis - Typical setup for private packages // satis.json { "name": "ReputationVIP repository", "homepage": "http://private.repositories.rvip", "repositories": [ { "type": "vcs", "url": "git@gitlab:rvip/awesome-library" }, { "type": "vcs", "url": "git@gitlab:rvip/even-more-awesome-library" }, { "type": "vcs", "url": "git@gitlab:rvip/stunning-tool" }, // and 20 more repositories ... ], "require-all": true } // composer.json { "type": "project", "repositories": [ { "type": "composer", "url": "http://private.repositories.rvip" } ], "require": { // ... } } Then you can replace your private repositories in all your composer.json by this simple line: 19
20. Satis - Typical setup to proxify packagist // satis.json { "name": "ReputationVIP public repository", "homepage": "http://public.repositories.rvip", "archive": { "directory": "dist", "format": "tar", "skip-dev": false }, "repositories": [ { "type": "composer", "url": "https://packagist.org" } ], "require": { "symfony/symfony": "~2.6", "doctrine/orm": "~2.2,>=2.2.3,<2.5", "doctrine/dbal": "<2.5", "doctrine/doctrine-bundle": "~1.2", "twig/extensions": "~1.0", // ... }, "require-dependencies": true } 20
21. Satis - Typical setup to proxify packagist // composer.json { "type": "project", "repositories": [ { "type": "composer", "url": "http://private.repositories.rvip" }, { "type": "composer", "url": "http://public.repositories.rvip" }, { "packagist": false } ], "require": { // ... } } Then you should completely disable packagist: 16:33:46:gouzigouza:~ $ composer install --prefer-dist Loading composer repositories with package information ... Note that this will only work with `prefer-dist` flag which is default option for stable packages: 21
22. Satis - Enhancing user experience Satisfy Satis Composer repository manager with a simple web UI. Doesn’t evolve since a few months but still do the job. https://github.com/ludofleury/satisfy 22
23. Satis - Enhancing user experience Satis-Admin Similar to satisfy. Silex powered. Allows build triggered directly from the interface. Doesn’t evolve either since few months. https://github.com/yohang/satis-admin 23
24. Satis - Enhancing user experience Terramar Labs / Packages Supra nice UX. Register Gitlab / Github web hooks to build satis. But doesn’t allow custom satis.json options needed and packagist proxy :-( http://docs.terramarlabs.com/packages/3.1 24
25. Going further Composer official documentation https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md Toran Proxy https://toranproxy.com M6 Web tech blog Olivier Mansour published a year ago a great and still relevant blog post on that exact same subject: http://tech.m6web.fr/composer-installation-without-github.html GemFury Hosted service for private packages for rubygems, npm and composer https://gemfury.com/l/composer-repository Docker Satis Easily setup one or more satis with pre-configured cron & shared composer cache https://registry.hub.docker.com/u/ypereirareis/docker-satis 25
26. Thanks Any questions? /overnetcity /bobey March, 2015 http://reputationvip.io 26