Save 37% off PRO during our Black Friday Sale! »

Manage Private Internal Dependencies With Composer and Satis

Manage Private Internal Dependencies With Composer and Satis

In this talk we'll discuss how to set up and use Composer to pull open-source packages from public repositories on packagist.org. Then we'll learn how to create your own private/corporate package management server using Satis. And we'll go through an example of how to integrate your own private code packaging server into your daily development workflow.

Note: This presentation contained a fair amount of video which is lost on Speaker Deck

F4bb45b2a18ee44c4a28b1664de150bd?s=128

Andrew Cassell

May 20, 2015
Tweet

Transcript

  1. Manage Private Internal Dependencies with Composer and Satis

  2. None
  3. None
  4. None
  5. None
  6. $ whois cassell

  7. $ whois cassell Andrew Cassell @alc277

  8. None
  9. None
  10. None
  11. None
  12. None
  13. None
  14. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE

  15. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE

  16. None
  17. GOOD ARTISTS COPY GOOD ARTISTS COPY GREAT ARTISTS STEAL GREAT

    ARTISTS STEAL
  18. GOOD ARTISTS COPY GOOD ARTISTS COPY GREAT ARTISTS STEAL GREAT

    ARTISTS STEAL https://nancyprager.wordpress.com/2007/05/08/good-poets-borrow-great-poets-steal/
  19. None
  20. GOOD DEVELOPERS COPY GOOD DEVELOPERS COPY

  21. GOOD DEVELOPERS COPY GOOD DEVELOPERS COPY GREAT DEVELOPERS ! DEVELOPERS!

    DEVELOPERS! USE COMPOSER GREAT DEVELOPERS ! DEVELOPERS! DEVELOPERS! USE COMPOSER
  22. None
  23. None
  24. Dependency Manager

  25. Dependency Manager 1. Download Project Libraries

  26. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries

  27. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries

    3. Verify Compatibility
  28. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries

    3. Verify Compatibility 4. Autoloader for ALL THE THINGS
  29. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE

  30. None
  31. $ curl -sS https://getcomposer.org/installer | php $ sudo mv composer.phar

    /usr/local/bin/composer Installing Composer
  32. $ composer require <vendor>/<package> Adding a Dependency

  33. $ composer require aws/aws-sdk-php Adding a Dependency

  34. cassell$ composer require aws/aws-sdk-php Using version ^2.8 for aws/aws-sdk-php ./composer.json

    has been created
  35. cassell$ composer require aws/aws-sdk-php Using version ^2.8 for aws/aws-sdk-php ./composer.json

    has been created Loading composer repositories with package information Updating dependencies (including require-dev) - Installing symfony/event-dispatcher (v2.6.7) Loading from cache - Installing guzzle/guzzle (v3.9.3) Downloading: 100% - Installing aws/aws-sdk-php (2.8.5) Downloading: 100%
  36. None
  37. None
  38. { "require": { "aws/aws-sdk-php": "^2.8" } } composer.json

  39. { "require": { "aws/aws-sdk-php": "~2.8" } } composer.json

  40. { "require": { "aws/aws-sdk-php": “2.8.2" } } composer.json

  41. { "require": { "aws/aws-sdk-php": “2.8.2", "league/flysystem": "1.0.3", "monolog/monolog": "1.13.1" }

    } Adding libraries to composer.json
  42. Semantic Versioning http://semver.org/

  43. 1.0.0

  44. 1.0.0 PATCH

  45. 1.0.0 PATCH MINOR

  46. 1.0.0 PATCH MAJOR MINOR

  47. 0.6.10

  48. 1.0.0

  49. backwards compatible bugfix? 1.0.0

  50. 1.0.1

  51. 15 more backwards compatible bugfixes? 1.0.1

  52. 1.0.16

  53. 1.0.16 new feature that doesn’t break existing code?

  54. 1.1.0

  55. 1.1.0 old code incompatible?

  56. 2.0.0

  57. { "require": { "aws/aws-sdk-php": “2.8.2", "league/flysystem": “1.0.3”, "monolog/monolog": “1.13.1" }

    } composer.json
  58. { "require": { "aws/aws-sdk-php": “~2.8.2”, "league/flysystem": “~1.0.0”, "monolog/monolog": “~1.13.0” }

    } composer.json
  59. ~2.8.2 2.8.2 2.9.0

  60. None
  61. None
  62. ^

  63. ^2.8.2 2.8.2 3.0.0

  64. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” }

    } composer.json
  65. None
  66. CHECK IN CHECK IN

  67. /vendor Add /vendor to your .gitignore

  68. None
  69. composer install composer update - or -

  70. composer install composer update - or -

  71. composer install composer update - or -

  72. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” },

    "require-dev": {
 "phpunit/phpunit": "^4.1.0"
 } } Dev Dependencies in composer.json
  73. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” },

    "autoload": {
 "psr-4": {"ProjectName\\": "classes/"}
 } } Autoloader in composer.json
  74. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” }

    } Where do these packages come from?
  75. None
  76. https://packagist.org/

  77. None
  78. None
  79. None
  80. None
  81. None
  82. PHPUnit

  83. PHPUnit

  84. PHPUnit

  85. None
  86. None
  87. http://thephpleague.com/

  88. EXAMPLE

  89. None
  90. None
  91. None
  92. XML is like violence. If it doesn’t solve your problem,

    you’re not using enough of it.
  93. None
  94. None
  95. None
  96. None
  97. {
 "require": {
 "fabpot/goutte": "^2.0.4",
 "symfony/yaml": "^2.6.7"
 }
 } composer.json

  98. $ composer install Loading composer repositories with package information Installing

    dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100% - Installing symfony/dom-crawler (v2.6.7) Downloading: 100% - Installing symfony/css-selector (v2.6.7) Downloading: 100% - Installing symfony/browser-kit (v2.6.7) Downloading: 100% - Installing fabpot/goutte (v2.0.4) Downloading: 100% - Installing symfony/yaml (v2.6.7) Downloading: 100% symfony/browser-kit suggests installing symfony/process () Writing lock file Generating autoload files
  99. $ composer install Loading composer repositories with package information Installing

    dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100%
  100. scraper.php

  101. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; scraper.php

  102. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new

    \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
  103. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new

    \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
  104. <?php
 
 require_once __DIR__ . "/vendor/autoload.php";
 use \Symfony\Component\DomCrawler\Crawler;
 
 $client

    = new \Goutte\Client();
 
 $crawler = $client->request('GET', 'http://tek.phparch.com/speakers/');
 
 $speakers = $crawler->filter('#speakerlist > div')->each(function (Crawler $node) {
 
 $speaker = [];
 $speaker["name"] = $node->filter('div.headshot > img')->attr("alt");
 $speaker["gravatar"] = $node->filter('div.headshot > img')->attr("src");
 $speaker["company"] = $node->filter('div.info > h4')->text();
 try {
 $speaker["twitter"] = $node->filter('div.info > h3 > a')->text();
 } catch (\Exception $e) {
 // might fail
 $speaker["twitter"] = "";
 }
 
 $speaker["talks"] = $node->filter('div.info > dl')->first()->siblings()->filter('dl')->each(function (Crawler $talkNode) {
 
 $talk = [];
 $talk['type'] = $talkNode->filter('dt > div')->eq(0)->text();
 $talk['level'] = $talkNode->filter('dt > div')->eq(1)->text();
 $talk['title'] = $talkNode->filter('dd > h5')->text();
 $texts = [];
 foreach ($talkNode->filter('dd')->getNode(0)->childNodes as $child) {
 if ($child instanceof DOMText) {
 $texts[] = trim($child->textContent);
 }
 
 }
 $talk['room'] = $texts[2];
 $talk['when'] = $texts[3];
 
 return $talk;
 
 });
 
 return $speaker;
 
 });
 
 
 $dumper = new \Symfony\Component\Yaml\Dumper();
 
 echo $dumper->dump(["speakers" => $speakers],999);

  105. None
  106. None
  107. None
  108. None
  109. { "require": { “company/secret-sauce“: “^4.0” }, "repositories": [ { "type":

    "vcs", "url": “git@github.com:company/secret-sauce.git” } ] } composer.json
  110. None
  111. DEPENDENCY DEPENDENCY DEPENDENCY DEPENDENCY

  112. None
  113. None
  114. None
  115. None
  116. $ php bin/satis build <configuration file> <build dir>

  117. satis.json

  118. None
  119. None
  120. None
  121. None
  122. EXAMPLE

  123. None
  124. None
  125. {
 "require": {
 "league/plates": "^3.1.0"
 }
 }

  126. None
  127. None
  128. None
  129. None
  130. None
  131. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": “https://packageserver.example.com/“
 }
 ]
 }
  132. SATIS http://packageserver.example.com/

  133. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [


    {
 "type": "vcs",
 "url": "git@github.com:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-cache-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-s3-wrapper-package.git"
 }
 ],
 
 "require-all": true,
 
 "archive": {
 "directory": "dist",
 "format": "tar",
 "prefix-url": “https://packageserver.example.com/“, "skip-dev": true }
 }
  134. "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 php

    bin/satis build satis.json web
  135. "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]


  136. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  137. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

  138. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

  139. None
  140. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  141. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “dev-master“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "http://localhost:9088/"
 }
 ]
 }
  142. Loading composer repositories with package information Updating dependencies (including require-dev)

    - Removing secret-company-example/phptek-scraper (1.0.2) - Installing secret-company-example/phptek-scraper (dev-master ab88f08) Cloning ab88f0877781202929049bf3516b23ddb6c0fa12 Writing lock file Generating autoload files cassell:example cassell$ composer update
  143. cassell:phptek-scraper cassell$ pwd ~/example/vendor/secret-company-example/phptek-scraper cassell:phptek-scraper cassell$ git status On branch

    master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean cassell:phptek-scraper cassell$
  144. None
  145. None
  146. None
  147. None
  148. None
  149. "require": {
 "php": ">=5.4.0",
 "fabpot/goutte": "^2.0.4",
 "secret-company-example/example-cache-package" : "^1.0.0"
 }

  150. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$
  151. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add .
  152. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper"
  153. cassell:phptek-scraper cassell$ git status On branch master Your branch is

    up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper" [master 003abe2] added caching to Scraper 2 files changed, 42 insertions(+), 2 deletions(-) cassell:phptek-scraper cassell$
  154. 2 files changed, 42 insertions(+), 2 deletions(-) cassell:phptek-scraper cassell$

  155. cassell:phptek-scraper cassell$

  156. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master
  157. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m "2.0.0"
  158. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done.

    Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m “2.0.0" cassell:phptek-scraper cassell$ git push origin --tags Counting objects: 1, done. Writing objects: 100% (1/1), 166 bytes | 0 bytes/s, done. Total 1 (delta 0), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git * [new tag] 2.0.0 -> 2.0.0
  159. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “dev-master“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  160. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^2.0.0“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
  161. <?php
 
 require_once '../vendor/autoload.php';
 
 class SpeakerCache extends SecretCorporation\ExampleCache\Cache
 {


    
 }
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client(), new SpeakerCache());
 
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);
  162. None
  163. None
  164. Private Package Update Worlflow

  165. 1. Switch to “dev-master” Private Package Update Worlflow

  166. 1. Switch to “dev-master” 2. Make changes to package Private

    Package Update Worlflow
  167. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass Private Package Update Worlflow
  168. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version Private Package Update Worlflow
  169. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags Private Package Update Worlflow
  170. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* Private Package Update Worlflow
  171. 1. Switch to “dev-master” 2. Make changes to package 3.

    Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* 7. Composer update Private Package Update Worlflow
  172. https://www.flickr.com/photos/protohiro/

  173. https://www.flickr.com/photos/protohiro/

  174. Satis Improvements

  175. • Cron the rebuild Satis Improvements

  176. • Cron the rebuild • Git hook for rebuild Satis

    Improvements
  177. • Cron the rebuild • Git hook for rebuild •

    Github Webhook Satis Improvements
  178. * * * * * cd /path/to/satis && test -f

    web/rebuild && rm web/ rebuild && php bin/satis build -q satis.json ./web <?php // web/postcommit.php touch(__DIR__.'/rebuild'); Jérôme Tamarelle @GromNaN
  179. Satis Improvements • Cron the rebuild • Git hook for

    rebuild • Github Webhook • Mirror Github
  180. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [


    {
 "type": "vcs",
 "url": "git@github.com:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-cache-package.git"
 },
 { "type": “composer”, "url": “https://packagist.org" }
 ], "require": { "monolog/monolog": “*", "aws/aws-sdk-php": "*", "phpunit/phpunit": "*" }, "require-dependencies": true
 
 "require-all": true,
 

  181. Securing Satis

  182. • No Credentials in Code Securing Satis

  183. • No Credentials in Code • Firewall Securing Satis

  184. • No Credentials in Code • Firewall • Use SSH

    Securing Satis
  185. • No Credentials in Code • Firewall • Use SSH

    • Use HTTPS Securing Satis
  186. • No Credentials in Code • Firewall • Use SSH

    • Use HTTPS • Generate to Secret Folder Securing Satis
  187. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “2.0.0“
 },
 "repositories":

    [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/6PeTZwIhRBOiSm
 }
 ]
 }
  188. ,
 hptek-scraper" : “2.0.0“
 server.example.com/6PeTZwIhRBOiSm


  189. None
  190. Sheraton Grand 5:00pm

  191. getcomposer.org

  192. None
  193. None
  194. None
  195. https://joind.in/13763 @alc277 andrewcassell.com

  196. Responsive Web Design Tomorrow 2pm

  197. None
  198. T hank Y ou!