Manage Private Internal Dependencies With Composer and Satis

In this talk we'll discuss how to set up and use Composer to pull open-source packages from public repositories on packagist.org. Then we'll learn how to create your own private/corporate package management server using Satis. And we'll go through an example of how to integrate your own private code packaging server into your daily development workflow.

Note: This presentation contained a fair amount of video which is lost on Speaker Deck

Transcript

1. Manage Private Internal Dependencies with Composer and Satis
6. $ whois cassell
7. $ whois cassell Andrew Cassell @alc277
14. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE
15. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE
17. GOOD ARTISTS COPY GOOD ARTISTS COPY GREAT ARTISTS STEAL GREAT ARTISTS STEAL
18. GOOD ARTISTS COPY GOOD ARTISTS COPY GREAT ARTISTS STEAL GREAT ARTISTS STEAL https://nancyprager.wordpress.com/2007/05/08/good-poets-borrow-great-poets-steal/
20. GOOD DEVELOPERS COPY GOOD DEVELOPERS COPY
21. GOOD DEVELOPERS COPY GOOD DEVELOPERS COPY GREAT DEVELOPERS ! DEVELOPERS! DEVELOPERS! USE COMPOSER GREAT DEVELOPERS ! DEVELOPERS! DEVELOPERS! USE COMPOSER
24. Dependency Manager
25. Dependency Manager 1. Download Project Libraries
26. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries
27. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries 3. Verify Compatibility
28. Dependency Manager 1. Download Project Libraries 2. Download Dependent Libraries 3. Verify Compatibility 4. Autoloader for ALL THE THINGS
29. DEPENDENCIES NDENCIES DEPENDE DEPENDENCI NCIES S D DEPENDENCIE NCIES NDENCIESDEPENDE
31. $ curl -sS https://getcomposer.org/installer | php $ sudo mv composer.phar /usr/local/bin/composer Installing Composer
32. $ composer require <vendor>/<package> Adding a Dependency
33. $ composer require aws/aws-sdk-php Adding a Dependency
34. cassell$ composer require aws/aws-sdk-php Using version ^2.8 for aws/aws-sdk-php ./composer.json has been created
35. cassell$ composer require aws/aws-sdk-php Using version ^2.8 for aws/aws-sdk-php ./composer.json has been created Loading composer repositories with package information Updating dependencies (including require-dev) - Installing symfony/event-dispatcher (v2.6.7) Loading from cache - Installing guzzle/guzzle (v3.9.3) Downloading: 100% - Installing aws/aws-sdk-php (2.8.5) Downloading: 100%
38. { "require": { "aws/aws-sdk-php": "^2.8" } } composer.json
39. { "require": { "aws/aws-sdk-php": "~2.8" } } composer.json
40. { "require": { "aws/aws-sdk-php": “2.8.2" } } composer.json
41. { "require": { "aws/aws-sdk-php": “2.8.2", "league/flysystem": "1.0.3", "monolog/monolog": "1.13.1" } } Adding libraries to composer.json
42. Semantic Versioning http://semver.org/
43. 1.0.0
44. 1.0.0 PATCH
45. 1.0.0 PATCH MINOR
46. 1.0.0 PATCH MAJOR MINOR
47. 0.6.10
48. 1.0.0
49. backwards compatible bugfix? 1.0.0
50. 1.0.1
51. 15 more backwards compatible bugfixes? 1.0.1
52. 1.0.16
53. 1.0.16 new feature that doesn’t break existing code?
54. 1.1.0
55. 1.1.0 old code incompatible?
56. 2.0.0
57. { "require": { "aws/aws-sdk-php": “2.8.2", "league/flysystem": “1.0.3”, "monolog/monolog": “1.13.1" } } composer.json
58. { "require": { "aws/aws-sdk-php": “~2.8.2”, "league/flysystem": “~1.0.0”, "monolog/monolog": “~1.13.0” } } composer.json
59. ~2.8.2 2.8.2 2.9.0
62. ^
63. ^2.8.2 2.8.2 3.0.0
64. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” } } composer.json
66. CHECK IN CHECK IN
67. /vendor Add /vendor to your .gitignore
69. composer install composer update - or -
70. composer install composer update - or -
71. composer install composer update - or -
72. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” }, "require-dev": {
 "phpunit/phpunit": "^4.1.0"
 } } Dev Dependencies in composer.json
73. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” }, "autoload": {
 "psr-4": {"ProjectName\\": "classes/"}
 } } Autoloader in composer.json
74. { "require": { "aws/aws-sdk-php": “^2.8.2", "league/flysystem": “^1.0.0”, "monolog/monolog": “^1.13.0” } } Where do these packages come from?
76. https://packagist.org/
82. PHPUnit
83. PHPUnit
84. PHPUnit
87. http://thephpleague.com/
88. EXAMPLE
92. XML is like violence. If it doesn’t solve your problem, you’re not using enough of it.
97. {
 "require": {
 "fabpot/goutte": "^2.0.4",
 "symfony/yaml": "^2.6.7"
 }
 } composer.json
98. $ composer install Loading composer repositories with package information Installing dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100% - Installing symfony/dom-crawler (v2.6.7) Downloading: 100% - Installing symfony/css-selector (v2.6.7) Downloading: 100% - Installing symfony/browser-kit (v2.6.7) Downloading: 100% - Installing fabpot/goutte (v2.0.4) Downloading: 100% - Installing symfony/yaml (v2.6.7) Downloading: 100% symfony/browser-kit suggests installing symfony/process () Writing lock file Generating autoload files
99. $ composer install Loading composer repositories with package information Installing dependencies (including require-dev) - Installing react/promise (v2.2.0) Downloading: 100% - Installing guzzlehttp/streams (3.0.0) Downloading: 100% - Installing guzzlehttp/ringphp (1.0.7) Downloading: 100% - Installing guzzlehttp/guzzle (5.2.0) Downloading: 100%
100. scraper.php
101. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; scraper.php
102. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
103. 
 <?php
 
 require_once __DIR__ . “/vendor/autoload.php”; $client = new \Goutte\Client();
 
 $crawler = $client->request('GET', 'http:// tek.phparch.com/speakers/'); scraper.php
104. <?php
 
 require_once __DIR__ . "/vendor/autoload.php";
 use \Symfony\Component\DomCrawler\Crawler;
 
 $client = new \Goutte\Client();
 
 $crawler = $client->request('GET', 'http://tek.phparch.com/speakers/');
 
 $speakers = $crawler->filter('#speakerlist > div')->each(function (Crawler $node) {
 
 $speaker = [];
 $speaker["name"] = $node->filter('div.headshot > img')->attr("alt");
 $speaker["gravatar"] = $node->filter('div.headshot > img')->attr("src");
 $speaker["company"] = $node->filter('div.info > h4')->text();
 try {
 $speaker["twitter"] = $node->filter('div.info > h3 > a')->text();
 } catch (\Exception $e) {
 // might fail
 $speaker["twitter"] = "";
 }
 
 $speaker["talks"] = $node->filter('div.info > dl')->first()->siblings()->filter('dl')->each(function (Crawler $talkNode) {
 
 $talk = [];
 $talk['type'] = $talkNode->filter('dt > div')->eq(0)->text();
 $talk['level'] = $talkNode->filter('dt > div')->eq(1)->text();
 $talk['title'] = $talkNode->filter('dd > h5')->text();
 $texts = [];
 foreach ($talkNode->filter('dd')->getNode(0)->childNodes as $child) {
 if ($child instanceof DOMText) {
 $texts[] = trim($child->textContent);
 }
 
 }
 $talk['room'] = $texts[2];
 $talk['when'] = $texts[3];
 
 return $talk;
 
 });
 
 return $speaker;
 
 });
 
 
 $dumper = new \Symfony\Component\Yaml\Dumper();
 
 echo $dumper->dump(["speakers" => $speakers],999);

109. —
110. —
111. { "require": { “company/secret-sauce“: “^4.0” }, "repositories": [ { "type": "vcs", "url": “git@github.com:company/secret-sauce.git” } ] } composer.json
113. DEPENDENCY DEPENDENCY DEPENDENCY DEPENDENCY
118. $ php bin/satis build <configuration file> <build dir>
119. satis.json
124. EXAMPLE
127. {
 "require": {
 "league/plates": "^3.1.0"
 }
 }
133. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories": [
 {
 "type": "composer",
 "url": “https://packageserver.example.com/“
 }
 ]
 }
134. SATIS http://packageserver.example.com/
135. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-cache-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-s3-wrapper-package.git"
 }
 ],
 
 "require-all": true,
 
 "archive": {
 "directory": "dist",
 "format": "tar",
 "prefix-url": “https://packageserver.example.com/“, "skip-dev": true }
 }
136. "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 php bin/satis build satis.json web
137. "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]

138. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
139. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());
 
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

140. <?php
 
 require_once '../vendor/autoload.php';
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());
 
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

142. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^1.0.0"
 },
 "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
143. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “dev-master“
 },
 "repositories": [
 {
 "type": "composer",
 "url": "http://localhost:9088/"
 }
 ]
 }
144. Loading composer repositories with package information Updating dependencies (including require-dev) - Removing secret-company-example/phptek-scraper (1.0.2) - Installing secret-company-example/phptek-scraper (dev-master ab88f08) Cloning ab88f0877781202929049bf3516b23ddb6c0fa12 Writing lock file Generating autoload files cassell:example cassell$ composer update
145. cassell:phptek-scraper cassell$ pwd ~/example/vendor/secret-company-example/phptek-scraper cassell:phptek-scraper cassell$ git status On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean cassell:phptek-scraper cassell$
151. "require": {
 "php": ">=5.4.0",
 "fabpot/goutte": "^2.0.4",
 "secret-company-example/example-cache-package" : "^1.0.0"
 }
152. cassell:phptek-scraper cassell$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$
153. cassell:phptek-scraper cassell$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add .
154. cassell:phptek-scraper cassell$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper"
155. cassell:phptek-scraper cassell$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: composer.json modified: src/SpeakerScraper.php no changes added to commit (use "git add" and/or "git commit -a") cassell:phptek-scraper cassell$ git add . cassell:phptek-scraper cassell$ git commit "added caching to Scraper" [master 003abe2] added caching to Scraper 2 files changed, 42 insertions(+), 2 deletions(-) cassell:phptek-scraper cassell$
156. 2 files changed, 42 insertions(+), 2 deletions(-) cassell:phptek-scraper cassell$
157. cassell:phptek-scraper cassell$
158. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done. Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master
159. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done. Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m "2.0.0"
160. cassell:phptek-scraper cassell$ git push origin master Counting objects: 5, done. Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done. Total 5 (delta 2), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git ab88f08..003abe2 master -> master cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m “2.0.0" cassell:phptek-scraper cassell$ git push origin --tags Counting objects: 1, done. Writing objects: 100% (1/1), 166 bytes | 0 bytes/s, done. Total 1 (delta 0), reused 0 (delta 0) To git@github.com:cassell/example-phptek-scraper-package.git * [new tag] 2.0.0 -> 2.0.0
161. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “dev-master“
 },
 "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
162. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “^2.0.0“
 },
 "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/"
 }
 ]
 }
163. <?php
 
 require_once '../vendor/autoload.php';
 
 class SpeakerCache extends SecretCorporation\ExampleCache\Cache
 {
 
 }
 
 $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client(), new SpeakerCache());
 
 // Create new Plates engine
 $templates = new League\Plates\Engine(__DIR__ . '/../templates');
 
 // Create a new template
 echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);
166. Private Package Update Worlflow
167. 1. Switch to “dev-master” Private Package Update Worlflow
168. 1. Switch to “dev-master” 2. Make changes to package Private Package Update Worlflow
169. 1. Switch to “dev-master” 2. Make changes to package 3. Package tests pass Private Package Update Worlflow
170. 1. Switch to “dev-master” 2. Make changes to package 3. Package tests pass 4. Tag new version Private Package Update Worlflow
171. 1. Switch to “dev-master” 2. Make changes to package 3. Package tests pass 4. Tag new version 5. Push changes and tags Private Package Update Worlflow
172. 1. Switch to “dev-master” 2. Make changes to package 3. Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* Private Package Update Worlflow
173. 1. Switch to “dev-master” 2. Make changes to package 3. Package tests pass 4. Tag new version 5. Push changes and tags 6. Satis rebuild* 7. Composer update Private Package Update Worlflow
174. https://www.flickr.com/photos/protohiro/
175. https://www.flickr.com/photos/protohiro/
176. Satis Improvements
177. • Cron the rebuild Satis Improvements
178. • Cron the rebuild • Git hook for rebuild Satis Improvements
179. • Cron the rebuild • Git hook for rebuild • Github Webhook Satis Improvements
180. * * * * * cd /path/to/satis && test -f web/rebuild && rm web/ rebuild && php bin/satis build -q satis.json ./web <?php // web/postcommit.php touch(__DIR__.'/rebuild'); Jérôme Tamarelle @GromNaN
181. Satis Improvements • Cron the rebuild • Git hook for rebuild • Github Webhook • Mirror Github
182. {
 "name": "Secret Corporation Package Server",
 "homepage": "https://packageserver.example.com/",
 "repositories": [
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-phptek-scraper-package.git"
 },
 {
 "type": "vcs",
 "url": "git@github.com:cassell/example-cache-package.git"
 },
 { "type": “composer”, "url": “https://packagist.org" }
 ], "require": { "monolog/monolog": “*", "aws/aws-sdk-php": "*", "phpunit/phpunit": "*" }, "require-dependencies": true
 
 "require-all": true,
 

183. Securing Satis
184. • No Credentials in Code Securing Satis
185. • No Credentials in Code • Firewall Securing Satis
186. • No Credentials in Code • Firewall • Use SSH Securing Satis
187. • No Credentials in Code • Firewall • Use SSH • Use HTTPS Securing Satis
188. • No Credentials in Code • Firewall • Use SSH • Use HTTPS • Generate to Secret Folder Securing Satis
189. {
 "require": {
 "league/plates": "^3.1.0",
 "secret-company-example/phptek-scraper" : “2.0.0“
 },
 "repositories": [
 {
 "type": "composer",
 "url": "https://packageserver.example.com/6PeTZwIhRBOiSm
 }
 ]
 }
190. ,
 hptek-scraper" : “2.0.0“
 server.example.com/6PeTZwIhRBOiSm

192. Sheraton Grand 5:00pm
193. getcomposer.org
197. https://joind.in/13763 @alc277 andrewcassell.com
198. Responsive Web Design Tomorrow 2pm
200. T hank Y ou!