Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Manage Private Internal Dependencies With Composer and Satis

Manage Private Internal Dependencies With Composer and Satis

In this talk we'll discuss how to set up and use Composer to pull open-source packages from public repositories on packagist.org. Then we'll learn how to create your own private/corporate package management server using Satis. And we'll go through an example of how to integrate your own private code packaging server into your daily development workflow.

Note: This presentation contained a fair amount of video which is lost on Speaker Deck

Andrew Cassell

May 20, 2015
Tweet

More Decks by Andrew Cassell

Other Decks in Programming

Transcript

  1. Manage Private Internal
    Dependencies with
    Composer and Satis

    View full-size slide

  2. $ whois cassell

    View full-size slide

  3. $ whois cassell
    Andrew Cassell
    @alc277

    View full-size slide

  4. DEPENDENCIES
    NDENCIES DEPENDE
    DEPENDENCI
    NCIES
    S D
    DEPENDENCIE
    NCIES
    NDENCIESDEPENDE

    View full-size slide

  5. DEPENDENCIES
    NDENCIES DEPENDE
    DEPENDENCI
    NCIES
    S D
    DEPENDENCIE
    NCIES
    NDENCIESDEPENDE

    View full-size slide

  6. GOOD ARTISTS COPY
    GOOD ARTISTS COPY
    GREAT ARTISTS STEAL
    GREAT ARTISTS STEAL

    View full-size slide

  7. GOOD ARTISTS COPY
    GOOD ARTISTS COPY
    GREAT ARTISTS STEAL
    GREAT ARTISTS STEAL
    https://nancyprager.wordpress.com/2007/05/08/good-poets-borrow-great-poets-steal/

    View full-size slide

  8. GOOD DEVELOPERS COPY
    GOOD DEVELOPERS COPY

    View full-size slide

  9. GOOD DEVELOPERS COPY
    GOOD DEVELOPERS COPY
    GREAT DEVELOPERS !
    DEVELOPERS! DEVELOPERS!
    USE COMPOSER
    GREAT DEVELOPERS !
    DEVELOPERS! DEVELOPERS!
    USE COMPOSER

    View full-size slide

  10. Dependency Manager

    View full-size slide

  11. Dependency Manager
    1. Download Project Libraries

    View full-size slide

  12. Dependency Manager
    1. Download Project Libraries
    2. Download Dependent Libraries

    View full-size slide

  13. Dependency Manager
    1. Download Project Libraries
    2. Download Dependent Libraries
    3. Verify Compatibility

    View full-size slide

  14. Dependency Manager
    1. Download Project Libraries
    2. Download Dependent Libraries
    3. Verify Compatibility
    4. Autoloader for ALL THE THINGS

    View full-size slide

  15. DEPENDENCIES
    NDENCIES DEPENDE
    DEPENDENCI
    NCIES
    S D
    DEPENDENCIE
    NCIES
    NDENCIESDEPENDE

    View full-size slide

  16. $ curl -sS https://getcomposer.org/installer | php
    $ sudo mv composer.phar /usr/local/bin/composer
    Installing Composer

    View full-size slide

  17. $ composer require /
    Adding a Dependency

    View full-size slide

  18. $ composer require aws/aws-sdk-php
    Adding a Dependency

    View full-size slide

  19. cassell$ composer require aws/aws-sdk-php
    Using version ^2.8 for aws/aws-sdk-php
    ./composer.json has been created

    View full-size slide

  20. cassell$ composer require aws/aws-sdk-php
    Using version ^2.8 for aws/aws-sdk-php
    ./composer.json has been created
    Loading composer repositories with package information
    Updating dependencies (including require-dev)
    - Installing symfony/event-dispatcher (v2.6.7)
    Loading from cache
    - Installing guzzle/guzzle (v3.9.3)
    Downloading: 100%
    - Installing aws/aws-sdk-php (2.8.5)
    Downloading: 100%

    View full-size slide

  21. {
    "require": {
    "aws/aws-sdk-php": "^2.8"
    }
    }
    composer.json

    View full-size slide

  22. {
    "require": {
    "aws/aws-sdk-php": "~2.8"
    }
    }
    composer.json

    View full-size slide

  23. {
    "require": {
    "aws/aws-sdk-php": “2.8.2"
    }
    }
    composer.json

    View full-size slide

  24. {
    "require": {
    "aws/aws-sdk-php": “2.8.2",
    "league/flysystem": "1.0.3",
    "monolog/monolog": "1.13.1"
    }
    }
    Adding libraries to composer.json

    View full-size slide

  25. Semantic Versioning
    http://semver.org/

    View full-size slide

  26. 1.0.0
    PATCH
    MINOR

    View full-size slide

  27. 1.0.0
    PATCH
    MAJOR MINOR

    View full-size slide

  28. backwards compatible bugfix?
    1.0.0

    View full-size slide

  29. 15 more backwards compatible
    bugfixes?
    1.0.1

    View full-size slide

  30. 1.0.16
    new feature that doesn’t break
    existing code?

    View full-size slide

  31. 1.1.0
    old code incompatible?

    View full-size slide

  32. {
    "require": {
    "aws/aws-sdk-php": “2.8.2",
    "league/flysystem": “1.0.3”,
    "monolog/monolog": “1.13.1"
    }
    }
    composer.json

    View full-size slide

  33. {
    "require": {
    "aws/aws-sdk-php": “~2.8.2”,
    "league/flysystem": “~1.0.0”,
    "monolog/monolog": “~1.13.0”
    }
    }
    composer.json

    View full-size slide

  34. ~2.8.2
    2.8.2 2.9.0

    View full-size slide

  35. ^2.8.2
    2.8.2 3.0.0

    View full-size slide

  36. {
    "require": {
    "aws/aws-sdk-php": “^2.8.2",
    "league/flysystem": “^1.0.0”,
    "monolog/monolog": “^1.13.0”
    }
    }
    composer.json

    View full-size slide

  37. CHECK IN
    CHECK IN

    View full-size slide

  38. /vendor
    Add /vendor to your .gitignore

    View full-size slide

  39. composer install
    composer update
    - or -

    View full-size slide

  40. composer install
    composer update
    - or -

    View full-size slide

  41. composer install
    composer update
    - or -

    View full-size slide

  42. {
    "require": {
    "aws/aws-sdk-php": “^2.8.2",
    "league/flysystem": “^1.0.0”,
    "monolog/monolog": “^1.13.0”
    },
    "require-dev": {

    "phpunit/phpunit": "^4.1.0"

    }
    }
    Dev Dependencies in composer.json

    View full-size slide

  43. {
    "require": {
    "aws/aws-sdk-php": “^2.8.2",
    "league/flysystem": “^1.0.0”,
    "monolog/monolog": “^1.13.0”
    },
    "autoload": {

    "psr-4": {"ProjectName\\": "classes/"}

    }
    }
    Autoloader in composer.json

    View full-size slide

  44. {
    "require": {
    "aws/aws-sdk-php": “^2.8.2",
    "league/flysystem": “^1.0.0”,
    "monolog/monolog": “^1.13.0”
    }
    }
    Where do these packages come from?

    View full-size slide

  45. https://packagist.org/

    View full-size slide

  46. http://thephpleague.com/

    View full-size slide

  47. XML is like violence.
    If it doesn’t solve your
    problem, you’re not using
    enough of it.

    View full-size slide

  48. {

    "require": {

    "fabpot/goutte": "^2.0.4",

    "symfony/yaml": "^2.6.7"

    }

    }
    composer.json

    View full-size slide

  49. $ composer install
    Loading composer repositories with package information
    Installing dependencies (including require-dev)
    - Installing react/promise (v2.2.0)
    Downloading: 100%
    - Installing guzzlehttp/streams (3.0.0)
    Downloading: 100%
    - Installing guzzlehttp/ringphp (1.0.7)
    Downloading: 100%
    - Installing guzzlehttp/guzzle (5.2.0)
    Downloading: 100%
    - Installing symfony/dom-crawler (v2.6.7)
    Downloading: 100%
    - Installing symfony/css-selector (v2.6.7)
    Downloading: 100%
    - Installing symfony/browser-kit (v2.6.7)
    Downloading: 100%
    - Installing fabpot/goutte (v2.0.4)
    Downloading: 100%
    - Installing symfony/yaml (v2.6.7)
    Downloading: 100%
    symfony/browser-kit suggests installing symfony/process ()
    Writing lock file
    Generating autoload files

    View full-size slide

  50. $ composer install
    Loading composer repositories with package information
    Installing dependencies (including require-dev)
    - Installing react/promise (v2.2.0)
    Downloading: 100%
    - Installing guzzlehttp/streams (3.0.0)
    Downloading: 100%
    - Installing guzzlehttp/ringphp (1.0.7)
    Downloading: 100%
    - Installing guzzlehttp/guzzle (5.2.0)
    Downloading: 100%

    View full-size slide



  51. require_once __DIR__ . “/vendor/autoload.php”;
    scraper.php

    View full-size slide



  52. require_once __DIR__ . “/vendor/autoload.php”;
    $client = new \Goutte\Client();


    $crawler = $client->request('GET', 'http://
    tek.phparch.com/speakers/');
    scraper.php

    View full-size slide



  53. require_once __DIR__ . “/vendor/autoload.php”;
    $client = new \Goutte\Client();


    $crawler = $client->request('GET', 'http://
    tek.phparch.com/speakers/');
    scraper.php

    View full-size slide


  54. require_once __DIR__ . "/vendor/autoload.php";

    use \Symfony\Component\DomCrawler\Crawler;


    $client = new \Goutte\Client();


    $crawler = $client->request('GET', 'http://tek.phparch.com/speakers/');


    $speakers = $crawler->filter('#speakerlist > div')->each(function (Crawler $node) {


    $speaker = [];

    $speaker["name"] = $node->filter('div.headshot > img')->attr("alt");

    $speaker["gravatar"] = $node->filter('div.headshot > img')->attr("src");

    $speaker["company"] = $node->filter('div.info > h4')->text();

    try {

    $speaker["twitter"] = $node->filter('div.info > h3 > a')->text();

    } catch (\Exception $e) {

    // might fail

    $speaker["twitter"] = "";

    }


    $speaker["talks"] = $node->filter('div.info > dl')->first()->siblings()->filter('dl')->each(function (Crawler $talkNode) {


    $talk = [];

    $talk['type'] = $talkNode->filter('dt > div')->eq(0)->text();

    $talk['level'] = $talkNode->filter('dt > div')->eq(1)->text();

    $talk['title'] = $talkNode->filter('dd > h5')->text();

    $texts = [];

    foreach ($talkNode->filter('dd')->getNode(0)->childNodes as $child) {

    if ($child instanceof DOMText) {

    $texts[] = trim($child->textContent);

    }


    }

    $talk['room'] = $texts[2];

    $talk['when'] = $texts[3];


    return $talk;


    });


    return $speaker;


    });



    $dumper = new \Symfony\Component\Yaml\Dumper();


    echo $dumper->dump(["speakers" => $speakers],999);


    View full-size slide

  55. {
    "require": {
    “company/secret-sauce“: “^4.0”
    },
    "repositories": [
    {
    "type": "vcs",
    "url": “[email protected]:company/secret-sauce.git”
    }
    ]
    }
    composer.json

    View full-size slide

  56. DEPENDENCY
    DEPENDENCY
    DEPENDENCY
    DEPENDENCY

    View full-size slide

  57. $ php bin/satis build

    View full-size slide

  58. {

    "require": {

    "league/plates": "^3.1.0"

    }

    }

    View full-size slide

  59. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “^1.0.0"

    },

    "repositories": [

    {

    "type": "composer",

    "url": “https://packageserver.example.com/“

    }

    ]

    }

    View full-size slide

  60. SATIS
    http://packageserver.example.com/

    View full-size slide

  61. {

    "name": "Secret Corporation Package Server",

    "homepage": "https://packageserver.example.com/",

    "repositories": [

    {

    "type": "vcs",

    "url": "[email protected]:cassell/example-phptek-scraper-package.git"

    },

    {

    "type": "vcs",

    "url": "[email protected]:cassell/example-cache-package.git"

    },

    {

    "type": "vcs",

    "url": "[email protected]:cassell/example-s3-wrapper-package.git"

    }

    ],


    "require-all": true,


    "archive": {

    "directory": "dist",

    "format": "tar",

    "prefix-url": “https://packageserver.example.com/“,
    "skip-dev": true
    }

    }

    View full-size slide

  62. "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]

    php bin/satis build satis.json web

    View full-size slide

  63. "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]


    View full-size slide

  64. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “^1.0.0"

    },

    "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]

    }

    View full-size slide


  65. require_once '../vendor/autoload.php';


    $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    // Create new Plates engine

    $templates = new League\Plates\Engine(__DIR__ . '/../templates');


    // Create a new template

    echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);


    View full-size slide


  66. require_once '../vendor/autoload.php';


    $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client());


    // Create new Plates engine

    $templates = new League\Plates\Engine(__DIR__ . '/../templates');


    // Create a new template

    echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);


    View full-size slide

  67. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “^1.0.0"

    },

    "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]

    }

    View full-size slide

  68. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “dev-master“

    },

    "repositories": [

    {

    "type": "composer",

    "url": "http://localhost:9088/"

    }

    ]

    }

    View full-size slide

  69. Loading composer repositories with package information
    Updating dependencies (including require-dev)
    - Removing secret-company-example/phptek-scraper (1.0.2)
    - Installing secret-company-example/phptek-scraper (dev-master ab88f08)
    Cloning ab88f0877781202929049bf3516b23ddb6c0fa12
    Writing lock file
    Generating autoload files
    cassell:example cassell$ composer update

    View full-size slide

  70. cassell:phptek-scraper cassell$ pwd
    ~/example/vendor/secret-company-example/phptek-scraper
    cassell:phptek-scraper cassell$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    nothing to commit, working directory clean
    cassell:phptek-scraper cassell$

    View full-size slide

  71. "require": {

    "php": ">=5.4.0",

    "fabpot/goutte": "^2.0.4",

    "secret-company-example/example-cache-package" : "^1.0.0"

    }

    View full-size slide

  72. cassell:phptek-scraper cassell$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes not staged for commit:
    (use "git add ..." to update what will be committed)
    (use "git checkout -- ..." to discard changes in working directory)
    modified: composer.json
    modified: src/SpeakerScraper.php
    no changes added to commit (use "git add" and/or "git commit -a")
    cassell:phptek-scraper cassell$

    View full-size slide

  73. cassell:phptek-scraper cassell$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes not staged for commit:
    (use "git add ..." to update what will be committed)
    (use "git checkout -- ..." to discard changes in working directory)
    modified: composer.json
    modified: src/SpeakerScraper.php
    no changes added to commit (use "git add" and/or "git commit -a")
    cassell:phptek-scraper cassell$ git add .

    View full-size slide

  74. cassell:phptek-scraper cassell$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes not staged for commit:
    (use "git add ..." to update what will be committed)
    (use "git checkout -- ..." to discard changes in working directory)
    modified: composer.json
    modified: src/SpeakerScraper.php
    no changes added to commit (use "git add" and/or "git commit -a")
    cassell:phptek-scraper cassell$ git add .
    cassell:phptek-scraper cassell$ git commit "added caching to Scraper"

    View full-size slide

  75. cassell:phptek-scraper cassell$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes not staged for commit:
    (use "git add ..." to update what will be committed)
    (use "git checkout -- ..." to discard changes in working directory)
    modified: composer.json
    modified: src/SpeakerScraper.php
    no changes added to commit (use "git add" and/or "git commit -a")
    cassell:phptek-scraper cassell$ git add .
    cassell:phptek-scraper cassell$ git commit "added caching to Scraper"
    [master 003abe2] added caching to Scraper
    2 files changed, 42 insertions(+), 2 deletions(-)
    cassell:phptek-scraper cassell$

    View full-size slide

  76. 2 files changed, 42 insertions(+), 2 deletions(-)
    cassell:phptek-scraper cassell$

    View full-size slide

  77. cassell:phptek-scraper cassell$

    View full-size slide

  78. cassell:phptek-scraper cassell$ git push origin master
    Counting objects: 5, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (4/4), done.
    Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done.
    Total 5 (delta 2), reused 0 (delta 0)
    To [email protected]:cassell/example-phptek-scraper-package.git
    ab88f08..003abe2 master -> master

    View full-size slide

  79. cassell:phptek-scraper cassell$ git push origin master
    Counting objects: 5, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (4/4), done.
    Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done.
    Total 5 (delta 2), reused 0 (delta 0)
    To [email protected]:cassell/example-phptek-scraper-package.git
    ab88f08..003abe2 master -> master
    cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m "2.0.0"

    View full-size slide

  80. cassell:phptek-scraper cassell$ git push origin master
    Counting objects: 5, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (4/4), done.
    Writing objects: 100% (5/5), 812 bytes | 0 bytes/s, done.
    Total 5 (delta 2), reused 0 (delta 0)
    To [email protected]:cassell/example-phptek-scraper-package.git
    ab88f08..003abe2 master -> master
    cassell:phptek-scraper cassell$ git tag -a 2.0.0 -m “2.0.0"
    cassell:phptek-scraper cassell$ git push origin --tags
    Counting objects: 1, done.
    Writing objects: 100% (1/1), 166 bytes | 0 bytes/s, done.
    Total 1 (delta 0), reused 0 (delta 0)
    To [email protected]:cassell/example-phptek-scraper-package.git
    * [new tag] 2.0.0 -> 2.0.0

    View full-size slide

  81. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “dev-master“

    },

    "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]

    }

    View full-size slide

  82. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “^2.0.0“

    },

    "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/"

    }

    ]

    }

    View full-size slide


  83. require_once '../vendor/autoload.php';


    class SpeakerCache extends SecretCorporation\ExampleCache\Cache

    {


    }


    $scraper = new SecretCorporation\Phptek\SpeakerScraper(new Goutte\Client(), new SpeakerCache());


    // Create new Plates engine

    $templates = new League\Plates\Engine(__DIR__ . '/../templates');


    // Create a new template

    echo $templates->render('page',["speakers" => $scraper->getSpeakers() ]);

    View full-size slide

  84. Private Package Update Worlflow

    View full-size slide

  85. 1. Switch to “dev-master”
    Private Package Update Worlflow

    View full-size slide

  86. 1. Switch to “dev-master”
    2. Make changes to package
    Private Package Update Worlflow

    View full-size slide

  87. 1. Switch to “dev-master”
    2. Make changes to package
    3. Package tests pass
    Private Package Update Worlflow

    View full-size slide

  88. 1. Switch to “dev-master”
    2. Make changes to package
    3. Package tests pass
    4. Tag new version
    Private Package Update Worlflow

    View full-size slide

  89. 1. Switch to “dev-master”
    2. Make changes to package
    3. Package tests pass
    4. Tag new version
    5. Push changes and tags
    Private Package Update Worlflow

    View full-size slide

  90. 1. Switch to “dev-master”
    2. Make changes to package
    3. Package tests pass
    4. Tag new version
    5. Push changes and tags
    6. Satis rebuild*
    Private Package Update Worlflow

    View full-size slide

  91. 1. Switch to “dev-master”
    2. Make changes to package
    3. Package tests pass
    4. Tag new version
    5. Push changes and tags
    6. Satis rebuild*
    7. Composer update
    Private Package Update Worlflow

    View full-size slide

  92. https://www.flickr.com/photos/protohiro/

    View full-size slide

  93. https://www.flickr.com/photos/protohiro/

    View full-size slide

  94. Satis Improvements

    View full-size slide

  95. • Cron the rebuild
    Satis Improvements

    View full-size slide

  96. • Cron the rebuild
    • Git hook for rebuild
    Satis Improvements

    View full-size slide

  97. • Cron the rebuild
    • Git hook for rebuild
    • Github Webhook
    Satis Improvements

    View full-size slide

  98. * * * * * cd /path/to/satis && test -f web/rebuild && rm web/
    rebuild && php bin/satis build -q satis.json ./web
    // web/postcommit.php
    touch(__DIR__.'/rebuild');
    Jérôme Tamarelle
    @GromNaN

    View full-size slide

  99. Satis Improvements
    • Cron the rebuild
    • Git hook for rebuild
    • Github Webhook
    • Mirror Github

    View full-size slide

  100. {

    "name": "Secret Corporation Package Server",

    "homepage": "https://packageserver.example.com/",

    "repositories": [

    {

    "type": "vcs",

    "url": "[email protected]:cassell/example-phptek-scraper-package.git"

    },

    {

    "type": "vcs",

    "url": "[email protected]:cassell/example-cache-package.git"

    },

    {
    "type": “composer”,
    "url": “https://packagist.org"
    }

    ],
    "require": {
    "monolog/monolog": “*",
    "aws/aws-sdk-php": "*",
    "phpunit/phpunit": "*"
    },
    "require-dependencies": true


    "require-all": true,


    View full-size slide

  101. Securing Satis

    View full-size slide

  102. • No Credentials in Code
    Securing Satis

    View full-size slide

  103. • No Credentials in Code
    • Firewall
    Securing Satis

    View full-size slide

  104. • No Credentials in Code
    • Firewall
    • Use SSH
    Securing Satis

    View full-size slide

  105. • No Credentials in Code
    • Firewall
    • Use SSH
    • Use HTTPS
    Securing Satis

    View full-size slide

  106. • No Credentials in Code
    • Firewall
    • Use SSH
    • Use HTTPS
    • Generate to Secret Folder
    Securing Satis

    View full-size slide

  107. {

    "require": {

    "league/plates": "^3.1.0",

    "secret-company-example/phptek-scraper" : “2.0.0“

    },

    "repositories": [

    {

    "type": "composer",

    "url": "https://packageserver.example.com/6PeTZwIhRBOiSm

    }

    ]

    }

    View full-size slide

  108. ,

    hptek-scraper" : “2.0.0“

    server.example.com/6PeTZwIhRBOiSm


    View full-size slide

  109. Sheraton Grand
    5:00pm

    View full-size slide

  110. getcomposer.org

    View full-size slide

  111. https://joind.in/13763
    @alc277
    andrewcassell.com

    View full-size slide

  112. Responsive Web Design
    Tomorrow 2pm

    View full-size slide