Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Travis CI

Travis CI

My slides from Øredev 2012

Konstantin Haase

November 07, 2012
Tweet

More Decks by Konstantin Haase

Other Decks in Technology

Transcript

  1. Travis CI
    I hear you like Pull Requests

    View Slide

  2. For the code is dark and full of errors.
    Game Of Thrones
    Home Box Office Inc.
    A Time Warner Company

    View Slide

  3. Konstantin Haase
    Open Source Developer
    github.com/rkh
    @konstantinhaase

    View Slide

  4. View Slide

  5. Maintainer of Sinatra
    Rack, Rubinius, Travis CI

    View Slide

  6. “Second to K&R, the
    most lagom technical
    book I’ve read.”
    Peter Cooper (Ruby Inside)
    Discount Code: AUTHD
    50% off ebook ($6.50)
    40% off print

    View Slide

  7. View Slide

  8. GitHub
    Oh My God

    View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. Time
    release branches
    master
    develop hot xes
    feature
    branches
    Feature
    for future
    release
    Tag
    1.0
    Major
    feature for
    next release
    From this point on,
    “next release”
    means the release
    after 1.0
    Severe bug
    xed for
    production:
    hot x 0.2
    Bug xes from
    rel. branch
    may be
    continuously
    merged back
    into develop
    Tag
    0.1
    Tag
    0.2
    Incorporate
    bug x in
    develop
    Only
    bug xes!
    Start of
    release
    branch for
    1.0
    Author: Vincent Driessen
    Original blog post: http://nvie.com/archives/323
    License: Creative Commons

    View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. View Slide

  23. One does not simply merge a Pull Request!
    The Lord of the Rings
    New Line Cinema
    A Time Warner Company

    View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. ravis CI

    View Slide

  32. View Slide

  33. CIaaS
    aka the “one click CI system”

    View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. View Slide

  49. View Slide

  50. Three different JVMs, GCC & clang, Six Erlang versions, Clojure, Go, Groovy, Three Node.js versions,
    Haskell, Four Perl versions, Four Python versions plus PyPy, Two PHP versions, Ten (!) different Ruby
    versions/implementations, Two Scala versions, anything that runs on Linux
    GUI and Browser Testing (headless X11, Firefox and Chrome), with PhantomJS, Selenium, etc.
    MySQL, PostgreSQL, MongoDB, CouchDB, Redis, Riak, RabbitMQ, Memcached, Cassandra, Neo4J,
    ElasticSearch, Kestrel, SQLite, full root access, custom build steps, 3rd-party PPAs
    Notifications per Email, IRC, GitHub, Campfire, HipChat, Flowdock, Pusher, Web hook
    more than
    15 Programming Languages
    with multiple implementations

    View Slide

  51. more than
    2,250,000 Test Suits
    25,000 Open Source Projects
    more than 17,000 Test Suits, 1,000 Private Projects

    View Slide

  52. more than
    2,250,000 Test Suits
    25,000 Open Source Projects
    for free

    View Slide

  53. 100% Open Source

    View Slide

  54. “What Github did for sharing code, Travis CI
    will do for testing code. The value of this
    service cannot be overstated.
    Travis CI is a paradigm shift.
    The world will be a better place when every
    open-source developer is running their test
    suite on Travis CI.”
    Brian Ford
    Lead Developer, Rubinius
    Engine Yard

    View Slide

  55. View Slide

  56. ← Home
    Travis - an experimental, distributed CI
    server on Heroku
    We had the I18n gem tested on runcoderun, a great service for open source projects to run continous integration
    on the web, and this was working very well for us. Unfortunately runcoderun was taken down.
    So I figured it should be easy to setup a few apps on Heroku and have them run the I18n test suite on different
    stacks (i.e. Ruby configurations on Heroku). It turns out it is not, apparently. Or I'm just too stupid to do it. Anyway
    I though I'd write things down so maybe someone else can pick this experiment up and push it a few steps farther.
    The current state is that Travis can automatically set up a single ci server and 3 ci runners on Heroku. The server
    app takes the ping from github and pings the 3 runners. Each of the runners forks and immediately returns from
    the parent (forking) process while the child process runs the build command and posts the result back to the
    server. The server stores the result and displays it on the builds index page.
    Everything's cool except that the runner does not work properly on neither aspen-mri-186 (i.e. Ruby 1.8.6) nor
    bamboo-mri-191 (i.e. Ruby 1.9.1). It seems to work fine on the bamboo-ree-187 stack though and I will leave it
    running for a while so we can see how it works out over a few commits to the I18n repository.
    2010-06-16

    View Slide

  57. ← Home
    Travis - a distributed build server tool
    for the Ruby community
    So, I’ve started playing with the idea of a distributed build server tool once again in last year’s December … and
    Travisis what I came up with.
    Status quo
    ■ Travis is currently running on http://travis.heroku.com and already notifies me about builds that pass or fail
    when someone pushes to those repositories I’ve registered for testing purposes.
    ■ The builds run on a virtual server that Julian Fischer has very kindly granted for this experiment.
    ■ The frontend is implemented using Backbone.js.
    ■ Build output and status information is “tailed” from the workers to the frontend (i.e. browsers) using
    websockets via Pusher. It is also pushed back to application (running on Heroku) using a standard REST API.
    All of these building blocks might change in the future, but here’s an overview of how they currently work together.
    And here’s a short screencast (the UI has changed a bit in the meanwhile but you’ll get the idea): 1:20 quick demo
    screencast.
    2011-02-05

    View Slide

  58. 5,000
    10,000
    15,000
    20,000
    25,000
    30,000
    2010-12 2011-03 2011-06 2011-09 2011-12 2012-03 2012-06 2012-09
    Active Projects
    Open Source Private

    View Slide

  59. 50,000
    100,000
    150,000
    200,000
    250,000
    300,000
    2011-03 2011-05 2011-07 2011-09 2011-11 2012-01 2012-03 2012-05 2012-07 2012-09
    Test Suit Executions per Month
    Open Source Private

    View Slide

  60. Java (1112)
    5%
    Python (2413)
    10%
    PHP (3319)
    13%
    JavaScript (3837)
    16%
    Ruby (11752)
    48%
    Ruby (11752) JavaScript (3837) PHP (3319) Python (2413) Java (1112)
    Erlang (371) C (362) Clojure (333) Haskell (310) Perl (232)
    Scala (201) Go (134) C++ (132) Other (60) Groovy (34)
    Open Source Projects by Language 2012-10-26

    View Slide

  61. 2012-10-26
    Python (43)
    4%
    PHP (83)
    9%
    JavaScript (47)
    5%
    Ruby (742)
    77%
    Ruby (742) JavaScript (47) PHP (83) Python (43) Java (17) Erlang (9)
    C (1) Clojure (12) Haskell (3) Scala (1)
    Private Projects by Language

    View Slide

  62. Ruby
    C++
    Scala
    Perl
    Erlang
    PHP
    Java
    Python
    Haskell
    Groovy
    C
    JavaScript
    Clojure
    Other
    Go
    72.7%
    75%
    83%
    66.7%
    65.1%
    64.7%
    61.4%
    44.4%
    58.9%
    90.3%
    88.3%
    87.7%
    87.3%
    85.4%
    85.3%
    79%
    79%
    78%
    76.3%
    76.3%
    74.1%
    74.1%
    70.5%
    61.2%
    Success Rate by Language 2012-10-26
    Open Source Private

    View Slide

  63. 20,000
    40,000
    60,000
    80,000
    100,000
    120,000
    11-10 11-12 12-02 12-04 12-06 12-08 12-10
    GitHub Events for Open Source
    Push Pull Request

    View Slide

  64. 20,000
    40,000
    60,000
    80,000
    100,000
    120,000
    12-04 12-06 12-08 12-10
    GitHub Events for Private Projects
    Push Pull Request

    View Slide

  65. scalability is not optional

    View Slide

  66. View Slide

  67. ci

    View Slide

  68. ci
    travis-ci.org

    View Slide

  69. ci
    travis-ci.org

    View Slide

  70. ci
    travis-ci.org
    web
    next.travis-ci.org

    View Slide

  71. ci
    travis-ci.org
    web
    next.travis-ci.org
    api
    api.travis-ci.org

    View Slide

  72. ci
    travis-ci.org
    web
    next.travis-ci.org
    api
    api.travis-ci.org

    View Slide

  73. ci
    web
    api

    View Slide

  74. ci
    web
    api
    postgres

    View Slide

  75. ci
    web
    api
    postgres
    hub

    View Slide

  76. ci
    web
    api
    postgres
    hub
    AMQP/Redis

    View Slide

  77. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener

    View Slide

  78. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener

    View Slide

  79. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine

    View Slide

  80. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin

    View Slide

  81. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin

    View Slide

  82. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin

    View Slide

  83. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    admin

    View Slide

  84. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    gate-
    keeper
    admin

    View Slide

  85. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    gate-
    keeper
    tasks
    admin

    View Slide

  86. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    cli
    enqueue
    gate-
    keeper
    tasks
    admin

    View Slide

  87. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    cli
    enqueue
    gate-
    keeper
    tasks
    admin
    billing

    View Slide

  88. View Slide

  89. 553

    View Slide

  90. View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. “When you don't create things,
    you become defined by your
    tastes rather than ability.
    Your tastes only narrow
    and exclude people.
    So create.”
    why the lucky stiff
    Ruby Art Project
    by Jonathan G.

    View Slide

  95. View Slide

  96. Thanks
    [email protected]
    @konstantinhaase
    github.com/rkh

    View Slide