Travis CI

My slides from Øredev 2012

Transcript

1. Travis CI
   I hear you like Pull Requests
   

   View Slide

2. For the code is dark and full of errors.
   Game Of Thrones
   Home Box Office Inc.
   A Time Warner Company
   

   View Slide

3. Konstantin Haase
   Open Source Developer
   github.com/rkh
   @konstantinhaase
   

   View Slide

4. View Slide

5. Maintainer of Sinatra
   Rack, Rubinius, Travis CI
   

   View Slide

6. “Second to K&R, the
   most lagom technical
   book I’ve read.”
   Peter Cooper (Ruby Inside)
   Discount Code: AUTHD
   50% off ebook ($6.50)
   40% off print
   

   View Slide

7. View Slide

8. GitHub
   Oh My God
   

   View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. Time
    release branches
    master
    develop hot xes
    feature
    branches
    Feature
    for future
    release
    Tag
    1.0
    Major
    feature for
    next release
    From this point on,
    “next release”
    means the release
    after 1.0
    Severe bug
    xed for
    production:
    hot x 0.2
    Bug xes from
    rel. branch
    may be
    continuously
    merged back
    into develop
    Tag
    0.1
    Tag
    0.2
    Incorporate
    bug x in
    develop
    Only
    bug xes!
    Start of
    release
    branch for
    1.0
    Author: Vincent Driessen
    Original blog post: http://nvie.com/archives/323
    License: Creative Commons
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. View Slide

23. One does not simply merge a Pull Request!
    The Lord of the Rings
    New Line Cinema
    A Time Warner Company
    

    View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. View Slide

30. View Slide

31. ravis CI
    

    View Slide

32. View Slide

33. CIaaS
    aka the “one click CI system”
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. Three different JVMs, GCC & clang, Six Erlang versions, Clojure, Go, Groovy, Three Node.js versions,
    Haskell, Four Perl versions, Four Python versions plus PyPy, Two PHP versions, Ten (!) different Ruby
    versions/implementations, Two Scala versions, anything that runs on Linux
    GUI and Browser Testing (headless X11, Firefox and Chrome), with PhantomJS, Selenium, etc.
    MySQL, PostgreSQL, MongoDB, CouchDB, Redis, Riak, RabbitMQ, Memcached, Cassandra, Neo4J,
    ElasticSearch, Kestrel, SQLite, full root access, custom build steps, 3rd-party PPAs
    Notifications per Email, IRC, GitHub, Campfire, HipChat, Flowdock, Pusher, Web hook
    more than
    15 Programming Languages
    with multiple implementations
    

    View Slide

51. more than
    2,250,000 Test Suits
    25,000 Open Source Projects
    more than 17,000 Test Suits, 1,000 Private Projects
    

    View Slide

52. more than
    2,250,000 Test Suits
    25,000 Open Source Projects
    for free
    

    View Slide

53. 100% Open Source
    

    View Slide

54. “What Github did for sharing code, Travis CI
    will do for testing code. The value of this
    service cannot be overstated.
    Travis CI is a paradigm shift.
    The world will be a better place when every
    open-source developer is running their test
    suite on Travis CI.”
    Brian Ford
    Lead Developer, Rubinius
    Engine Yard
    

    View Slide

55. View Slide

56. ← Home
    Travis - an experimental, distributed CI
    server on Heroku
    We had the I18n gem tested on runcoderun, a great service for open source projects to run continous integration
    on the web, and this was working very well for us. Unfortunately runcoderun was taken down.
    So I figured it should be easy to setup a few apps on Heroku and have them run the I18n test suite on different
    stacks (i.e. Ruby configurations on Heroku). It turns out it is not, apparently. Or I'm just too stupid to do it. Anyway
    I though I'd write things down so maybe someone else can pick this experiment up and push it a few steps farther.
    The current state is that Travis can automatically set up a single ci server and 3 ci runners on Heroku. The server
    app takes the ping from github and pings the 3 runners. Each of the runners forks and immediately returns from
    the parent (forking) process while the child process runs the build command and posts the result back to the
    server. The server stores the result and displays it on the builds index page.
    Everything's cool except that the runner does not work properly on neither aspen-mri-186 (i.e. Ruby 1.8.6) nor
    bamboo-mri-191 (i.e. Ruby 1.9.1). It seems to work fine on the bamboo-ree-187 stack though and I will leave it
    running for a while so we can see how it works out over a few commits to the I18n repository.
    2010-06-16
    

    View Slide

57. ← Home
    Travis - a distributed build server tool
    for the Ruby community
    So, I’ve started playing with the idea of a distributed build server tool once again in last year’s December … and
    Travisis what I came up with.
    Status quo
    ■ Travis is currently running on http://travis.heroku.com and already notifies me about builds that pass or fail
    when someone pushes to those repositories I’ve registered for testing purposes.
    ■ The builds run on a virtual server that Julian Fischer has very kindly granted for this experiment.
    ■ The frontend is implemented using Backbone.js.
    ■ Build output and status information is “tailed” from the workers to the frontend (i.e. browsers) using
    websockets via Pusher. It is also pushed back to application (running on Heroku) using a standard REST API.
    All of these building blocks might change in the future, but here’s an overview of how they currently work together.
    And here’s a short screencast (the UI has changed a bit in the meanwhile but you’ll get the idea): 1:20 quick demo
    screencast.
    2011-02-05
    

    View Slide

58. 5,000
    10,000
    15,000
    20,000
    25,000
    30,000
    2010-12 2011-03 2011-06 2011-09 2011-12 2012-03 2012-06 2012-09
    Active Projects
    Open Source Private
    

    View Slide

59. 50,000
    100,000
    150,000
    200,000
    250,000
    300,000
    2011-03 2011-05 2011-07 2011-09 2011-11 2012-01 2012-03 2012-05 2012-07 2012-09
    Test Suit Executions per Month
    Open Source Private
    

    View Slide

60. Java (1112)
    5%
    Python (2413)
    10%
    PHP (3319)
    13%
    JavaScript (3837)
    16%
    Ruby (11752)
    48%
    Ruby (11752) JavaScript (3837) PHP (3319) Python (2413) Java (1112)
    Erlang (371) C (362) Clojure (333) Haskell (310) Perl (232)
    Scala (201) Go (134) C++ (132) Other (60) Groovy (34)
    Open Source Projects by Language 2012-10-26
    

    View Slide

61. 2012-10-26
    Python (43)
    4%
    PHP (83)
    9%
    JavaScript (47)
    5%
    Ruby (742)
    77%
    Ruby (742) JavaScript (47) PHP (83) Python (43) Java (17) Erlang (9)
    C (1) Clojure (12) Haskell (3) Scala (1)
    Private Projects by Language
    

    View Slide

62. Ruby
    C++
    Scala
    Perl
    Erlang
    PHP
    Java
    Python
    Haskell
    Groovy
    C
    JavaScript
    Clojure
    Other
    Go
    72.7%
    75%
    83%
    66.7%
    65.1%
    64.7%
    61.4%
    44.4%
    58.9%
    90.3%
    88.3%
    87.7%
    87.3%
    85.4%
    85.3%
    79%
    79%
    78%
    76.3%
    76.3%
    74.1%
    74.1%
    70.5%
    61.2%
    Success Rate by Language 2012-10-26
    Open Source Private
    

    View Slide

63. 20,000
    40,000
    60,000
    80,000
    100,000
    120,000
    11-10 11-12 12-02 12-04 12-06 12-08 12-10
    GitHub Events for Open Source
    Push Pull Request
    

    View Slide

64. 20,000
    40,000
    60,000
    80,000
    100,000
    120,000
    12-04 12-06 12-08 12-10
    GitHub Events for Private Projects
    Push Pull Request
    

    View Slide

65. scalability is not optional
    

    View Slide

66. View Slide

67. ci
    

    View Slide

68. ci
    travis-ci.org
    

    View Slide

69. ci
    travis-ci.org
    

    View Slide

70. ci
    travis-ci.org
    web
    next.travis-ci.org
    

    View Slide

71. ci
    travis-ci.org
    web
    next.travis-ci.org
    api
    api.travis-ci.org
    

    View Slide

72. ci
    travis-ci.org
    web
    next.travis-ci.org
    api
    api.travis-ci.org
    

    View Slide

73. ci
    web
    api
    

    View Slide

74. ci
    web
    api
    postgres
    

    View Slide

75. ci
    web
    api
    postgres
    hub
    

    View Slide

76. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    

    View Slide

77. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    

    View Slide

78. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    

    View Slide

79. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    

    View Slide

80. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin
    

    View Slide

81. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin
    

    View Slide

82. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    admin
    

    View Slide

83. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    admin
    

    View Slide

84. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    gate-
    keeper
    admin
    

    View Slide

85. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    enqueue
    gate-
    keeper
    tasks
    admin
    

    View Slide

86. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    cli
    enqueue
    gate-
    keeper
    tasks
    admin
    

    View Slide

87. ci
    web
    api
    postgres
    hub
    AMQP/Redis
    listener
    status
    notifier
    log
    writer
    Pusher
    worker
    Virtual
    Machine
    Virtual
    Machine
    Virtual
    Machine
    cli
    enqueue
    gate-
    keeper
    tasks
    admin
    billing
    

    View Slide

88. View Slide

89. View Slide

90. View Slide

91. View Slide

92. View Slide

93. View Slide

94. “When you don't create things,
    you become defined by your
    tastes rather than ability.
    Your tastes only narrow
    and exclude people.
    So create.”
    why the lucky stiff
    Ruby Art Project
    by Jonathan G.
    

    View Slide

95. View Slide

96. Thanks
    konstantin@travis-ci.org
    @konstantinhaase
    github.com/rkh
    

    View Slide