How StackStorm Builds & Operates StackStorm

This is the slide deck from a talk I gave about the StackStorm CI/CD pipeline.

https://www.youtube.com/watch?v=nVXVqWnEdtE

Transcript

1. 1
   How StackStorm Builds & Runs
   StackStorm
   May 2015
   

   View Slide

2. Actions
   Automations
   Infrastructure – Cloud – Applications – Tools – Processes
   {.}
   {.}
   Sensors
   Audit
   

   View Slide

3. StackStorm Solution
   3
   Master
   Content Repo
   PLATFORM
   Web GUI CLI Chatops
   CLIENTS
   Sensor Plugins
   (inbound integrations)
   Action Plugins
   (outbound integrations)
   INTEGRATIONS
   .yml
   REST API Rules Engine Workflow Engine
   Audit…
   A
   M
   Q
   P
   m
   e
   s
   s
   a
   g
   e
   b
   u
   s
   A
   M
   Q
   P
   m
   e
   s
   s
   a
   g
   e
   b
   u
   s
   /
   /
   /
   /
   {*} IFTTT
   No N! issue
   No N! issue
   Loosely coupled; users
   abstract automation
   from integration
   Loosely coupled; users
   abstract automation
   from integration
   Workflow and rules
   enable simple scripts to
   be tied together like
   Legos. Automate &
   share pipelines.
   Workflow and rules
   enable simple scripts to
   be tied together like
   Legos. Automate &
   share pipelines.
   Polished – with GUI,
   ChatOps, CLI, RBAC,
   error handling, help &
   more
   Polished – with GUI,
   ChatOps, CLI, RBAC,
   error handling, help &
   more
   

   View Slide

4. Why now?
   4
   IT does matter. A lot.
   Orchestration can deliver agility
   More change, more tools, DevOps
   = new requirements for automation
   

   View Slide

5. What Problems Are We Solving?
   • Automated Testing (CI)
   – Integration testing on every commit to all supported
   versions
   • Packaging & Shipping (CD)
   – Distro specific packages on every successful build
   – Fully automated delivery to all repositories
   5
   

   View Slide

6. Private and confidential
   

   View Slide

7. View Slide

8. How Is It Built?
   • GitHub Sensor
   – GitHub post commit hooks are received by a GitHub
   sensor
   • ActionChains
   – All workflows are written using the sequential
   workflow runner - ActionChain
   • ActionTriggers
   – Workflows are tied together using the trigger that is
   emitted on completion of an action execution.
   • Rules
   – Rules are used to enforce workflow execution
   preconditions
   8
   

   View Slide

9. Other Technologies
   • StackStorm Datastore
   – Last build number, stable release branch
   • Consul
   – Service Discovery for build & packaging
   • AWS
   9
   

   View Slide

10. 1. GitHub Commit
    10
    On each merge in to the monitored branches, both the unit tests, and the Mistral
    Integration Tests are fired.
    

    View Slide

11. 2. Packaging for Staging
    11
    Successful unit tests fire off packaging of all components, including documentation,
    and ship to the staging environment.
    

    View Slide

12. 3. Deploy to Staging
    12
    Packaging workflows will trigger upgrades on the staging hosts of the appropriate
    distro.
    

    View Slide

13. 4. Smoke Test - Staging
    13
    On successful completion of the upgrade, basic smoke tests are ran.
    

    View Slide

14. 5. End to End Test - Staging
    14
    Successful completion of the smoke tests, will trigger the full end to end testing suite.
    This includes all documented examples and deployment scripts.
    

    View Slide

15. 6. Tag & Cut Release
    15
    Upon completion of end to end tests, the release is tagged and a new patch branch
    is created.
    

    View Slide

16. 7. Package & Deploy
    16
    The release triggers the production packaging workflow. This workflow takes the
    blessed packages and delivers them to the production repositories.
    

    View Slide

17. Why Build It Ourselves?
    • Eating Our Own Dog Food
    – Continuous validation of StackStorm as a CI/CD
    solution
    • The Ultimate Acceptance Testing
    • Bragging Rights
    17
    

    View Slide

18. What's Next?
    • Containers
    – Unit Tests & Packaging Moved to Containers
    • ChatOps Improvements
    – Full Two Way ChatOps Integration
    18
    

    View Slide

19. Want More Info?
    • St2 Continuous Delivery Integration Pack
    https://github.com/StackStorm/st2incubator/tree/master/packs/st2
    cd
    • IRC: Freenode #stackstorm
    • support@stackstorm.com
    19
    

    View Slide

20. Check Out Our new Meetup!
    Auto-Remediation and Event Driven Automation
    • http://www.meetup.com/Auto-Remediation-and-Event-Driven-Automation/
    • Thursday May 14, 2015 6:30pm
    • LinkedIn: 2029 Stierlin Ct, Mountain View, CA
    20
    

    View Slide

21. Thank You
    

    View Slide