How StackStorm Builds & Operates StackStorm

This is the slide deck from a talk I gave about the StackStorm CI/CD pipeline.

https://www.youtube.com/watch?v=nVXVqWnEdtE

Transcript

1. 1 How StackStorm Builds & Runs StackStorm May 2015
2. Actions Automations Infrastructure – Cloud – Applications – Tools – Processes {.} {.} Sensors Audit
3. StackStorm Solution 3 Master Content Repo PLATFORM Web GUI CLI Chatops CLIENTS Sensor Plugins (inbound integrations) Action Plugins (outbound integrations) INTEGRATIONS .yml REST API Rules Engine Workflow Engine Audit… A M Q P m e s s a g e b u s A M Q P m e s s a g e b u s / / / / {*} IFTTT No N! issue No N! issue Loosely coupled; users abstract automation from integration Loosely coupled; users abstract automation from integration Workflow and rules enable simple scripts to be tied together like Legos. Automate & share pipelines. Workflow and rules enable simple scripts to be tied together like Legos. Automate & share pipelines. Polished – with GUI, ChatOps, CLI, RBAC, error handling, help & more Polished – with GUI, ChatOps, CLI, RBAC, error handling, help & more
4. Why now? 4 IT does matter. A lot. Orchestration can deliver agility More change, more tools, DevOps = new requirements for automation
5. What Problems Are We Solving? • Automated Testing (CI) – Integration testing on every commit to all supported versions • Packaging & Shipping (CD) – Distro specific packages on every successful build – Fully automated delivery to all repositories 5
6. Private and confidential
8. How Is It Built? • GitHub Sensor – GitHub post commit hooks are received by a GitHub sensor • ActionChains – All workflows are written using the sequential workflow runner - ActionChain • ActionTriggers – Workflows are tied together using the trigger that is emitted on completion of an action execution. • Rules – Rules are used to enforce workflow execution preconditions 8
9. Other Technologies • StackStorm Datastore – Last build number, stable release branch • Consul – Service Discovery for build & packaging • AWS 9
10. 1. GitHub Commit 10 On each merge in to the monitored branches, both the unit tests, and the Mistral Integration Tests are fired.
11. 2. Packaging for Staging 11 Successful unit tests fire off packaging of all components, including documentation, and ship to the staging environment.
12. 3. Deploy to Staging 12 Packaging workflows will trigger upgrades on the staging hosts of the appropriate distro.
13. 4. Smoke Test - Staging 13 On successful completion of the upgrade, basic smoke tests are ran.
14. 5. End to End Test - Staging 14 Successful completion of the smoke tests, will trigger the full end to end testing suite. This includes all documented examples and deployment scripts.
15. 6. Tag & Cut Release 15 Upon completion of end to end tests, the release is tagged and a new patch branch is created.
16. 7. Package & Deploy 16 The release triggers the production packaging workflow. This workflow takes the blessed packages and delivers them to the production repositories.
17. Why Build It Ourselves? • Eating Our Own Dog Food – Continuous validation of StackStorm as a CI/CD solution • The Ultimate Acceptance Testing • Bragging Rights 17
18. What's Next? • Containers – Unit Tests & Packaging Moved to Containers • ChatOps Improvements – Full Two Way ChatOps Integration 18
19. Want More Info? • St2 Continuous Delivery Integration Pack https://github.com/StackStorm/st2incubator/tree/master/packs/st2 cd • IRC: Freenode #stackstorm • support@stackstorm.com 19
20. Check Out Our new Meetup! Auto-Remediation and Event Driven Automation • http://www.meetup.com/Auto-Remediation-and-Event-Driven-Automation/ • Thursday May 14, 2015 6:30pm • LinkedIn: 2029 Stierlin Ct, Mountain View, CA 20
21. Thank You