ORCONF2016 LibreCores CI. Project Overview

This presentation is a status update regarding LibreCores CI, which is a part of LibreCores - Community Hub for open-source hardware projects. The project is in the early prototyping stage now.

https://www.librecores.org/

Transcript

1. LibreCores Free and Open Digital Hardware LibreCores CI. Project Overview Bologna, October 29th, 2016 Oleg Nenashev (@oleg_nenashev) Contributor, LibreCores
2. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •LibreCores CI •What’s inside? Vision •Future work Disclaimer: the project is in early prototyping stage 2 Agenda @oleg_nenashev oleg-nenashev Speaker LibreCores Free and Open Digital Hardware #automation, #jenkinsci, #hardware, #embedded, #cloudbees
3. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Contribute to OSS hardware ecosystem quality •Improve project contributor experience • Continuous integration of changes • Pull Request builders, build status reporting • Result reporting in UI (unit test, coverage, parsed logs) •Improve project maintainer experience • Easy CI management and configuration • No need to host internal Jenkins instance •Improve project user experience • Public and easily accessible project health status 3 LibreCores Project Automation
4. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Automation service for projects being hosted on LibreCores •Build and test automation •Integration testing of OSS hardware projects and toolchains •Powered by Jenkins (see FAQ) 4 LibreCores CI
5. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Limited resource quotas •Not a good place for hundreds of simulations… •We cannot scale much there •No SLAs for reasonably- expensive versions •Not enough integrations •Hard to connect external peripherals 5 FAQ. Why NOT Travis/Appveyor/etc.?
6. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Free and open-source project •It’s a framework •Self-hosted setup • We can do “whatever we want” • E.g. remote build agents •Features we may need • Configuration as Code (Jenkins Pipeline) • Multi-branch pipeline • Tool Management Plugins • Security and shared access engines • General-purpose plugins •Success stories in the Area 6 FAQ. Why Jenkins?
7. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Hosting of project automation flows •Basic infrastructure: •Few nodes for light-weight tasks •Images of nodes with EDA tools •Automation bots •Integration with external infrastructure 7 LibreCores does provide….
8. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Dozens of build machines •Computing grids for simulations •FPGA grids for hardware testing •Licenses for all EDA tools 8 LibreCores does NOT* provide… *According to the current plan/resources
9. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 9 Vision. How it works? LibreCores CI GitHub SSO User management LibreCores Web Top-level Web UI Project repositories GitHub BitBucket SVN GitHub is a main target system, others may have limited integrations
10. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 10 Vision. Infrastructure Jenkins Docker (Swarm) Other Integrated Components Physical nodes FPGAs and other HW peripherals LibreCores CI Infrastructure Project-specific Infrastructure On-demand build agents (from predefined images) SSH, JNLP over VPN • Projects connect their infrastructure to LibreCores CI • Quick setup of nodes by project maintainers • Node access is restricted => dedicated hardware and licenses
11. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Connection options •Encrypted JNLP Connection •SSH for Jenkins-driven connection •VPN for large clusters •5-minute setup of a node with preconfigured environment 1. Install Java 7 + 2. Download executable JAR file 3. Launch command line generated by Jenkins 11 Vision. Connecting nodes
12. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 12 Example. Build definition
13. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 • Integrations: • Build statuses on LibreCores Web and project pages • E-mail notifications • Simple initial setup • All the code is in the project SCM (Jenkinsfile) • Jenkinsfile is trivial if you use a package manager • Pipeline libraries are provided by LibreCores CI 13 Vision. Integration with projects @Library(’org.librecores.ci.fusesoc') import org.librecores.ci.fusesoc.FuseSoCBuild FuseSoCBuild.run([ sim: ‘verilator’ corePath: ‘myprj.core’ ]);
14. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Multi-branch •Support of separate build histories for branches •Branches for pull requests •Common reporting features •Unit test reporting (Junit, TAP, etc.) •Build log parsing •Coverage report publishing •Build artifacts (e.g. packaged IP libs) 14 Vision. Build Flow features
15. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 • The most of the projects are publicly readable • Ownership-based security • Each project has owners coming from LibreCores Web or GitHub • Only owners can edit and launch Jenkins jobs • Owners can restrict access to their build agents 15 LibreCores Security
16. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Very first prototype •https://ci.librecores.org •Mostly empty •It’s too early to start deploying projects •Infra changes ahead (Pipeline libraries, etc.) •Usage policies still need to be defined •Many plans, not so much time 16 Current state
17. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Setup integration with LibreCores Web •Build definition libraries for FuseSoC •Setup sample flows for real-world demos •OpTiMSoC •openrisc/mor1kx •Start defining usage conditions for projects •Ensure the infrastructure can scale enough 17 Future work
18. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •Discussions: •Mailing list: dev@lists.librecores.org •IRC: #librecores on FreeNode •GitHub: •Code repository and task tracker •https://github.com/librecores/librecores-ci 18 Contributing
19. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 Q & A Q&A Contacts: • E-mail: dev@lists.librecores.org • IRC: #librecores @oleg_nenashev oleg-nenashev
20. @oleg_nenashev, #librecores, #orconf2016 October 8, 2016 •PhD, Hardware Engineering •St. Petersburg Polytechnic University •Contributor, LibreCores •Working on LibreCores CI •Jenkins project, Core Team •Jenkins core and community 20 > whoami -summary @oleg_nenashev oleg-nenashev