How important it is to invest in a good software factory and automate the build process as much as possible ...if possible Nicolas Mas - [email protected] https://www.linkedin.com/in/nicolasmas

Acknowledgment “Alone we can do so little; together we can do so much” Eric Domingo Shuang Xia Lokesh Boghte Cedric Arnoult Andrew Tiu

What this talk won’t do ⬗ Elaborate on Yet Another Software Factory Definition ⬘ Or only if, as a consultant once told me: “a common language needs to be defined to understand each other” © Ajit Alwe ;-) ⬗ Enumerate a list of tools ⬗ Talk about any PM methodology ⬗ Show any code in action ⬗ Entertain you with funny pictures (sorry)

So what is it going to be? ⬗ A discussion about a real experience ⬗ back to basic: breaking down a complex problem into smaller chunks and finding solution ⬗ An example of software factory designed to handle a specific problem ⬗ Bonus: Discuss if it’s worth automating the build of the software factory as well as the automation of the build process (inception style) ⬗ B Tracks: if we really have time

Actually we have two topics here... ...and they are complementary 1. Investing in a good software factory 2. Automate the build process as much as possible

Investing in a good software factory: YES YOU CAN! Part One:

Quick Survey ⬗ What’s a software factory for you? ⬗ Are you using one?

For us, it all started with a unique context... ⬗ A Java Spring project ⬘ Team of 5 developers ◆ A “A Team”, with distinct and partially overlapping skills and experience ⬘ A delivery manager ◆ NOT from a technical background ⬘ A product owner on the client side ◆ NOT from a technical background ◆ NOT collocated ⬘ Tight deadline

...which induced ⬗ Small team with high pressure ⬗ Delivery manager does not speak the same language as the developers ⬗ Product owner in another country ⬗ Customer dev. processes are very different ⬗ Deliverables: ⬘ Code quality ⬘ Documentation ⬘ processes on top of the usuals ⬗ Start from scratch ⬗ A bit of innovation

So we needed to: Rethink the way we would build the product & Customize the process to build the product WHY?

To manage the risks generated by these factors & Make sure we deliver an amazing great product which will provide our customer the greatest satisfaction (hint: bullshit)

Moving on from there We can segregate our problem by vertical domains/aspects: ⬗ The product scope ⬗ Delivery management ⬗ Development process ⬘ Dependency Management ⬘ Code repository ⬘ Build process ⬘ Software Quality ⬘ Deployment process ⬗ Delivery process ⬗ Feedback ⬗ Communication

Mapping ⬗ The product scope ⬗ Delivery management ⬗ Development process ⬘ Dependency Management ⬘ Code repository ⬘ Build process ⬘ Software Quality ⬘ Deployment process ⬗ Delivery process ⬗ Feedback ⬗ Communication => Jira => Human (yes!) => Maven => GitHub => Jenkins => Sonar => Jenkins => Shell => Jira => Slack

A few remarks ⬗ The tools chosen are NOT the only one available: ⬘ Actually many !! ⬗ Rather than being a fixed structure, the Software Factory is a living system and must be design accordingly ⬗ Setup, test before the development starts ⬗ Complete transparency within the team and with the client

What did we gain doing this? ⬗ We are confident we can deliver anything, anytime ⬗ Newcomers to the project are not a potential problem ⬗ We minimized the impact of human factor ⬗ The delivery pipe is unlikely to be a blocking factor for the project ⬘ We won’t be able to put it on the tools if we fail

Is there anything else we can improve? ⬗ We designed our software factory in such a way that our main point of failure is the human factor How could we minimize that?

Automate as much as possible the build process: YES YOU CAN! (BUT...) Part Two:

What we can’t automate ⬗ code commit by the developers ⬗ stakeholders communication ⬘ at least partially (Slack technically allows us to do it, but can’t adapt to the stakeholder feedback).

What we decided not automate ⬗ Code review ⬗ Delivery ⬘ (YES this can be discussed … and YES I am aware of continuous delivery).

So finally, what we can automate The build process + UAT deployment “Jenkins is an open source continuous integration tool written in Java. Jenkins provides continuous integration services for software development. It is a server-based system running in a servlet container such as Apache Tomcat.”

⬗ A maven build is triggered (code pulled from github) ⬗ Shell scripts deploy on UAT targets ⬗ Postprocess action: email + Slack message

So finally, what we can automate The software quality - SonarQube “SonarQube is an open platform for continuous inspection of code quality”

Code coverage

Code Quality

So how did it go? ⬗ Anytime delivery was real ⬘ BUT complexity increased with time and integration with other systems (what a surprise!) ⬗ Code coverage tools sides effects ⬘ Require dev to code in a sometimes not so efficient manner ⬗ Client communication is better ⬘ Scope management ⬘ improved transparency Delivery happened no matter what

Was it worth it? YES

Bonus: Automating the creation and setup of the software factory Can or Cannot? Part Three:

Automating the whole thing ⬗ We want to be able to recreate from a master the whole software factory stack ⬘ In case of a crash (but can it be stateful?) ⬘ A new project is started in //, with similar constraints ⬗ We want to maintain a backup infra just in case ⬘ How do we keep them aligned? ⬘ AWS + local VBOx for example

Our current setup (manual) What we manually setup so far DB Stack CI Stack Staging Stack Github virtualbox vagrant Physical server Ubuntu server Ubuntu server Ubuntu server

Dev machine: MB Pro ⬗ Eclipse IDE + JDK 1.7 ⬗ Spring 4.0.4 + Apache CXF ⬗ Maven 3.0.3 ⬗ Tomcat 7 ⬗ Findbugs Staging Server: VM: Ubuntu 14.04 LTS ⬗ JDK 1.7.0_11 ⬗ Tomcat 7.0.37 DB Server: VM: Ubuntu 14.04 LTS ⬗ Oracle Entreprise Edition 11.2.0.3 ⬗ Gemfire CI Server: VM: Ubuntu 14.04 LTS ⬗ JDK 1.7.0_11 ⬗ Jenkins ⬗ Archiva ⬗ GitHub ⬗ Jira Agile ⬗ Jira Crucible ⬗ Sonar Cube 4.1.2 Infra: Ubuntu 14.04 LTS ⬗ Vagrant ⬗ Saltstack ⬗ packer ⬗ Virtualbox

Our target through automation DB Stack CI Stack Staging Stack Github virtualbox vagrant AWS Physical server WebApp - Software Factory Builder Ubuntu server Ubuntu server Ubuntu server

The Software Factory Builder ⬗ Node.js (locally hosted) front end ⬗ Underlying technologies (main ones) ⬘ Packer (OS baking) ⬘ SaltStack (provisioning) ⬘ Vagrant (VM management)

What it looks like ⬗ A node.js interface ⬘ You can imagine anything for the VM type (VMWare, digital ocean etc.)

What it looks like

What it looks like

What it looks like

Pros & Cons What’s cool.. ⬗ Traditionally, software factories are built once and used for projects for years. Stacks are updated, patched, reconfigured => lower and lower reliability. In our case, we can rebuild it every time ⬗ We can duplicate on different infrastructures local or in the cloud ⬗ Once defined, the build pattern is fast to execute What’s not… ⬗ Maintaining the setup and testing it before a “gold” version takes time and is complexe => it’s an upfront investment you might not benefit from ⬗ You can’t automate 100%, rather 90%. So you still need a bit of work

Bonus Track - Did you know ? “The mother of all demos” http://en.wikipedia. org/wiki/The_Mother_of_All_Demos ⬗ Douglas Engelbart's December 9, 1968 ⬗ 90-minute live public demonstration of the online system, NLS, they had been working on since 1962 ⬗ 1000 attendees and a demo of almost all features we use today ⬘ Mouse ⬘ hypertext, object addressing and dynamic file linking ⬘ shared-screen collaboration involving two persons at different sites communicating over a network with audio and video interface. http://web.stanford. edu/dept/SUL/library/extra4/sloan/mousesite/1968Demo.html

https://www.youtube.com/playlist?list=PL76DBC8D6718B8FD3 Watch it !!

Bye Bye ! Nicolas Mas - [email protected] https://www.linkedin.com/in/nicolasmas