Engineering at Scale at Pipedrive

How and why Pipedrive’s engineering organization radically transformed itself throughout the past year by creating and implementing a innovative development model that will enable it to grow and deliver amazing features to our users across multiple offices, countries and developers in the years to come.

Transcript

1. Engineering at Scale at Pipedrive On launchpad, missions and tribes DIOGO CORREIA・PRINCIPAL ENGINEER
2. Pipedrive was founded back in 2010 by five guys in Estonia, and their mission was to make salespeople kickass. That is still our mission. A very, very brief story on Pipedrive
3. +150 Engineers 11 Tribes +300 Microservices in production +100 Deployments to production, every day 2 Data centers 4 Supported languages, JS, TS, PHP & Go Pipedrive Engineering in Numbers
4. Teams were independent & multidisciplinary groups people in Pipedrive. Each Team was about 9 people in total, and they designed, built & maintained their own product Features, services and libraries within Pipedrive. FROM MID 2013 TO MID 2018 Teams
5. The good stuff A Pipedrive Team, when formed with the right people in place, could be tremendously fast delivering value to the product with their own singular unencumbered focus. Usually teams in this phase were sources of innovation within the organization by bringing and implementing new tools and technologies. TEAMS CAN BE REALLY GREAT WHEN THEY ARE NEW Teams
6. The not so good stuff Teams became slower and slower as they became older. They lost their focus and drive to innovate as the became divided between supporting existing features and delivering new ones. Usually teams in this phase could often times become a slag with people leaving the company looking for greener pastures elsewhere and valuable know-how would get lost. TEAMS COME WITH A BEST BEFORE DATE Teams
7. A new model was needed As the company grew, and Teams got older and therefore slower, it was clear that the current model wasn’t going to cut it going forward. It simply wasn’t scaling up for the current and future needs of the company in terms of speed and capacity of execution.
8. Just like when teams were new and the people in it could focus, the new model would have to allow people to gain extremely focused periods of time, where they can dedicate themselves entirely. A new model would need to improve on INDIVIDUAL AND COLLECTIVE FOCUS Existing features can’t be left behind. The new model had to make the resolution of bugs and incidents to be done much faster than they were under the teams model. Large lists of bugs in the backlog needed to be a thing of the past. DEFECT TO RESOLUTION TIME Old feature code needs some love too, either through continuous refactoring, security patches & fixes, version upgrades & documentation. The new model also has to allow time for R&D and innovation to happen continuously. CONTINUOUS IMPROVEMENTS & INNOVATION People should be able to feel as motivated to work under this model, just like they were when teams are new. They should have space to grow, try their hand at leading their projects, as well as experimenting and trying new things. INDIVIDUAL MOTIVATION & ENGAGEMENT
9. Over the course of six months, from January to June of 2018 we, as whole company, designed, discussed, planned and adopted a new model which improves on all the previous points. IN PLACE SINCE MID 2018 Tribes
10. Just like Teams, Tribes are autonomous and independent. They own product features and roadmap. The biggest from difference from Teams is that they are much bigger, and their own sub-structures and supporting roles under it. The Components of the Tribe Model TRIBE Are where the Tribe’s new features or larger changes to existing features are designed and developed. TRIBE MISSIONS Happens every tuesday. Missions Pitches are presented company wide and any developer from any Tribe not currently engaged in a Mission can apply to the mission if he finds the objectives of the Mission interesting. MISSION PITCHING & MISSION CANDIDATES It’s where maintenance and support of existing features, R&D, refactoring, pre-mission preparation gets done, and where the Tribe’s On Call is handled. TRIBE LAUNCHPAD
11. 1 Engineering Manager 1 Lead Product Manager 1 Lead Product Designer 1 Solution Architect 3 Product Managers 3 Product Designers Tribe’s People 0..3 Mission Leads (Assigned per Mission) 1 Launchpad Lead (Assigned Monthly) 20 Developers
12. Just like with a new Team, Missions provide the focus that older Teams lost as they went by. Everyone who’s off to a Mission, is solely focused on the Mission and nothing else. Missions are always limited in time (usually 1 to 3 months), and its members are only temporarily assigned to it. WHERE NEW TRIBE FEATURES OR MAJOR CHANGES GET DONE Tribe Missions
13. PRODUCT MANAGER HAS AN IDEA The Tribe’s PMs come up with an hypothesis or idea and presents it to the Tribe’s EM PRODUCT DESIGNER IS SELECTED The Tribe’s Lead Product Designer assigns one Product Designer to the Mission PITCH IS SHOWN TO THE ENTIRE COMPANY The Mission Pitch is presented to the entire company. Devs. apply to the Mission AGREE, COMMIT AND GO! Every mission member discusses, plan & agrees to high level scope & goals CANDIDATE DEVELOPERS ARE SELECTED The EM and the Mission Lead make a final selection out of all the candidates MISSION PITCH GETS WRITTEN The Mission Lead, PM and Designer come up with a Pitch describing the Mission goals MISSION LEAD IS SELECTED The Tribe’s EM assigns a Senior Developer from the Tribe’s Launchpad as the Mission Lead Launching a Mission
15. During the Mission The only thing the Mission members have to do company wise is the Mission and only the Mission. No on call, no bug fixes, no support. ONE FOCUS: THE MISSION GOALS Preferably at the end of the week there's a demo/interview with one or more end users for validation and real user feedback in live. This is mostly true for Proof of Concept missions ITERATE WEEKLY: VALIDATE WITH END USERS Simple Mission Kanban Board with tasks "todo / in progress / done" lanes with no estimations; Short daily standups, and weekly retrospective to review what went nicely and not so nicely during the cycle. PROCESS: AS LIGHT AS IT CAN BE New features and changes are continuously deployed to to live production environment either hidden behind dev, beta or live feature flags. CONTINUOUS DELIVERY TO LIVE
16. Final Preparations for Mission Landing The Mission's Product Manager prepares documentation and gives training to the customer support on how the new feature is used. DOCS GET WRITTEN & SUPPORT IS TRAINED Missing polishing, load/security hardening and coverage is done in the final weeks of the Mission. POLISHING & HARDENING GET DONE Monitoring, alerting is configured and setup for all the services built or modified as part of the mission, so that we can be on top and mitigate quickly anything that impacts our users. MONITORING & ALERTING IS SETUP The Mission Lead meets with the Launchpad Lead a few weeks before the Landing date, and the Launchpad Lead makes sure the code, documentation and monitoring of the mission is in a state acceptable for it to land in the Tribe’s Launchpad ONE FINAL CHECK: CAN WE LAND?
17. As the Mission lands in the Tribe's Launchpad, the services and libraries built in the Mission are now owned and maintained by the Tribe as whole. All mission members are released from duty and join the Tribe’s Launchpad. THE MISSION LANDS IN THE LAUNCHPAD
18. Launchpad Has at minimum 5 to 6 developers, including the Launchpad Lead. Just like in a Mission, the Tribe’s Launchpad is minimal on process. Simple Kanban board, Daily standups, Bi-weekly retrospectives & planning. PROCESS: AS LIGHT AS IT CAN BE A lot of things happen at the Launchpad. While the Launchpad isn’t as focused as a Mission, a lot of interesting things happen here as they improve the Tribe experience with R&D, shared libraries, improvements and refactoring. DOES R&D, SMALL FEATURES, IMPROVEMENTS Everyone participates in a 1 week length paid on-call rotation for all the services currently owned by the Tribe that are not currently being modified in a Mission RESPONSIBLE FOR ON CALL, BUGS & SUPPORT It’s the place where everyone from the Tribe’s is at whenever they’re not out in a Mission. In here they’re doing any of the things that can be done as part of the Launchpad NOT IN A MISSION? YOU’RE IN LAUNCHPAD
19. We're still collecting data, but as a previous Engineering Team Lead it feels just like this right model to scale further scale Pipedrive Product & Engineering. It truly feels like that the pace of work, engagement and innovation has improved quite dramatically since we’ve implemented this model. TRIBES, MISSIONS & LAUNCHPAD
20. So why did we do this? Because... We want to deliver high quality features that our customers truly need and love, while making our product managers, designers and developers be and feel productive, focused and happy working at Pipedrive.
21. If something is not working, or can really be better, have the audacity to try something entirely new and different. There were a number of hurdles and challenges we as a company had to go through while implementing this new model, but it the end if feels like it really was worth it. Sometimes change is just what you need. CLOSING REMARKS
22. Thank you P.S. We’re hiring!