From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site

(Presented with Oliver Wegner, OTTO)

In this session, we will show how OTTO, Germany’s largest online fashion retailer, used a system-of-systems approach to enable modular, parallel development of its ambitious shop relaunch. We’ll talk about the Otto.de platform’s macro and micro architecture, the modularization into autonomous subsystems, as well as the challenges, both technical and organizational, in integrating the parts into a consistent whole using simple Web Technologies. Topics covered also include cross-cutting concerns such as an asset server, the approach to platform engineering and quality assurance, aggregation of front-end components including caching strategies, and the management of APIs and media types.

A green field approach based on open source components, a distributed architecture instead of a monolith, cross-functional and agile engineering teams with a strong devops focus, continuous delivery and fast feedback as the new culture on product development - these are the characteristics of OTTO's new online world. Launched in October 2013 the new shop managed the traffic peak before Christmas successfully.

Transcript

1. From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site QCon, London 03/06/2014 Oliver Wegner, Otto.de, @olliwegner Stefan Tilkov, innoQ, @stilkov
2. 1. Reviewing architectures
3. Generic Architecture Review Results Building features takes too long Technical debt is well-known and not addressed Deployment is way too complicated and slow Replacement would be way too expensive Scalability has reached its limit Architectural quality has degraded “-ility” problems abound
4. http://www.flickr.com/photos/krissymayhew/5463349254, Krissy Mayhew Any architecture’s quality is directly proportional to the number of bottlenecks limiting its evolution, development, and operations
5. Conway’s Law “Organizations which design systems are constrained to produce systems which are copies of the communication structures of these organizations.” – M.E. Conway Organization → Architecture
6. Reversal 1 Any particular architecture approach constraints organizational options – i.e. makes some organizational models simple and others hard to implement. Architecture → Organization
7. Reversal 2 Choosing a particular architecture can be a means of optimizing for a desired organizational structure. Architecture → Organization
8. 2. Rebuilding Otto.de
9. e-Commerce Solutions & Technology Product 7 April 2014 Seite 9 Percentage of turnover For the last 15 years the E-Commerce business has become more important 4.200 Employees > 2.1 Billion € turnover > 2 Million items on Otto.de 80% turnover online
10. But why rebuild Otto.de?
11. Non-functional: Functional: Goals Scalable Simple Personalized Realtime Time To Market Data-Driven Test-Driven Fast Reliable Features
12. OTTO Backend Infrastructure How green is the green field? Product Information Management Customer Management OTTO E-Commerce Frontend Infrastructure Articles Orders Order Management Customer
13. Start of the project LHOTSE Technical system architecture Open Source as core technologies One Prototype to define the technology stack Project organization with autonomous teams Scrum as an agile development method
14. 3. A system-of-systems approach
15. Macro (technical) architecture Domain architecture
16. JRuby C# Scala Groovy Java Clojure
17. RDBMS NoSQL K/V RDBMS RDBMS/ DWH NoSQL DocDB
18. RDBMS NoSQL K/V RDBMS RDBMS/ DWH NoSQL DocDB Micro architecture
19. Persistence Logic UI Module A Module B Module C
20. System A Persistence Logic UI System B Persistence Logic UI System C Persistence Logic UI
21. 4. The Otto architecture
22. Buying Process – as you already know it Search Discover Assess Order Check Customer Journey
23. The E-Commerce Business Architecture – Vertical and Horizontal aspects of the product Otto.de 1 Website = 1 Product = 1 System = 1 Engineering Team ? Discover Search Assess Order Check Usability Webanalytics and Testing Online and Performance Marketing Platform Engineering
24. System architecture is vertical Search Product Order User After Sales UI UI UI UI UI …
25. User Auth UI UI After Sales UI 1 Team à N Systems 1 System à 1 Team Organizational aspects
26. RESTful Architecture Shared Nothing Vertical Design Data Governance Buy when non core Common Technologies Macro-Architecture Micro-Architecture Architecture rules
27. The 3 Faces of Product Ownership Technical Lead Project Lead Business Lead Product Owner
28. Project start with distributed teams Team Search Team Discover Team Order … Team Check But how to deal with frontend integration?
29. 5. Frontend Integration
30. Development Deployment Storage Backend call Edge integration Server-side integration options RPC REST RMI ESI Homegrown (Portal server) Build tools Chef, Puppet, … Asset pipeline Git/SVN submodules Gems Maven artifacts DB replication Feeds
31. Link Replaced link Client-side integration options Client call Magical integration concept Unobtrusive JS ROCA-style oEmbed SPA-style
32. Otto.de – detail view
33. Otto.de - Basket
34. Management of JS and CSS from each team Order Personalization User System View
35. Separate teams for horizontal aspects Team Discover Team Search Team Order Team Assetserver
37. Decoupling Versioned storage GitHub Local SCM Asset Server
38. 6. A/B-Testing
39. AB-Testing in a distributed environment – What is an AB- Test 99% 1%
40. Solution with a centralized framework every team has to include in their repository Backoffice UI for Managing Tests Persistence DB R E S T Vertical (e.g. Search) Pull Experiment Data from Backoffice User Request User Response Testing specific and Vertical independent logic DB
41. Solution with a dedicated Vertical and loose coupling Backoffice UI for Managing Tests Persistence DB R E S T Pull Data from Back- office Testing Vertical Testing Logic Persists all Tests DB Vertical (e.g. Search) Implement and Deliver Alternative A and B DB Request Response R E S T ESI-Includes Frontend- Proxy
42. 7. Conclusion
43. Results of the project LHOTSE Finished before schedule: October, 24th à 4 months earlier 2 years in total Scaled to >100 people Finished in budget Finished in quality Minimum Viable Product
44. Lessons learned in applying a system-of-systems approach Independent, autonomous systems for maximum decoupling Strict macro architecture rules Teams with their own decisions Be skeptical of “easy” solutions Address cross- functional concerns Minimize cross- functional concerns Minimize need for coordination Prefer “pull” to “push” sharing
45. innoQ Deutschland GmbH Krischerstr. 100 40789 Monheim am Rhein Germany Phone: +49 2173 3366-0 innoQ Schweiz GmbH info@innoq.com Gewerbestr. 11 CH-6330 Cham Switzerland Phone: +41 41 743 0116 www.innoq.com Ohlauer Straße 43 10999 Berlin Germany Phone: +49 2173 3366-0 Robert-Bosch-Straße 7 64293 Darmstadt Germany Phone: +49 2173 3366-0 Radlkoferstraße 2 D-81373 München Phone: +49 (0) 89 741185-270 Stefan Tilkov, @stilkov stefan.tilkov@innoq.com Phone: +49 170 471 2625