Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Architectural Patterns in Machine Learning to Generate Sustainable Business Value

Architectural Patterns in Machine Learning to Generate Sustainable Business Value

In this talk, I aim to highlight common concerns with building and deploying Machine Learning solutions, and discuss various patterns that can be consistently used to deliver value sustainably and quickly across a broad range of industries and specific business problems.
We will dive into some common patterns such as building composable components for reusability and consistency, production readiness, optimizing for learning with deployed solutions and guidelines for investments in research, as well as discuss tradeoffs against different dimensions such as latency vs complexity, generalizability of modeling solutions vs problem specific optimization. You should walk away from this talk with some general heuristics and patterns that make decisions for your ML solutions simpler, easier to manage and more successful.


Nishan Subedi

March 22, 2021


  1. Architectural Patterns in Machine Learning Nishan Subedi VP, Algorithms @

    Overstock, Inc.
  2. Overstock.com

  3. What is a Pattern? Relationships between components which resolves a

    small system of interacting and conflicting forces, and is independent of all other forces in all possible contexts. - Components can be created independently. - Components can then be combined together. - Design is an error correcting process. - Good design is characterized by absence of misfits (painful to do something). Many subsets of forces work together to build a pattern.
  4. None
  5. None
  6. Driven by the Scientific Methodology (OODA Loop)

  7. Scientific Methodology (OODA Loop) applied to the ML Product life

  8. Decide Every action starts with a decision

  9. Experimentation requires flexibility to test new • Model architectures •

    Featurization & data aggregations • Optimization functions • Training data sampling • Evaluation metrics
  10. Changing environment as we Act on what we Decide

  11. Abstraction allows preserving context across changing environments https://www.tecton.ai/blog/what-is-a-feature-store/

  12. Production Forces: • Latency • Availability • Consistency • Operability

  13. Consistent model deployment abstraction https://www.tensorflow.org/tfx/serving/architecture

  14. Observe

  15. Forces at play during observation • A/B testing • Data

    drift • Hidden feedback loops • Feature erosion • Consistency of features across batch / real time contexts • Bias • Offline / online metric comparison
  16. Forms that facilitate Observation - Variant

  17. Designing for Observability

  18. Value = timely response to changing needs aka faster iterations

    through the OODA loop “Whoever can handle the quickest rate of change is the one who survives.” ― John Boyd
  19. Architect for ease of iteration on the OODA loop Encode

    good practices into the architecture, make observability a default.
  20. Reusability

  21. Pattern Language Each pattern forms a word or thought of

    a language rather than a prescriptive way to design or solve a problem. This language allows precise conversations in teams, organizations and the industry. Architecture allows you to transcend a specific context to communicate with others within similar forces.
  22. Organizational patterns No pattern is an isolated entity. Each pattern

    can exist in the world, only to the extent that it is supported by other patterns: the larger patterns in which it is embedded. You and what your architecture is part of your organization.
  23. Organizational patterns An organization is made of the conversations between

    its people - actions emerge from these. An organization’s ability to create language is synonymous with its ability to evolve.
  24. Organizational patterns Having an architecture gives you a language allows

    you to get faster iterations of your OODA loop since it allows to much more effectively shine light on the right aspects that you need to talk about.
  25. How to speed up your OODA loop? 1. Consistent architecture

    2. Encapsulation of higher order information through data refinery 3. Flexibility in optimization well integrated with productionization process 4. Variant identification and propagation through various stages 5. Codify life cycle of ML in the language & process of the company
  26. Thankyou! We are hiring: overstock.com/careers