Architectural Patterns in Machine Learning Nishan Subedi VP, Algorithms @ Overstock, Inc.

Overstock.com

What is a Pattern? Relationships between components which resolves a small system of interacting and conflicting forces, and is independent of all other forces in all possible contexts. - Components can be created independently. - Components can then be combined together. - Design is an error correcting process. - Good design is characterized by absence of misfits (painful to do something). Many subsets of forces work together to build a pattern.

No content

No content

Driven by the Scientific Methodology (OODA Loop)

Scientific Methodology (OODA Loop) applied to the ML Product life cycle

Decide Every action starts with a decision

Experimentation requires flexibility to test new ● Model architectures ● Featurization & data aggregations ● Optimization functions ● Training data sampling ● Evaluation metrics

Changing environment as we Act on what we Decide

Abstraction allows preserving context across changing environments https://www.tecton.ai/blog/what-is-a-feature-store/

Production Forces: ● Latency ● Availability ● Consistency ● Operability

Consistent model deployment abstraction https://www.tensorflow.org/tfx/serving/architecture

Observe

Forces at play during observation ● A/B testing ● Data drift ● Hidden feedback loops ● Feature erosion ● Consistency of features across batch / real time contexts ● Bias ● Offline / online metric comparison

Forms that facilitate Observation - Variant

Designing for Observability

Value = timely response to changing needs aka faster iterations through the OODA loop “Whoever can handle the quickest rate of change is the one who survives.” ― John Boyd

Architect for ease of iteration on the OODA loop Encode good practices into the architecture, make observability a default.

Reusability

Pattern Language Each pattern forms a word or thought of a language rather than a prescriptive way to design or solve a problem. This language allows precise conversations in teams, organizations and the industry. Architecture allows you to transcend a specific context to communicate with others within similar forces.

Organizational patterns No pattern is an isolated entity. Each pattern can exist in the world, only to the extent that it is supported by other patterns: the larger patterns in which it is embedded. You and what your architecture is part of your organization.

Organizational patterns An organization is made of the conversations between its people - actions emerge from these. An organization’s ability to create language is synonymous with its ability to evolve.

Organizational patterns Having an architecture gives you a language allows you to get faster iterations of your OODA loop since it allows to much more effectively shine light on the right aspects that you need to talk about.

How to speed up your OODA loop? 1. Consistent architecture 2. Encapsulation of higher order information through data refinery 3. Flexibility in optimization well integrated with productionization process 4. Variant identification and propagation through various stages 5. Codify life cycle of ML in the language & process of the company

Thankyou! We are hiring: overstock.com/careers