Tweet
Tweet

Slide 1

Slide 1 text

Design patterns Architecture of large-scale Python applications

Slide 2

Slide 2 text

Marek Stępniowski http://stepniowski.com @mstepniowski

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Gamma, et al. Design Patterns Time Hunt, Thomas The Pragmatic Programmer Martin Fowler Patterns of Enterprise Application Architecture Cormen, et al. Introduction to Algorithms Kent Beck Test-Driven Development Python Abelson, Sussman Structure and Interpretation of Computer Programs Chris Okasaki Purely Functional Data Structures My first language Stages of a programmer Level 9000 0

Slide 5

Slide 5 text

Gamma, et al. Design Patterns Hunt, Thomas The Pragmatic Programmer Cormen, et al. Introduction to Algorithms thon

Slide 6

Slide 6 text

Design pattern def. General reusable solution to a commonly occurring problem within a given context in software design. A design pattern is a description or template for how to solve a problem that can be used in many different situations.

Slide 7

Slide 7 text

Singleton

Slide 8

Slide 8 text

django.conf.settings

Slide 9

Slide 9 text

class Singleton { private static Singleton instance; private Singleton() {}; public static Singleton getInstance() { if (instance == null) instance = new Singleton(); return instance; } }

Slide 10

Slide 10 text

class Singleton(object): _instance = None def __new__(cls, *args, **kw): if not cls._instance: cls._instance = ( super(Singleton, cls) .__new__(cls, *args, **kw)) return cls._instance

Slide 11

Slide 11 text

Alan Shallowey James R. Trott

Slide 12

Slide 12 text

Design patterns are higher-order abstractions for program organization. Peter Norvig Machine code Source code Concept of procedure

Slide 13

Slide 13 text

Marek Stępniowski

Slide 14

Slide 14 text

Abstract Factory Factory Method Builder Prototype Singleton Adapter Bridge Composite Decorator Facade Flyweight Proxy Chain of Reponsibility Command Interpreter Iterator Mediator Memento Observer State Strategy Template Method Visitor 23 DP

Slide 15

Slide 15 text

Adapter Bridge Composite Decorator Facade Flyweight Proxy Memento Observer State Strategy Template Method Visitor 32 Multiton Object Pool RAII Front Controller Null Object Publish/Subscribe Blackboard Servant Specification CC

Slide 16

Slide 16 text

80 Foreign Key Mapping Association Table Mapping Dependent Mapping Embedded Value Serialized LOB Single Table Inheritance Class Table Inheritance Concrete Table Inheritance Inheritance Mappers Metadata Mapping Database Session State Gateway Mapper Layer Subtype Separated Interface Registry Value Object Money Special Case Plugin Service Stub Record Set PoEAA

Slide 17

Slide 17 text

86 Embedded Value Serialized LOB Single Table Inheritance Class Table Inheritance Concrete Table Inheritance Inheritance Mappers Metadata Mapping Separated Interface Registry Value Object Money Special Case Plugin Service Stub Record Set Model Template View Model View Presenter Model View ViewModel Layers KVO KVC inne

Slide 18

Slide 18 text

Design patterns Behavioral Structural Creational Architectural Concurrency

Slide 19

Slide 19 text

Money Model View Controller Prototype Smart Pointers? Procedures? Classes? Value Objects? Reference Counting? Closures? Asynchronous communication? Layers?

Slide 20

Slide 20 text

The first rule of Singletons is: you do not use Singletons The second rule of Singletons is: you do not use Singletons

Slide 21

Slide 21 text

django.conf.settings

Slide 22

Slide 22 text

class Borg: __shared_state = {} def __init__(self): self.__dict__ = self.__shared_state

Slide 23

Slide 23 text

class Singleton(type): def __init__(cls, name, bases, d): super(Singleton, cls).__init__( name, bases, d) cls._instance = None def __call__(cls, *args, **kw): if cls._instance is None: cls._instance = ( super(Singleton, cls) .__call__(*args, **kw)) return cls._instance class Settings(object): __metaclass__ = Singleton

Slide 24

Slide 24 text

# settings.py IM_A_CHARGIN_MAH_LAZER = True IM_A_FIRIN_MAH_LAZER = False tagline = '' # main.py import settings settings.tagline = 'Shoop da Whoop'

Slide 25

Slide 25 text

Multiton

Slide 26

Slide 26 text

import logging log = logging.getLogger( 'setjam.classification.movie') log.warning( 'New ep of Bold & Beautiful!') logging

Slide 27

Slide 27 text

Design patterns Formal Informal Invisible Peter Norvig

Slide 28

Slide 28 text

Singleton Invisible Informal Formal

Slide 29

Slide 29 text

Multiton Invisible Informal Formal

Slide 30

Slide 30 text

of original design patterns are either invisible or simplified in Python 15/23

Slide 31

Slide 31 text

First-class types: Abstract Factory, Factory Method, Flyweight, State, Proxy, Chain of Responsibility First-class functions: Command, Strategy, Template Method, Visitor Decorators: Mediator, Observer Modules: Singleton, Facade +Iterator

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

)))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) ))))))))

Slide 34

Slide 34 text

Adapter Abstract Factory Method Factory Template Method Null Object Plugin Strategy Observer State 9

Slide 35

Slide 35 text

a hypothetical company

Slide 36

Slide 36 text

SeriesMatcher().match(data) ManualSeriesMatcher().match(data, pattern, trusted_keys) MovieMatcher().match(subscription, data)

Slide 37

Slide 37 text

+match(4) ClassificationEngine +match(4) SeriesEngine +match(4) ManualSeriesEngine +match(4) MovieEngine +match(1) SeriesMatcher +match(4) ManualSeriesMatcher +match(2) MovieMatcher Adapter

Slide 38

Slide 38 text

class ClassificationEngine(object): pass class SeriesEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return SeriesMatcher().match(data) class ManualSeriesEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return ManualSeriesMatcher().match(data, pattern, trusted_keys) class MovieEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return MovieMatcher().match(subscription, data)

Slide 39

Slide 39 text

class SeriesEngine(ClassificationEngine): def __init__(self): self.matcher = SeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): def __init__(self): self.matcher = ManualSeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(ClassificationEngine): def __init__(self): self.matcher = MovieMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(subscription, data)

Slide 40

Slide 40 text

Factory

Slide 41

Slide 41 text

class ClassificationEngine(object): matcher_class = None def __init__(self, matcher=None): self.matcher = (matcher if matcher else self.matcher_class()) class SeriesEngine(ClassificationEngine): matcher_class = SeriesMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): matcher_class = ManualSeriesMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(object): matcher_class = MovieMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(subscription, data)

Slide 42

Slide 42 text

MatcherFactory Series Factory Nicer Fact

Slide 43

Slide 43 text

class ClassificationEngine(object): def __init__(self, matcher=None): self.matcher = self.get_matcher() class SeriesEngine(ClassificationEngine): def get_matcher(self): return SeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): def get_matcher(self): return SeriesMatcher(defaults=settings.TRUSTED_KEYS) def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(object): def get_matcher(self): return MovieMatcher() def match(self, subscription, data): return self.matcher.match(subscription, data)

Slide 44

Slide 44 text

Template Method

Slide 45

Slide 45 text

class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data): preprocessed_data = self.preprocess(subscription, data) filtered_data = self.filter(subscription, preprocessed_data) matched_data = self.match(subscription, subscription_data, filtered_data)) classified_data = self.merge(subscription, matched_data) postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data

Slide 46

Slide 46 text

class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data, timer): with timer.label('preprocess'): preprocessed_data = self.preprocess(subscription, data) with timer.label('filter'): filtered_data = self.filter(subscription, preprocessed_data) with timer.label('match'): matched_data = self.match(subscription, subscription_data, filtered_data)) with timer.label('merge'): classified_data = self.merge(subscription, matched_data, extra_priorities) with timer.label('postprocess'): postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data

Slide 47

Slide 47 text

Class-based generic views

Slide 48

Slide 48 text

+label(1) +dump(0) AbstractTimer +label(1) +dump(0) Timer +label(1) +dump(0) DummyTimer Client Do nothing Null Object

Slide 49

Slide 49 text

class Timer(object): def __init__(self, identifier=None): self.identifier, register = identifier, {} @contextmanager def label(self, label): t0 = time.time() yield t1 = time.time() self.register[label] = t1 - t0 def dump(self): r = dict(self.register) if self.identifier: r['_identifier'] = self.identifier return r class DummyTimer(Timer): def label(self, label): yield def dump(self): return {'_identifier': ''}

Slide 50

Slide 50 text

class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data, timer=None): if timer is None: timer = DummyTimer() with timer.label('preprocess'): preprocessed_data = self.preprocess(subscription, data) with timer.label('filter'): filtered_data = self.filter(subscription, preprocessed_data) with timer.label('match'): matched_data = self.match(subscription, subscription_data, filtered_data)) with timer.label('merge'): classified_data = self.merge(subscription, matched_data, extra_priorities) with timer.label('postprocess'): postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data

Slide 51

Slide 51 text

+run(1) +runAll(0) HarvesterCommand DisneyHarvester AmazonHarvester CrackleHarvester NetflixHarvester ITunesHarvester

Slide 52

Slide 52 text

def get_class(path): module_path, class_name = path.rsplit('.', 1) package_path = ([module_path.rsplit('.', 1)[0]] if '.' in module_path else None) module = __import__(module_path, fromlist=package_path) return getattr(module, class_name) # settings.py HARVESTERS = ( 'harvesters.disney_harvester.harvester.DisneyHarvester', 'harvesters.amazon_harvester.harvester.AmazonHarvester', 'harvesters.crackle_harvester.harvester.CrackleHarvester', 'harvesters.netflix_harvester.harvester.NetflixHarvester', 'harvesters.itunes_harvester.harvester.ITunesHarvester' ) Plugin

Slide 53

Slide 53 text

admin.site.register(MyModel, MyModelAdmin)

Slide 54

Slide 54 text

def check_and_fixup_data(new_data, old_data, fixup_func=fixup_data, flagging_func=flag_data): is_worse, report = flagging_func(old_data, new_data) if is_worse: new_data = fixup_func(new_data, old_data) is_worse, report = flagging_func(old_data, new_data) return (is_worse, new_data, report)

Slide 55

Slide 55 text

Strategy

Slide 56

Slide 56 text

sorted(iterable)

Slide 57

Slide 57 text

sorted(iterable, cmp=None, key=None, reverse=False)

Slide 58

Slide 58 text

+notify(*) Observer +notify(*) ObserverA +notify(*) ObserverB notifyObservers(*) for observer in observerCollection: observer.notify(*) +registerObserver(1) +unregisterObserver(1) +notifyObservers(*) observerCollection Observable Observer

Slide 59

Slide 59 text

django.core.signals django.db.models.signals

Slide 60

Slide 60 text

State

Slide 61

Slide 61 text

var Post = function (text, state) { this.initialize(text, state); }; _.extend(Post.prototype, { initialize: function (text, state) { this.text = text; this.state = state; }, getMaxCharacterCount: function () { this.state.getMaxCharacterCount(); }, getCharacterCount: function (text) { this.state.getCharacterCount(text); } });

Slide 62

Slide 62 text

var FacebookBehavior = function () {}; _.extend(FacebookBehavior.prototype, { getMaxCharacterCount: function () { return 450; }, getCharacterCount: function (text) { return text.length; } }); CH20 = 'xxxxxxxxxxxxxxxxxxxx'; var TwitterBehavior = function () {}; _.extend(TwitterBehavior.prototype, { getMaxCharacterCount: function () { return 140; }, getCharacterCount: function (text) { return text.replace(URL_REGEXP, CH20).length; } });

Slide 63

Slide 63 text

My God! It’s full of patterns!

Slide 64

Slide 64 text

A B

Slide 65

Slide 65 text

A B

Slide 66

Slide 66 text

A B C

Slide 67

Slide 67 text

A B C

Slide 68

Slide 68 text

A B C

Slide 69

Slide 69 text

Patternitis

Slide 70

Slide 70 text

Application architecture

Slide 71

Slide 71 text

Design patterns

Slide 72

Slide 72 text

No silver bullet!

Slide 73

Slide 73 text

Design patterns

Slide 74

Slide 74 text

Design patterns + TDD

Slide 75

Slide 75 text

Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code

Slide 76

Slide 76 text

Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code + 2nd pair of eyes

Slide 77

Slide 77 text

Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code + 2nd pair of eyes + 3rd pair of eyes

Slide 78

Slide 78 text

Seems to work

Slide 79

Slide 79 text

Marek Stępniowski http://stepniowski.com @mstepniowski

Slide 80

Slide 80 text

http://stepniowski.com/p/patterns