Design patterns

Architecture of large-scale Python applications. A tour of interesting design patterns in Python, showing their uses in a real project.

Transcript

1. Design patterns Architecture of large-scale Python applications
2. Marek Stępniowski http://stepniowski.com @mstepniowski
4. Gamma, et al. Design Patterns Time Hunt, Thomas The Pragmatic Programmer Martin Fowler Patterns of Enterprise Application Architecture Cormen, et al. Introduction to Algorithms Kent Beck Test-Driven Development Python Abelson, Sussman Structure and Interpretation of Computer Programs Chris Okasaki Purely Functional Data Structures My first language Stages of a programmer Level 9000 0
5. Gamma, et al. Design Patterns Hunt, Thomas The Pragmatic Programmer Cormen, et al. Introduction to Algorithms thon
6. Design pattern def. General reusable solution to a commonly occurring problem within a given context in software design. A design pattern is a description or template for how to solve a problem that can be used in many different situations.
7. Singleton
8. django.conf.settings
9. class Singleton { private static Singleton instance; private Singleton() {}; public static Singleton getInstance() { if (instance == null) instance = new Singleton(); return instance; } }
10. class Singleton(object): _instance = None def __new__(cls, *args, **kw): if not cls._instance: cls._instance = ( super(Singleton, cls) .__new__(cls, *args, **kw)) return cls._instance
11. Alan Shallowey James R. Trott
12. Design patterns are higher-order abstractions for program organization. Peter Norvig Machine code Source code Concept of procedure
13. Marek Stępniowski
14. Abstract Factory Factory Method Builder Prototype Singleton Adapter Bridge Composite Decorator Facade Flyweight Proxy Chain of Reponsibility Command Interpreter Iterator Mediator Memento Observer State Strategy Template Method Visitor 23 DP
15. Adapter Bridge Composite Decorator Facade Flyweight Proxy Memento Observer State Strategy Template Method Visitor 32 Multiton Object Pool RAII Front Controller Null Object Publish/Subscribe Blackboard Servant Specification CC
16. 80 Foreign Key Mapping Association Table Mapping Dependent Mapping Embedded Value Serialized LOB Single Table Inheritance Class Table Inheritance Concrete Table Inheritance Inheritance Mappers Metadata Mapping Database Session State Gateway Mapper Layer Subtype Separated Interface Registry Value Object Money Special Case Plugin Service Stub Record Set PoEAA
17. 86 Embedded Value Serialized LOB Single Table Inheritance Class Table Inheritance Concrete Table Inheritance Inheritance Mappers Metadata Mapping Separated Interface Registry Value Object Money Special Case Plugin Service Stub Record Set Model Template View Model View Presenter Model View ViewModel Layers KVO KVC inne
18. Design patterns Behavioral Structural Creational Architectural Concurrency
19. Money Model View Controller Prototype Smart Pointers? Procedures? Classes? Value Objects? Reference Counting? Closures? Asynchronous communication? Layers?
20. The first rule of Singletons is: you do not use Singletons The second rule of Singletons is: you do not use Singletons
21. django.conf.settings
22. class Borg: __shared_state = {} def __init__(self): self.__dict__ = self.__shared_state
23. class Singleton(type): def __init__(cls, name, bases, d): super(Singleton, cls).__init__( name, bases, d) cls._instance = None def __call__(cls, *args, **kw): if cls._instance is None: cls._instance = ( super(Singleton, cls) .__call__(*args, **kw)) return cls._instance class Settings(object): __metaclass__ = Singleton
24. # settings.py IM_A_CHARGIN_MAH_LAZER = True IM_A_FIRIN_MAH_LAZER = False tagline = '' # main.py import settings settings.tagline = 'Shoop da Whoop'
25. Multiton
26. import logging log = logging.getLogger( 'setjam.classification.movie') log.warning( 'New ep of Bold & Beautiful!') logging
27. Design patterns Formal Informal Invisible Peter Norvig
28. Singleton Invisible Informal Formal
29. Multiton Invisible Informal Formal
30. of original design patterns are either invisible or simplified in Python 15/23
31. First-class types: Abstract Factory, Factory Method, Flyweight, State, Proxy, Chain of Responsibility First-class functions: Command, Strategy, Template Method, Visitor Decorators: Mediator, Observer Modules: Singleton, Facade +Iterator
33. )))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) )))))))))))))))))))))) ))))))))
34. Adapter Abstract Factory Method Factory Template Method Null Object Plugin Strategy Observer State 9
35. a hypothetical company
36. SeriesMatcher().match(data) ManualSeriesMatcher().match(data, pattern, trusted_keys) MovieMatcher().match(subscription, data)
37. +match(4) ClassificationEngine +match(4) SeriesEngine +match(4) ManualSeriesEngine +match(4) MovieEngine +match(1) SeriesMatcher +match(4) ManualSeriesMatcher +match(2) MovieMatcher Adapter
38. class ClassificationEngine(object): pass class SeriesEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return SeriesMatcher().match(data) class ManualSeriesEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return ManualSeriesMatcher().match(data, pattern, trusted_keys) class MovieEngine(ClassificationEngine): def match(self, subscription, data, pattern, trusted_keys): return MovieMatcher().match(subscription, data)
39. class SeriesEngine(ClassificationEngine): def __init__(self): self.matcher = SeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): def __init__(self): self.matcher = ManualSeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(ClassificationEngine): def __init__(self): self.matcher = MovieMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(subscription, data)
40. Factory
41. class ClassificationEngine(object): matcher_class = None def __init__(self, matcher=None): self.matcher = (matcher if matcher else self.matcher_class()) class SeriesEngine(ClassificationEngine): matcher_class = SeriesMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): matcher_class = ManualSeriesMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(object): matcher_class = MovieMatcher def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(subscription, data)
42. MatcherFactory Series Factory Nicer Fact
43. class ClassificationEngine(object): def __init__(self, matcher=None): self.matcher = self.get_matcher() class SeriesEngine(ClassificationEngine): def get_matcher(self): return SeriesMatcher() def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data) class ManualSeriesEngine(ClassificationEngine): def get_matcher(self): return SeriesMatcher(defaults=settings.TRUSTED_KEYS) def match(self, subscription, data, pattern, trusted_keys): return self.matcher.match(data, pattern, trusted_keys) class MovieEngine(object): def get_matcher(self): return MovieMatcher() def match(self, subscription, data): return self.matcher.match(subscription, data)
44. Template Method
45. class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data): preprocessed_data = self.preprocess(subscription, data) filtered_data = self.filter(subscription, preprocessed_data) matched_data = self.match(subscription, subscription_data, filtered_data)) classified_data = self.merge(subscription, matched_data) postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data
46. class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data, timer): with timer.label('preprocess'): preprocessed_data = self.preprocess(subscription, data) with timer.label('filter'): filtered_data = self.filter(subscription, preprocessed_data) with timer.label('match'): matched_data = self.match(subscription, subscription_data, filtered_data)) with timer.label('merge'): classified_data = self.merge(subscription, matched_data, extra_priorities) with timer.label('postprocess'): postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data
47. Class-based generic views
48. +label(1) +dump(0) AbstractTimer +label(1) +dump(0) Timer +label(1) +dump(0) DummyTimer Client Do nothing Null Object
49. class Timer(object): def __init__(self, identifier=None): self.identifier, register = identifier, {} @contextmanager def label(self, label): t0 = time.time() yield t1 = time.time() self.register[label] = t1 - t0 def dump(self): r = dict(self.register) if self.identifier: r['_identifier'] = self.identifier return r class DummyTimer(Timer): def label(self, label): yield def dump(self): return {'_identifier': '<DummyTimer>'}
50. class ClassificationEngine(object): # ... def classify(self, subscription, subscription_data, data, timer=None): if timer is None: timer = DummyTimer() with timer.label('preprocess'): preprocessed_data = self.preprocess(subscription, data) with timer.label('filter'): filtered_data = self.filter(subscription, preprocessed_data) with timer.label('match'): matched_data = self.match(subscription, subscription_data, filtered_data)) with timer.label('merge'): classified_data = self.merge(subscription, matched_data, extra_priorities) with timer.label('postprocess'): postprocessed_data = self.postprocess(subscription, classified_data) return postprocessed_data
51. +run(1) +runAll(0) HarvesterCommand DisneyHarvester AmazonHarvester CrackleHarvester NetflixHarvester ITunesHarvester
52. def get_class(path): module_path, class_name = path.rsplit('.', 1) package_path = ([module_path.rsplit('.', 1)[0]] if '.' in module_path else None) module = __import__(module_path, fromlist=package_path) return getattr(module, class_name) # settings.py HARVESTERS = ( 'harvesters.disney_harvester.harvester.DisneyHarvester', 'harvesters.amazon_harvester.harvester.AmazonHarvester', 'harvesters.crackle_harvester.harvester.CrackleHarvester', 'harvesters.netflix_harvester.harvester.NetflixHarvester', 'harvesters.itunes_harvester.harvester.ITunesHarvester' ) Plugin
53. admin.site.register(MyModel, MyModelAdmin)
54. def check_and_fixup_data(new_data, old_data, fixup_func=fixup_data, flagging_func=flag_data): is_worse, report = flagging_func(old_data, new_data) if is_worse: new_data = fixup_func(new_data, old_data) is_worse, report = flagging_func(old_data, new_data) return (is_worse, new_data, report)
55. Strategy
56. sorted(iterable)
57. sorted(iterable, cmp=None, key=None, reverse=False)
58. +notify(*) Observer +notify(*) ObserverA +notify(*) ObserverB notifyObservers(*) for observer in observerCollection: observer.notify(*) +registerObserver(1) +unregisterObserver(1) +notifyObservers(*) observerCollection Observable Observer
59. django.core.signals django.db.models.signals
60. State
61. var Post = function (text, state) { this.initialize(text, state); }; _.extend(Post.prototype, { initialize: function (text, state) { this.text = text; this.state = state; }, getMaxCharacterCount: function () { this.state.getMaxCharacterCount(); }, getCharacterCount: function (text) { this.state.getCharacterCount(text); } });
62. var FacebookBehavior = function () {}; _.extend(FacebookBehavior.prototype, { getMaxCharacterCount: function () { return 450; }, getCharacterCount: function (text) { return text.length; } }); CH20 = 'xxxxxxxxxxxxxxxxxxxx'; var TwitterBehavior = function () {}; _.extend(TwitterBehavior.prototype, { getMaxCharacterCount: function () { return 140; }, getCharacterCount: function (text) { return text.replace(URL_REGEXP, CH20).length; } });
63. My God! It’s full of patterns!
64. A B
65. A B
66. A B C
67. A B C
68. A B C
69. Patternitis
70. Application architecture
71. Design patterns
72. No silver bullet!
73. Design patterns
74. Design patterns + TDD
75. Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code
76. Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code + 2nd pair of eyes
77. Design patterns + TDD + Seams Michael Feathers, Working Effectively with Legacy Code + 2nd pair of eyes + 3rd pair of eyes
78. Seems to work
79. Marek Stępniowski http://stepniowski.com @mstepniowski
80. http://stepniowski.com/p/patterns