Dynamic Code Patterns: Extending Your Applications with Plugins by Doug Hellmann

Python makes loading code dynamically easy, allowing you to configure and extend your application by discovering and loading extensions at runtime. This presentation will discuss the techniques for dynamic code loading used in several well-known applications and weigh the pros and cons of each approach.

PyCon 2013

Transcript

1. Dynamic Code Patterns: Extending Your Applications with Plugins Doug Hellmann – @doughellmann – PyCon 2013 1 Saturday, March 16, 13
2. What is a “Plugin”? • Loaded Dynamically • Extends Core • Possibly Unknown Source 2 Saturday, March 16, 13
3. Why Plugins? • Better API Abstraction • Reduce Core Dependencies • Strategy Pattern • Visitor Pattern • Indirect Code Contributions 3 Saturday, March 16, 13
4. • OpenStack Metering • Measures Clouds • Varied Billing Requirements • Deployers Extend and Customize Ceilometer 4 Saturday, March 16, 13
5. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
6. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
7. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
8. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
9. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
10. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
11. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
12. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
13. Ceilometer Design Compute Images Volumes Network Objects Notification Message Bus Event Listener Compute Pollsters Central Pollsters Collector Meter Data Message Bus 5 Saturday, March 16, 13
14. Ceilometer Plugins • Message Bus • Receiving Notifications • Polling Compute • Polling Other • Storage 6 Saturday, March 16, 13
15. Research • Blogofile • Sphinx • Mercurial • cliff • virtualenvwrapper • Nose • Trac • Django • Pyramid • SQLAlchemy • Diamond • Nova 7 Saturday, March 16, 13
16. Discovery Explicit Scan File Import Reference Mercurial Diamond Blogofile Mercurial Django Pyramid Sphinx Nova Trac Nose cliff virtualenvwrapper 8 Saturday, March 16, 13
17. Enabling Explicit Implicit Django Pyramid SQLAlchemy Blogofile Mercurial Trac Sphinx virtualenvwrapper cliff 9 Saturday, March 16, 13
18. Importing Custom pkg_resources Django Pyramid Sphinx Diamond Nova Nose SQLAlchemy Blogofile Trac Nose SQLAlchemy Blogofile cliff virtualenvwrapper 10 Saturday, March 16, 13
19. Integration Fine Coarse Prompt Inspect Nose virtualenvwrapper Sphinx Trac cliff Diamond Mercurial Blogofile Pyramid Django 11 Saturday, March 16, 13
20. Integration Fine Coarse Prompt Inspect Nose virtualenvwrapper Sphinx Trac cliff Diamond Mercurial Blogofile Pyramid Django 12 Saturday, March 16, 13
21. API Enforcement Convention Base Class / Interface Blogofile Mercurial Django Sphinx Pyramid virtualenvwrapper Nose (optional) Trac (interface) Diamond cliff (abc) 13 Saturday, March 16, 13
22. Invocation Driver Dispatcher Iterator SQLAlchemy Nova Mercurial Django Pyramid cliff Nose Diamond virtualenvwrapper 14 Saturday, March 16, 13
23. Discovery / Importing • Entry Points • Distribute and pkg_resources • Be Consistent 15 Saturday, March 16, 13
24. Enabling • Explicit disabling • Automatic disabling 16 Saturday, March 16, 13
25. Integration • Fine • Inspect • Application Owns Relationship 17 Saturday, March 16, 13
26. API Enforcement • Abstract Base Classes • Duck Typing 18 Saturday, March 16, 13
27. Invocation • Storage – Driver • Notifications – Dispatcher • Pollsters – Iterator 19 Saturday, March 16, 13
28. stevedore • Implements Patterns • Wraps pkg_resources 20 Saturday, March 16, 13
29. NamedExtensionManager • Multiple Plugins • Only Loads Named Plugins • map() 21 Saturday, March 16, 13
30. EnabledExtensionManager • Multiple Plugins • Checks Each with Function on Load • map() 22 Saturday, March 16, 13
31. DispatchExtensionManager • Multiple Plugins • Invokes Subset on map() 23 Saturday, March 16, 13
32. DriverManager • Single Plugin • Direct Access 24 Saturday, March 16, 13
33. Dynamic Code Patterns: Extending Your Applications with Plugins Doug Hellmann – @doughellmann – PyCon 2013 http://packages.python.org/distribute/pkg_resources.html https://github.com/dreamhost/stevedore https://launchpad.net/ceilometer http://doughellmann.com 25 Saturday, March 16, 13