A Plug for Plugins (PyGotham 2017)

A Plug for Plugins (PyGotham 2017)

The philosophy of small, focused software has its merits, but for most of us, software is big. From your browser to your kernel, the size of these applications is lost on most of us. How do these programs even grow to this size?

In all of these cases, the answer involves an often-overlooked pattern: the plugin. Call them modules or extensions, if you’d prefer, but they are wildly successful. In fact, the only thing wider than the success of plugin-based architecture is the variety of implementations, especially in Python’s dynamic environment.

This talk covers the basis for plugins, reviews Python’s current offerings using examples, and provides guidance as to the bright future of plugin architecture.

B4bbc497062643a8913884e7aba305f2?s=128

Mahmoud Hashemi

October 08, 2017
Tweet

Transcript

  1. 6.
  2. 8.

    Plugins everywhere! □ Browser extensions □ Kernel modules □ Shells

    (.bashrc & .bash_profile) □ Good editors (from Sublime to emacs) □ So many more! When plugins work, you might not even notice.
  3. 9.

    Plugins! Up next: □ Why plugins exist □ Reasons to

    use plugins □ Using plugins in Python applications
  4. 10.

    Stages of development skills □ Sticking values together >>> 2

    + 2 □ Sticking lines together □ Sticking functions together □ Sticking modules together □ Sticking behaviors together (Unsticking those modules & functions)
  5. 11.

    When to use plugin architecture □ Reduced core size □

    Work around licensing issues □ Community development ▪ Establish an API for code integration ▪ Especially for applications
  6. 17.

    Pros and cons of conda’s plugins □ Pros ▪ Simple

    ▪ Dynamic loading of hooks ▪ Only requires standard library □ Cons ▪ Not systemized (copyable but not general) ▪ Not user-facing
  7. 19.

    pluginbase From the makers of Flask and Click, 3-step plugins:

    https://github.com/pluginbase/pluginbase
  8. 20.

    Pros and cons of pluginbase □ Pros ▪ Generalized ▪

    User-facing □ Cons ▪ No distribution story ▪ Path wrangling ▪ Attempts to: ➢ “Localize” modules ➢ Enable simultaneous loading of multiple versions of plugins
  9. 21.

    Let’s talk about modules □ Python’s built-in plugin system □

    Simple, but also not so much ▪ PEP 302 Finders and Loaders □ Global by design ▪ sys.modules
  10. 23.

    First: Entry Points One of many setuptools features: “Entry points

    are a simple way for distributions to ‘advertise’ Python functions for use by other packages.” For instance: import pkg_resources for entry_point in pkg_resources.iter_entry_points('label'): registry[entry_point.name] = entry_point.load() http://amir.rachum.com/blog/2017/07/28/python-entry-points/
  11. 28.

    Pros and cons of gather □ Pros ▪ Uses standard

    setuptools ▪ Developer-facing ▪ Distribution through pip ▪ Plugins remain independently usable/testable □ Cons ▪ Only user-facing if your users are developers
  12. 29.

    Other systems and use cases □ stevedore □ twisted.plugin □

    Mercurial extensions □ pytest plugins (pluggy) □ venusian □ straight.plugin □ pylint plugins □ flake8 plugins □ zope.component □ Django ▪ Apps ▪ Middlewares ▪ Command extensions □ SQLAlchemy dialects/DBAPIs □ Sphinx extensions □ Buildout extensions □ Pike □ Dectate and Reg
  13. 30.

    Wrapping up We learned that plugins are: □ Redistributable components

    for composing behavior □ The architecture for scaling development ▪ Internal / External ▪ Open / Closed-source □ Not all plugin implementations are the same. □ Still plenty of room for innovation!