Building and Releasing a CakePHP Plugin

This talk will cover the ins and outs of building a CakePHP plugin, including deciding on how to test built code, what a typical release process looks like, and when to build something from scratch vs integrate with an existing PHP library. We’ll cover CakePHP Queue as a case-study in how the plugin building process happens in the core, and what practices can be transferred to your organization.

Transcript

1. Building and Releasing a CakePHP Plugin Jose Diaz-Gonzalez
2. What is this talk about? • Not much code, sorry • CakePHP, sorry • Some dinosaurs • Mostly drawings and anecdotes So what is this talk about? Unfortunately, I won’t be going into too much code, as I don’t think that makes for an interesting talk - I would probably also fall asleep, which maybe isn’t good for the talk. There will also be some CakePHP references, so I apologize in advance if you were hoping for a different framework. I’ll also reference some other frameworks as well, for the CakePHP haters tuning in :) I also apologize in advance for the lack of presentation transitions. I spent all the animation budget on the design work.
3. Why Not build a plugin? And other things I thought of during my fever dream last night So why wouldn’t we build a plugin? In what cases does it make sense to keep code within your application? Why should you just stop listening to the rest of this talk?
4. Code wont be shared One thing that I see happen very often is that folks create plugins just to create plugins. They extract logic from the main application, but the plugin never actually leaves their codebase. Plugins are a great way to separate business concerns, but more importantly, they are a great way to share code. Sharing code can mean a few things: - Open sourcing: This is one of the more popular ways, and something everyone here is familiar with. Everyone who has used cake bake before has generated an app that uses a plugin, and most people have likely installed one or more open source plugins from our plugins site or from the awesome-cakephp list. - Sharing internally between apps: This is more common for companies that tend to consult on the same kind of applications, and wish to reduce the amount of work necessary between customers. This can also be done at companies with a business product made of a set of services, where each service is a new app that may inherit some of the same functionality. - Commercial packages: This has been attempted a few times in the CakePHP community, without much success. I do think this is only really possible for very complex plugins, such as customizable backend admin panels, or for organizations that have a few different plugins to sell. Cartalyst is a good example of a company that sells a library of commercial packages, and I do hope that folks who sell code can recoup their invested time and effort. If you’re not sharing code, plugin creation and maintenance can cause extra, unnecessary overhead in developing your app, and I would not recommend it.
5. Code is not modular If your code is not modular, it maybe should not be a plugin. Code that cannot be reused is not a great candidate for pluginizing. For instance, creating a plugin that exposes a Datasource class for a database used in a specific industry - and nowhere else - where CakePHP is not prevalent is maybe not a great use of time. You are better suited to just having that be a class within your application. Additionally, while the process of making code modular may be interesting, it isn’t necessarily a great use of your time. In particular, if you don’t mean to share the code, then maybe you don’t need to spend billable hours on it.
6. Business-specific logic As an extension of non-modular code, code that contains business logic is often not a great candidate for pluginizing. In many cases, this may just be a single file, in which case, why are you adding the overhead of plugin loading and referencing for that class? Oftentimes, this sort of code will never make it outside of a given codebase, let alone your organization, in which case it is better suited to staying within the app. When I think about this particular reason, I like to think about whether the code is useful in any of the following contexts: - Is this something that would be a feature in an Issue Tracker? - Would a Content Management System utilize this plugin? - Could you think of a way to addd it to an Online Store? If the above types of software can use it, then maybe it isn’t tied to your specific business and is a good candidate to turn into a plugin. One exception here is if you are versioning Web APIs and sunsetting an old version. Having the old API be a plugin means you can maintain both simultaneously. Nothing I have in this presentation is gospel, and I’d love if everyone can keep that in mind.
7. Generic Utility Is this a generic class that can be used in any framework, or without a framework? In these cases, it is better to create a generic composer package and use it directly. Tying things to CakePHP just because it is the best framework and all other frameworks are just pale imitations isn’t always the right choice, and will certainly decrease the usage of your package. My favorite example of this is the Collection class. It probably didn’t need to be a CakePHP core plugin, and you will see this sort of class implemented as standalone libraries or features of other frameworks. If the class could have been a language feature, then thats not a great plugin. > Aside, yes, we wrote our own Collection class. It’s not in the language core, and we needed it for features within our framework, so here we are. Maybe you’re fine with it being tied to CakePHP, which works if you’re the only developer. Unfortunately, for larger organizations, frameworks change and evolve over time, and maybe someday you find yourself needing this utility in another application that maybe isn’t using CakePHP, or isn’t even using a framework. In that case, it would have been better to just keep it standalone, since there wasn’t ever anything tying it to the CakePHP framework itself.
8. Good candidates for plugins Basically a list of rails extensions from rubygems.org So what are good candidates for plugins? In the past, I would actually go to other frameworks and take inspiration for features and functionality that could be in CakePHP. Many of the ideas that made it to the Cake core were actually inspired by frameworks such as Django, Flask, Rails, Symfony, etc. I do think it makes sense to look around at what other people are doing, and sometimes that does mean begrudgingly accepting the notion that some other framework does something CakePHP does not. It also means we can do it better, so its less stealing and more borrowing and enhancing.
9. Framework extensions • New adapters to existing functionality (Datasources, Mail Transports…) • Code generation (Bake!) • Missing core functionality (File Uploading, Search) Easy things to build as new plugins are adapters to existing functionality. CakePHP was built with the idea of “Batteries included, but Replaceable”. You’ll see this if you’ve ever attempted to build a new datasource for something we didn’t have in the core (such as for Elasticsearch or MongoDB). The same idea exists for Mail transports (Amazon SES, Mailgun, etc.), Auth adapters, etc. This extends even to inner classes, such as datasource column types. While we do try to include a wide range of adapters in the core, you’re always welcome to build your own (and maybe someday they will land in the core!). I also think Code generation is an area where extending CakePHP makes a ton of sense. Custom bake plugins to generate new files or to enhance existing templates are great ideas to reduce the amount of time it takes to write code. Every release of CakePHP makes this a bit easier, and I always find myself writing a new Bake plugin for functionality I wish was in the core but I can’t get Jose Lorenzo Rodriguez to write tests for. Finally, if the core is missing some functionality that is common in other frameworks, that is a good chance to start working on something great. There have been a ton of File Upload plugins (I’ve personally worked on 3-4 different ones) and there are other areas where the Core team either hasn’t had time to develop a great implementation or isn’t interested in maintaining something. We’re not gods, Mark breaks the build, and I definitely think this is a neat way to explore ecosystem improvements.
10. Borrowed Ideas I borrow a lot of ideas. I actually regularly look at the docs for the following frameworks: - Django - Flask - Laravel - Rails - Symfony Not just because I’m a masochist and like looking at frameworks that I think aren’t as good as CakePHP, but because there are some generally interesting ideas coming out of those communities. Sometimes they have the best implementation for a feature CakePHP has, or exposes patterns that are useful to unlock new uses cases. Sometimes it’s something I never even thought about as a problem that others had. I definitely encourage people to also look at major/minor release notes for other frameworks. This is the best place for you to “borrow” ideas, either for your own app, or to develop as plugins for CakePHP. If you’re looking at ways to contribute to CakePHP, this is a good way to contribute blog content. I did this my email preview plugin - an idea I took from Rails - that eventually made its way into the DebugKit. This ended up being super useful to me later on, as I was working on an email-heavy application and hated needing to send myself emails in order to test how things looked while iterating on designs.
11. Releasing Plugins Or at least what I’ve done How do we release plugins? I think there are a few things you need to do in order for folks to feel comfortable with using your plugin.
12. Write Tests Writing tests This one is pretty easy. You definitely want to have test coverage of your plugin code. Why? Plugins are usually installed in one or more codebases with varying needs. If the plugin has no tests and the underlying implementation breaks functionality, tests are a good way to say “hey, we didn’t expect anyone to use it that way.” Tests are always evolving and I don’t think are ever complete. For core framework functionality, the Cake core team strives for 90% test coverage where possible. This enables us to at least cover everything we consider public API usage, and gives us the confidence to make sweeping changes and still be backwards compatible. As we find new ways to use the code, we also write new tests to ensure we cover those use cases. This is actually why our test suite is so massive, because we have a ton of use cases that have been contributed over the past decade. If in doubt, ask Jose Lorenzo Rodriguez for help with writing tests. He always writes mine, and it has worked out great so far
13. Use semver Use semver I try to use semantic versioning for all my plugins. Its a good way for me to announce to people that: - I am fixing a bug, which therefore results in a patch release - I am adding a feature, which results in either a patch or a minor release, depending on the size of the feature - I am changing how things work and you may need to edit usage within your codebase, which would result in a major release By using semantic versioning, I can avoid issues where someone says I broke their code in cases where I didn’t really mean to. Test coverage will also help give me confidence in the type of release I am making, and overall this helps me reduce maintenance burden.
14. Tag a release Tagging a release is CRITICAL. Without a release, I can consider all the code to be fluid and implementations non-static. Folks should 100% not try to depend on a non- tagged release - specifically branches - as there is no guarantee around the implementation being the same in a year or so. For people using Git, simply adding a git tag is good enough. Don’t fiddle with versions in files such as .semver or composer.json, as that adds overhead to the release process and makes it less likely that you will make a release. The fewer steps in your release process, the better for overall release cadence. As an aside, please distribute your code on packagist.org as composer packages when releasing open source. A git repository is great, but not the be-all end-all. For private or commercial CakePHP packages, you can continue referencing a Git repository or use software such as Satis to distribute that code.
15. CakePHP Queue The dos and donts of plugin development Finally, I’d like to end this talk with a small section on the CakePHP Queue plugin. This was something that I thought was a long-time coming and had done a ton of background work on over the years.
16. What went well • Generic, well-tested backend • Major feature other frameworks have • Based on pre-existing ideas around queuing In the case of a queuing plugin, having a backend that contains adapters for many common queueing systems was key to the CakePHP integration. There was a ton of interest internally on providing adapters for real-world messaging systems such as RabbitMQ, while still keeping the implementation simple enough for something like Redis (or at worst, SQL-backed queues) We also wanted to set folks off on the right path with something that was both well-tested and had usage outside of CakePHP. Maintaining a new framework feature is a huge burden, something we wanted to generally avoid. In this area, queue-interop is a great complement to the CakePHP adapter system, and fit in well with our mindset of “Batteries included, but replaceable”. Additionally, this is a feature that is common amongst other frameworks. Rails has had this for years via ActiveJob. Symfony has Commands. The Python world has celery and rq. Background Messaging is a common feature for almost every framework - built-in or otherwise - and is a common requirement for many applications. It’s even something we recommend in our docs as a workaround for long-running commands. I’d say that the ideas in our particular implementation have been around for a while now - php-resque, Queuesadilla, Cake-Queue - and bringing something more generic that followed some of the patterns into the core made a ton of sense.
17. What went poorly • Yet another queueing wrapper • Code Coverage below 50% • Not yet released Unfortunately, this process hasn’t been super great overall. This plugin existed in some fashion before. Multiple core devs - Hi Mark Scherer! - have written great plugins in this space in the past. Could we have reused something from the past vs re-inventing the wheel? Perhaps extending the existing plugin would have resulted in a better result. This is always something to think about before diving into plugin development. I would check the awesome-cakephp list before starting a new plugin. In particular, there were some early ideas around Cake\Event\Event queueing that were both confusing to end users and not actually necessary. It is important to reduce the scope of code to things that are absolutely necessary and nothing more to decrease both maintenance burden and cognitive overhead during use. Additionally, I was unable to convince certain people to write more tests for me. We’re still not yet at 50% code coverage, a far cry from the 90% coverage we desire in order to do a broad release. And I guess crucially, the package has yet to be released. It has been more than a few months, and here we are, still without a release. This was certainly a failing on my part to bring it over the finish line, though I hope to use the remainder of Cakefest to push it forward.
18. Questions? And thats all I got. If we still have time: Does anyone have any questions I can answer? Thanks everyone!