Introducing The Instruction Pipeline

A presentation by @stuherbert  for @GanbaroDigital Introducing The Instruction Pipeline
How To Design And Build  Bespoke Middleware

@GanbaroDigital I’m going to show you how to build a
pipeline that’s built to your specs.

@GanbaroDigital We’re going to build a real-life, production-ready bespoke pipeline.

@GanbaroDigital Full design process, writing the code, and testing: all
in around 45 mins.

@GanbaroDigital We’re going to build it using an open-source package:
ganbarodigital/ php-mv-instruction-pipeline

@GanbaroDigital If you want to read along: https://github.com/ganbarodigital/  php-mv-messaging-pipeline

@GanbaroDigital What Is A Pipeline?

@GanbaroDigital “ A pipeline is an assembly of isolated modules,
for data to ﬂow along.

@GanbaroDigital Module 1 Module 2 Module 3 Module 4

@GanbaroDigital • Pipelines are ordered • We decide the order
of the modules on the pipeline

@GanbaroDigital Pipelines are perfect if you need ﬂexibility and the
ability to respond rapidly to change.

@GanbaroDigital “ We can chop and change the modules -
and their order - without having to touch the modules themselves

@GanbaroDigital Module 1 Module 3 Module 4

@GanbaroDigital The pipeline itself is deﬁned in conﬁg, not code.

@GanbaroDigital

@GanbaroDigital ?? ?? What Happens To The Data?

@GanbaroDigital Data In A Pipeline Can Be … 1. modiﬁed
2. transformed 3. or rejected

@GanbaroDigital PSR7 Middleware is a data transformation pipeline: It turns
HTTP requests into HTTP responses.

@GanbaroDigital Building A Pipeline

@GanbaroDigital Real-life use case: messaging in the #cloud

@GanbaroDigital Back Ofﬁce Service Public  Web App API

@GanbaroDigital Back Ofﬁce Service Public  Web App API Status Events

@GanbaroDigital Back Ofﬁce Service Public  Web App API Status Events
Message Queue

@GanbaroDigital Design Constraints 1. We must use the cloud-provider’s message
queueing service 2. Different programming languages in use for transmitting and receiving systems

@GanbaroDigital Principle: Always treat the cloud-provider’s infrastructure as an untrusted
third-party system.

@GanbaroDigital Trafﬁc to and from the message queue goes over
HTTPS. The messages are encrypted on the wire / in transit.

@GanbaroDigital Messages at rest inside the queue? Not guaranteed to
be encrypted.

@GanbaroDigital Encrypt

@GanbaroDigital Encrypt HMAC

@GanbaroDigital The message queue does not accept arbitrary binary data.
Only 7-bit ASCII. Their castle, their rules.

@GanbaroDigital Encrypt HMAC

@GanbaroDigital Encrypt HMAC Ascii-Safe

@GanbaroDigital Design Constraints 1. We must use the cloud-provider’s message
queueing service 2. Different programming languages in use for transmitting and receiving systems

@GanbaroDigital Encrypt HMAC Ascii-Safe

@GanbaroDigital JSON Encrypt HMAC Ascii-Safe

@GanbaroDigital In real life, the requirements to encrypt came late
in the day.

@GanbaroDigital JSON Ascii-Safe

@GanbaroDigital Because We Built A Pipeline • Created new modules
• Dropped them in where we needed them • Zero changes to signed-off code

@GanbaroDigital JSON Ascii-Safe Encrypt HMAC

@GanbaroDigital Because We Built A Pipeline • Created new modules
• Dropped them in where we needed them • Zero changes to signed-off code • Reduced impact on deployment pipeline

@GanbaroDigital Requirements drive the design of your pipelines.

@GanbaroDigital Let’s break down how to implement your requirements.

@GanbaroDigital JSON Encrypt HMAC Ascii-Safe Not the code that runs
inside the pipeline!

@GanbaroDigital JSON Encrypt HMAC Ascii-Safe Instruction Builders

@GanbaroDigital “Instruction Builders” are factories that instantiate pipeline “Instructions”.

@GanbaroDigital “Instructions” are the individual modules added to each pipeline.

@GanbaroDigital Scope “Instructions” to be small and ﬂexible.

@GanbaroDigital JSON Encrypt HMAC Ascii-Safe Instruction Builders

@GanbaroDigital Encode To JSON Encrypt Add HMAC Encode To Ascii-Safe
Instructions

Instructions Decode From Ascii-Safe Verify & Strip HMAC Decrypt Decode From JSON Transmit Receive

@GanbaroDigital “Instruction Builders” instantiate Instructions for both transmit and receive
pipelines.

@GanbaroDigital (Yes, you can build just the one pipeline if
you need to)

@GanbaroDigital Encrypt Instruction Builder HMAC Ascii-Safe JSON Transmit Pipeline Receive
Pipeline

@GanbaroDigital Encrypt Instruction Builder HMAC Encode To JSON Ascii-Safe JSON
Decode From JSON Transmit Pipeline Receive Pipeline

@GanbaroDigital Encrypt Add HMAC Encode To Ascii-Safe Instruction Builder HMAC
Verify & Strip HMAC Encode To JSON Decode From Ascii-Safe Ascii-Safe JSON Decode From JSON Encrypt Decrypt Transmit Pipeline Receive Pipeline

@GanbaroDigital We have discovered the individual Instructions that we need
to code up.

@GanbaroDigital ?? ?? So we’re ready to jump into writing
the code?!?

@GanbaroDigital There’s one more level of decomposition to do ﬁrst.
We need to break down our Instructions.

@GanbaroDigital Treat your Instructions like you would treat your MVC
controllers.

@GanbaroDigital Instruction Add HMAC Operation(s)

@GanbaroDigital Instruction Calc HMAC Add HMAC Operation(s) Add HMAC

@GanbaroDigital Instruction Verify & Strip HMAC Calc HMAC Add HMAC
Operation(s) Add HMAC

Find HMAC Remove HMAC Operation(s) Verify HMAC Add HMAC Calc HMAC

@GanbaroDigital “Operations” are the smallest units of work that it
makes sense to reuse.

@GanbaroDigital “Operations” have no dependency whatsoever on the pipeline framework.

@GanbaroDigital NOW we are ready to write some code :)

@GanbaroDigital

@GanbaroDigital Operations are very boring. They’re just plain PHP code.
In my experience, boring code is reliable code.

@GanbaroDigital Our Operations are ready to run. We can go
ahead and code up our Instructions.

@GanbaroDigital Instructions aggregate Operations to get the work done.

@GanbaroDigital Aggregation vs Inheritance: aggregation for reuse, inheritance for specialisation

@GanbaroDigital

@GanbaroDigital Thanks to identifying and building the Operations, the Instructions
are very lean and very clean.

@GanbaroDigital Unlike the Operations, the Instructions won’t execute yet. They
depend upon NextInstruction.

@GanbaroDigital Let’s talk about (and build) the NextInstruction class.

@GanbaroDigital NextInstruction is the secret sauce that powers Instruction pipelines!

Instructions

@GanbaroDigital NextInstruction Encode To JSON Encrypt Add HMAC Encode To
Ascii-Safe

@GanbaroDigital $next->process() Encode To JSON Encrypt Add HMAC Encode To
Ascii-Safe

$next->process() $next->process()

Ascii-Safe $next->process() $next->process() $next->process() $next->process()

Ascii-Safe $next->process() $next->process() $next->process()

@GanbaroDigital $next->process() Encode To JSON Encrypt Add HMAC $next->process() $next->process()
$next->process()

@GanbaroDigital $next->process() Encode To JSON

@GanbaroDigital $next->process()

@GanbaroDigital

@GanbaroDigital We’re now starting to see how to use the
underlying InstructionPipeline framework. ganbarodigital/ php-mv-instruction-pipeline

@GanbaroDigital

@GanbaroDigital Now that we have working Instructions, we can build
out their factories: the “Instruction Builders”

@GanbaroDigital

@GanbaroDigital GenericInstructionBuilder expects your Instruction to take a conﬁg array
as its only parameter.

@GanbaroDigital

@GanbaroDigital Follow the same process (Operations > Instructions > Builders)
to complete the pipeline’s code.

@GanbaroDigital How do we execute the Instruction Builders to assemble
the pipelines? We need to build one last helper class.

@GanbaroDigital

@GanbaroDigital We’re done building :)

@GanbaroDigital We Have Built … • 11 x Operations (inc
base classes) • 10 x Instructions • 4 x Instruction Builders

@GanbaroDigital They Are Supported By … • 8 x Robustness
helpers   (Checks & Requirements) • 11 x Exceptions • 2 x Type-safety Interfaces

@GanbaroDigital Let’s see it all together.

@GanbaroDigital

@GanbaroDigital Build Order 1. Operations 2. NextInstruction & build helper
3. Instructions 4. Instruction Builders

@GanbaroDigital Type Safety

@GanbaroDigital PSR7 (HTTP Messages) and the associated Middleware have been
loudly criticised for lack of type-safety.

@GanbaroDigital declare(strict_types=1); Enables enforcement of PHP 7 type declarations.

@GanbaroDigital “ Types are policy. Policy belongs in your code,
not in underlying libraries.

@GanbaroDigital ?? ?? How hard is it to add type
declarations to a library like MessagingPipeline?

@GanbaroDigital There are no problems adding type declarations to Operations.
They’re just plain PHP.

@GanbaroDigital

@GanbaroDigital Instructions … well, it depends on what your pipeline
does to the data it processes.

@GanbaroDigital Data In A Pipeline Can Be … 1. modiﬁed
2. transformed 3. or rejected

@GanbaroDigital When we transform data, we change its data type.

@GanbaroDigital Instructions are isolated from each other. They cannot know
what data transformations will happen elsewhere on the pipeline.

@GanbaroDigital The return type from your pipeline’s last Instruction determines
how type-strict you can be.

Pipeline mixed > string string > string string > string string > string string > string string > mixed string > string string > string

@GanbaroDigital You can deﬁne interfaces to enforce type safety on
your Instructions.

@GanbaroDigital

@GanbaroDigital NextInstruction also enforces type safety. It’s unique to your
library. Only Instructions written for your library will accept your NextInstruction.

@GanbaroDigital

@GanbaroDigital ?? ?? Did you notice the runtime checks that
we still needed?

@GanbaroDigital

@GanbaroDigital Strict typing is the equivalent of ‘ﬁlters’ from ‘input
ﬁltering and validation’. Strict typing doesn’t cover your data validation responsibilities.

@GanbaroDigital You still need appropriate runtime checks. You still need
appropriate unit tests.

@GanbaroDigital Testing

@GanbaroDigital Instruction Pipelines are easy to test.

@GanbaroDigital Unit Test Build Order 1. Operations 2. Instructions 3.
NextInstruction & build helper

@GanbaroDigital Operations are plain PHP, by design. There’s nothing special
required to test them.

@GanbaroDigital

@GanbaroDigital They have single responsibilities. We don’t need many tests
to prove that they work.

@GanbaroDigital

@GanbaroDigital “With 100% code coverage, you could still ship bugs.
Without 100% code coverage, you’re deﬁnitely shipping bugs.

@GanbaroDigital Full code available on GitHub & Packagist: ganbarodigital/ php-mv-messaging-pipeline

Thank You Any Questions? A presentation by @stuherbert  for @GanbaroDigital

Introducing The Instruction Pipeline

Introducing The Instruction Pipeline

More Decks by Stuart Herbert

Other Decks in Programming

Featured

Transcript