Pipeline Architectures at PHPNW15
Pipeline Architectures
View Slide
@mheap
Pipelines
Tea/Coffee
Waterfall
Manufacturing
Engine: 20 minsFrame: 40 minsWindows: 10 mins
1x Engine: 20 mins1x Frame: 40 mins1x Windows: 10 mins
2x Engine: 20 mins4x Frame: 40 mins1x Windows: 10 mins
Code Pipelines
This is a unix systemI know this
cat attendees.txt
cat attendees.txt | grepMichael
cat attendees.txt | grepMichael > michaels.txt
cat attendees.txt | grepMichael | sort >michaels.txt
Write programs that do one thingand do it well.Write programs to work together.Write programs to handle textstreams, because that is auniversal interface.
read(file.txt).map(u => return u.name).filter(n => return n ==“michael”)
producer.gochannel <- sumfoo.goval <- channel
app.php$bus->handle(new UserRegisteredEvent)
InputProcessingOutput
ControllerModelView
ActionDomainResponder
POST /fooDomainecho $twig->render()
$ php foo.php ‘bar’Domainecho cli_output($output);
zmqppzmqpp
zmqppredis-pipe
nsqnsq
Human Pipelines
+ =
Data???Profit!
DataProfit!
DataProfit!JSONJSONJSONJSON
DataProfit!JSONJSONJSONJSON??
DataProfit!PHPPHPC++
DataProfit!PHPGolangC++
DataProfit!PHPGolangMichael
Building Pipelines
Data-Driven
Demand-Driven
push/pull
Back Pressure
pub/sub
Sequence Numbers
Unique Jobs
Transport
Serialization
Error Handling
Testing Pipelines
Pros/Cons
Examples
https://www.flickr.com/photos/neotsn/2246759100/https://www.flickr.com/photos/theenmoy/15255488518https://www.flickr.com/photos/stuckincustoms/5109788502https://www.flickr.com/photos/danielfoster/4725849931https://www.flickr.com/photos/mzmo/8666989650https://www.flickr.com/photos/usacehq/5529362681https://www.flickr.com/photos/alpha_auer/5068482201/https://www.flickr.com/photos/109144586@N08/14863759614/https://www.flickr.com/photos/94418464@N08/9007117038https://www.flickr.com/photos/swamibu/2868288357/https://www.flickr.com/photos/sepehrehsani/5766453552/https://www.flickr.com/photos/scubasteveo/296747958/https://www.flickr.com/photos/56218409@N03/15371262455/https://www.flickr.com/photos/jvk/4417106https://www.flickr.com/photos/13523064@N03/14448815581https://www.flickr.com/photos/jamesshade/473206000https://www.flickr.com/photos/albertogp123/5843577306/https://www.flickr.com/photos/dpstyles/4835354126/
Questions?@mheap joind.in/15428