Slide 1

Slide 1 text

FRONTCONF How Frontend Developers can drive the Switch to Microservices Julie Ng Architect, Allianz Germany 9 December 2017

Slide 2

Slide 2 text

WHO AM I Designer • Developer • Architect Architect today Rich design & frontend experience got me here.

Slide 3

Slide 3 text

Web Designer Back in 1999 Websites as a hobby since 1999 Virtual Classroom As a designer, you don't make the rules. Someone else does. Designers are used to change. For a developer changing systems is expensive.

Slide 4

Slide 4 text

Interaction Designer Flash - click to Program Flash - early first experience with programming for UX and business value Example: deciper Sütterlin for potential customers

Slide 5

Slide 5 text

Interaction Developer(?) Flash - Actionscript Flash developers are in full control Limited Plugin from 1 vendor: more consistency You control screensize and interface size

Slide 6

Slide 6 text

Full-Stack Developer Frontend to Backend (Ruby) Basic frontend, HTML/CSS, limited JavaScript, mostly jQuery for years Left Ancestry and freelanced for startups and myself. Learned Backbone.JS, Rails MVC I was full-stack. I built monoliths. Now what?

Slide 7

Slide 7 text

Mike Monteiro on "What is a Designer?" solves problems within a set of constraints understands goals gathers information imposes order is a gatekeeper Enterprise Architect Designer at ♥ Why go from startups to corporate? I like problems and challenges. We have BIG problems. Plus I ran out of money

Slide 8

Slide 8 text

Allianz Germany Financial Year 2016 Revenue Operating Profit € 32.3 Billion € 2.0 Billion – Allianz Deutschland AG Zahlen, Daten und Fakten The bigger you are, the slower you are to move. But we're going through an IT transformation. Why?

Slide 9

Slide 9 text

Challenges from Outside • New Business Models • New Markets • 100% Digital

Slide 10

Slide 10 text

Hello Lemonade Insurance claims are processed in 3 Minutes and paid out

Slide 11

Slide 11 text

A Fast Architecture Event driven Microservices Background Video: Microservices at Netflix Scale: Principles, Tradeoffs & Lessons Learned By Ruslan Meshenberg, GOTO 2016, This is not easy! It's a complex mind fuck. https://youtu.be/57UK46qfBLY?t=28m33s

Slide 12

Slide 12 text

ATC: Open Offices My job is to help developers move in that direction. I work in our Agile Training Center (ATC).

Slide 13

Slide 13 text

ATC: we have Post-Its

Slide 14

Slide 14 text

ATC: we do Standups

Slide 15

Slide 15 text

MICROSERVICES Why is it so hard?

Slide 16

Slide 16 text

Offloading tasks to the cloud Why? For a better user experience. Google Photos AI analyzes my photos And made this lovely animated gif of Kilimanjaro from above for me without using my battery and resources

Slide 17

Slide 17 text

— Jonas Bonér, Reactive Microservices Architecture What is a microservice? That's Easy It's small. It does one thing. It's fast. Because it's small? ⁃ ⁃ Some developers split monoliths into multiple apps and think "that's a microservice!" But they have to be deployed together , i.e. coupled and not a microservice Where do most developers fail? State / Data

Slide 18

Slide 18 text

What is a microservice? Actually, it's hard State is not at rest → but in motion No beginning, no end → no indexes Distributed → where is my data (source of truth)? ⁃ ⁃ ⁃ Uncertainty philosophical mind fuck.

Slide 19

Slide 19 text

You have no control Everything is uncertain Tough problem to solve!

Slide 20

Slide 20 text

We never had control — Kevin Webber. A Journey into Reactive Streams. User picked their device and browser of choice We always think of all use cases responsive design We're more creative about worst case screnarios. Remember IE7? We've always juggled backend and frontend data and tried to re-connect

Slide 21

Slide 21 text

What beginning? Only reactions. Not when. Whenever. mouseenter mouseover mousemove mousedown click dblclick wheel select ⁃ ⁃ ⁃ ⁃ ⁃ ⁃ ⁃ ⁃ "when" is synchronous, coupled to time. "whenever" is reactive, no coupling to time.

Slide 22

Slide 22 text

What Which end? All your bases are covered. document.addEventListener("dragend", function(event) { … }, false); document.addEventListener("dragexit", function(event) { … }, false); document.addEventListener("dragleave", function(event) { … }, false);

Slide 23

Slide 23 text

DOM Event bubbling anyone? The force is with you frontend developers!

Slide 24

Slide 24 text

STATE AS STREAMS Many sources & always moving

Slide 25

Slide 25 text

Concat() — RxJS Official Docs

Slide 26

Slide 26 text

Merge() — RxJS Official Docs

Slide 27

Slide 27 text

CombineLatest() Looks intimidating… — RxJS Official Docs Looks Hard. Where many backend developers get lost Basically "holding" 2 values b2, b3, b4 -> always "b", no values from first stream

Slide 28

Slide 28 text

CombineLatest() BMI is always moving… var weight = Rx.Observable.of(70, 72, 76, 79, 75); var height = Rx.Observable.of(1.76, 1.77, 1.78); var bmi = Rx.Observable.combineLatest(weight, height, (w, h) => w / (h * h)); bmi.subscribe(x => console.log('BMI is ' + x)); 5 weights & 3 heights Streams -> determines calculated BMI sequence You might not get same BMIs every time.

Slide 29

Slide 29 text

Forms & Streams Example: Rate Calculator Browser is really fast. You can't control sequence Data may be invalid at a certain time But so fast, it's valid later

Slide 30

Slide 30 text

Forms & Models How to lose millions on the frontend '5.000.000,00' 500000000 5000000 '1,7' For a better UI: automatically insert thousands-separator in real-time on input Problem: What's a sum? rate (e.g. float)? currency (string)? Challenge: converting 1.000,00 to 1000, strings to floats and back at browser speeds - when you have state at rest. Streams are better. You will have inconsistent data at some point. But get to eventual consistency without user noticing.

Slide 31

Slide 31 text

Forms & Streams Functional programming — export const INITIAL_STATE: IAppState = { count: 0, }; export function rootReducer(lastState: IAppState, action: Action): IAppState { switch(action.type) { case CounterActions.INCREMENT: return { count: lastState.count + 1 }; case CounterActions.DECREMENT: return { count: lastState.count - 1 }; } return lastState; } @angular-redux/store Beginners' Tutorial Question: Isn't functional programming blindly following the rules? Example: what if my account has no money? Answer: you get charged a fine (new way to make money).

Slide 32

Slide 32 text

Similarly in the Backend Parallel requests are nice, but now what? — Jonas Bonér, Reactive Microservices Architecture What data do I send back? Worried about sending back inconsistent data?

Slide 33

Slide 33 text

Backend streams Brain hurt yet? — Jonas Bonér, Reactive Microservices Architecture

Slide 34

Slide 34 text

FRONTEND Most challenging development

Slide 35

Slide 35 text

— @wycats on Twitter retired member of the Ruby on Rails retired member jQuery Core Team creator of ember.js member of the Rust Core Team

Slide 36

Slide 36 text

Frontend Developers Advantage app development with SPAs real-time interactions asynchronous communication from multiple sources already event-driven user facing you never had control ⁃ ⁃ ⁃ ⁃ ⁃ ⁃ Not 1999. Today every frontend developer knows JavaScript. There is no request/response, only NOW. User facing: shorter feedback loop. And you can SEE what you do. Less abstract. Not fair to backend devs. Control → mindfuck

Slide 37

Slide 37 text

DEFINING BOUNDARIES Where Business meets Development Your user experience and short feedback loop helps you think differently, abstractly. Business = Design As an enterprise architect, you have to align business processes to technology, not just software.

Slide 38

Slide 38 text

Cloud Computing Challenges Cloud architectures are distributed systems What can you do in parallel? What's synchronous? What's asynchronous? Where do you split business logic? ⁃ ⁃ ⁃ ⁃ Not an all-or-nothing game Charts with parallel lines are easy to understand. Translating business to code is not

Slide 39

Slide 39 text

Charts with parallel lines are easy Translating business logic to code is not

Slide 40

Slide 40 text

One Customer

Slide 41

Slide 41 text

Competing Customers Who will get their coffee(s) first?

Slide 42

Slide 42 text

How & where to optimize?

Slide 43

Slide 43 text

Divide and Conquer

Slide 44

Slide 44 text

Smart is messy

Slide 45

Slide 45 text

CLOUD UPLOAD SERVICE Demo & Deep dive

Slide 46

Slide 46 text

How do you upload a 50MB file? A single HTTP request with SOAP?

Slide 47

Slide 47 text

Distributed Upload Service Demo

Slide 48

Slide 48 text

Demo shows broken worker with a bug → no need to redeploy app Queued uploads wait for available workers Large files are stored in chunks and streamed back as single file

Slide 49

Slide 49 text

Offload uploads to Workers Frontend (3 MB) Backend Proxy message broker worker (1 MB) worker (1 MB) worker (1 MB) e.g. /api/worker01/upload/123 upload is saved in n chunks and streamed back as single file ⁃ ⁃ ⁃ Everything scalable: frontend, backend, workers Bottle is backend, so break off worker job missing direct line between proxy and worker I stupidly realized this morning, instead of absolutely positioning div s, I should have used PowerPoint and screenshots

Slide 50

Slide 50 text

Uploading - Get identifier app.get('/api/upload/new', function (req, res) { let id = crypto.randomBytes(16).toString('hex') res.json({ subscription: { id: id, } }) }) synchronously fetch ID via REST better error handling. No need for timeout.

Slide 51

Slide 51 text

Uploading - worker waits for chunks queue.on('connect', () => { http.listen(port, () => { console.log(`=== [${workerName}] listening on ${port}… ===` ) }) }) queue.on('message', (attrs) => { let route = `/api/${workerName}/upload/${attrs.id}` queue.transmit('ready', Object.assign({ route: route }, attrs)) http.post(`/upload/${attrs.id}`, (req, res) => { let upload = new Upload(attrs.id, req.headers) queue.transmit('begin', attrs) upload.on('file:end', () => { queue.transmit('done', attrs) res.writeHead(200, { 'Connection': 'close' }) res.end("That's all folks!") }) return req.pipe(upload) }) }) worker listens for particular URL writes file/chunk to disk, could be stored in memory

Slide 52

Slide 52 text

Downloading - stream 4 chunks back as 1 file app.get('/api/download/:id', function (req, res) { let id = req.params.id Upload.findOne({id: id}, (err, upload) => { if (!err && upload) { let chunks = upload.chunks.split(',') res.set({ 'Content-Type': upload.contentType, 'Content-Disposition': `attachment; filename="${upload.name}"` }) // What is this? pipeChunks(chunks, res) } else { if (err) console.log(err) res.status(404).send('File not found') } }) })

Slide 53

Slide 53 text

Client (vueJS) Messages Command Destination Description SEND /upload/new send file details to backend SUBSCRIBE /upload/ready receive chunk upload end points SUBSCRIBE /upload/create receive download url a er upload is created on backend

Slide 54

Slide 54 text

Message Example, ack s and s >>> SEND (app) x-upload-id:73750a171c67a8d54d1cfe67f81853c3 x-upload-event:done content-type:application/json subscription:done@73750a171c67a8d54d1cfe67f81853c3 ack:client destination:/topic/upload.done timestamp:1504711122064 content-length:345 {"upload":{"id":"73750a171c67a8d54d1cfe67f81853c3","name":"jqueryconf_final_2.pdf","contentType":"app

Slide 55

Slide 55 text

Backend Messages - Chunks Command Destination Description SUBSCRIBE /topic/chunk.ready transform into upload/ready message for frontend SUBSCRIBE /topic/chunk.done check if all chunks are done. If so, send upload.done message subscribes are whenevers reactive programming change message route for frontend (in retrospect I'm not sure if this was a good idea?)

Slide 56

Slide 56 text

SUMMARY Advantages & Challenges

Slide 57

Slide 57 text

Microservices: advantages For best user experience Fast and real-time Resilient and fault tolerant ⁃ ⁃

Slide 58

Slide 58 text

Microservices: challenges But easier for frontend developers Easy in JavaScript - not much code. You already know real-time and streams. Leverage both synchronous and asynchronous calls. Leverage both REST and web sockets - error handling. Messy is OK, and by design. You don't need 100% perfect control. Splitting logic is difficult. ⁃ ⁃ ⁃ ⁃ ⁃ ⁃

Slide 59

Slide 59 text

Microservices - Beauty in Chaos Julie Ng @jng5