Building a full-stack app with Golang and Google Cloud Platform in one week

Slide 1

Slide 1 text

Memes in the Cloud – Building a full-stack app with Golang and Google Cloud Platform in one week Dr. Felix Raab | Head of Engineering @ KI labs | ki-labs.com [email protected] | medium.com/@fe9lix

Slide 2

Slide 2 text

SCOPE OF TALK

Slide 3

Slide 3 text

SCOPE OF TALK Case Study Project idea and journey for an MVP ﬁnished within one week

Slide 4

Slide 4 text

SCOPE OF TALK Case Study Project idea and journey for an MVP ﬁnished within one week Software Engineering for Cloud Apps The “full” stack: Frontend, Backend, Build, Deployment, Logging… Towards the end of the talk: Software Engineering practices in Go

Slide 5

Slide 5 text

Slide 6

Slide 6 text

BACK AT MY PREVIOUS EMPLOYER… Hours claiming – trying not get “shitlisted”… because the boss would be notiﬁed!

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Idea – Every Friday… Schedule an hours claim reminder mail containing a meme, to myself.

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

FIRST: HOW TO CHOOSE A CLOUD? …when you can start from scratch – no corporate constraints, no restrictions, no Frankfurt.

Slide 13

Slide 13 text

LOOK AT THE HISTORY OF CLOUD COMPANIES

Slide 14

Slide 14 text

LOOK AT THE HISTORY OF CLOUD COMPANIES

Slide 15

Slide 15 text

LOOK AT THE HISTORY OF CLOUD COMPANIES

Slide 16

Slide 16 text

LOOK AT THE HISTORY OF CLOUD COMPANIES

Slide 17

Slide 17 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE

Slide 18

Slide 18 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES

Slide 19

Slide 19 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM

Slide 20

Slide 20 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise

Slide 21

Slide 21 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops

Slide 22

Slide 22 text

LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops S(R)E

Slide 23

Slide 23 text

–Brave New Geek “Multi-Cloud Is a Trap” https:/ /bravenewgeek.com/multi-cloud-is-a-trap/

Slide 24

Slide 24 text

WORKS LIKE YOU WOULD EXPECT IT…

Slide 25

Slide 25 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail

Slide 26

Slide 26 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later…

Slide 27

Slide 27 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later… API access, permissions, conﬁg…

Slide 28

Slide 28 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later… CI/CD API access, permissions, conﬁg…

Slide 29

Slide 29 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) API access, permissions, conﬁg…

Slide 30

Slide 30 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) API access, permissions, conﬁg…

Slide 31

Slide 31 text

WORKS LIKE YOU WOULD EXPECT IT… Service Oﬀerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) Task Queue (with scheduling!) API access, permissions, conﬁg…

Slide 32

Slide 32 text

Slide 33

Slide 33 text

The Frontend – Diving into the world of JavaScript? https://exceptionnotfound.net/big-ball-of-mud-the-daily-software-anti-pattern/

Slide 34

Slide 34 text

THE FULLSTACK IN GO – SO HOW ABOUT WEBASSEMBLY (WASM)?

Slide 35

Slide 35 text

NO! BECAUSE…

Slide 36

Slide 36 text

NO! BECAUSE…

Slide 37

Slide 37 text

NO! BECAUSE… Readability, Debugging, Size…  (but maybe in the future)

Slide 38

Slide 38 text

NO WEBASSEMBLY. MAYBE ELM THEN?

Slide 39

Slide 39 text

NO! BECAUSE… Learning curve, different paradigm, transpiling to JS, libraries… – remember, we have one week to ﬁnish!

Slide 40

Slide 40 text

VANILLA JS (ES 6)!   BUT WHICH FRAMEWORK?

Slide 41

Slide 41 text

RESIST THE TEMPTATION TO…

Slide 42

Slide 42 text

RESIST THE TEMPTATION TO…

Slide 43

Slide 43 text

RESIST THE TEMPTATION TO… Also, do not use Github Stars as your only criterion! Popularity is one thing…

Slide 44

Slide 44 text

HOW TO CHOOSE

Slide 45

Slide 45 text

HOW TO CHOOSE 1) Does it solve your problem in a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base.

Slide 46

Slide 46 text

HOW TO CHOOSE 1) Does it solve your problem in a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base. 2) Look at the quality of the documentation! Because you need to read it and be able to understand it – quickly!

Slide 47

Slide 47 text

VUE.JS SEEMS TO MATCH. BUT WHICH ARCHITECTURE?

Slide 48

Slide 48 text

SO MANY APPROACHES… …MVVM, Flux, Redux, SAM, ELM Architecture…? What is the essence?

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

STATE MUTATION

Slide 51

Slide 51 text

STATE MUTATION Take notice when somebody tells you that you should or can entirely avoid mutating state.

Slide 52

Slide 52 text

MUTATING STATE. HERE’S HOW…

Slide 53

Slide 53 text

MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth.

Slide 54

Slide 54 text

Slide 55

Slide 55 text

MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads. 3) Accept or reject the state mutation in your model At the end of your (async.) action, “commit” / “propose” a new value to the model.

Slide 56

Slide 56 text

Slide 57

Slide 57 text

VUEX.JS KIND OF SUPPORTS THAT PATTERN

Slide 58

Slide 58 text

–Ryan Dahl “[...] if you’re building a server, I can’t imagine using anything other than ” THE BACKEND.

Slide 59

Slide 59 text

HOW TO BUILD?

Slide 60

Slide 60 text

HOW TO BUILD? Go 1.11 Because Go Modules!

Slide 61

Slide 61 text

HOW TO BUILD? Go 1.11 Because Go Modules! Docker To be ﬂexible with Go versions  Multi-stage build!

Slide 62

Slide 62 text

HOW TO BUILD? Go 1.11 Because Go Modules! Docker To be flexible with Go versions  Multi-stage build! entr + Makefile No Rake, Grunt, Gulp, NPM, Yarn – Makefile + Bash Script fine! Check entrproject.org

Slide 63

Slide 63 text

APP ENGINE – START SIMPLE

Slide 64

Slide 64 text

APP ENGINE – START SIMPLE

Slide 65

Slide 65 text

APP ENGINE – START SIMPLE App Engine Flex You can migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster.

Slide 66

Slide 66 text

APP ENGINE – START SIMPLE App Engine Flex You can migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster. Use defaults Start with App Engine default options and auto-scaling. Nobody knows how much traffic you will get. Watch and see what happens.

Slide 67

Slide 67 text

SEPARATE THE FRONTEND AND BACKEND?

Slide 68

Slide 68 text

YES AND NO!

Slide 69

Slide 69 text

Slide 70

Slide 70 text

YES AND NO! Expose your services as API to the frontend Frontend is rendered via a Go template. On that rendered page, a Single Page App can run in memory and call your backend API. Use Go templates for other static pages. Makes routing easier. But no need to create two deployment units No need for premature optimisation (scaling). One Go binary is simpler than two Go binaries!   Static assets are loaded from the ﬁle system in your container.

Slide 71

Slide 71 text

CI/CD

Slide 72

Slide 72 text

CI/CD Google Cloud Build Will trigger and deploy when you push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think.

Slide 73

Slide 73 text

CI/CD Google Cloud Build Will trigger and deploy when you push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think. App Engine Deployment Straightforward versioning and service concept via DNS – version.service.appname Promoting app to production, traffic splitting etc. –   supported out of the box

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

The cloud can be mean and slow.

Slide 77

Slide 77 text

PLAYGROUNDS SAVE TIME Create a GCP “Playground” project Got stuck with Cloud Datastore (vs. Cloud Firestore) and   Cloud Tasks (only certain EU region). Led me to test service offerings ﬁrst via playground.

Slide 78

Slide 78 text

WHICH ARCHITECTURE – ONIONS AND HEXAGONS?

Slide 79

Slide 79 text

SO MANY LAYERS AND CONCEPTS Also, ask yourself : Is there a rich domain? Or are you really building a CRUD app?

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

No content

Slide 82

Slide 82 text

Take notice when nobody tells you to separate functionality into lots and lots of really small methods and delegating objects.

Slide 83

Slide 83 text

A QUESTION OF PHILOSOPHIES?

Slide 84

Slide 84 text

A QUESTION OF PHILOSOPHIES?

Slide 85

Slide 85 text

A QUESTION OF PHILOSOPHIES? Robert Martin (Uncle Bob) http://2017.agilesummit.gr/speakers/uncle-bob/

Slide 86

Slide 86 text

A QUESTION OF PHILOSOPHIES? John Ousterhout https://www.youtube.com/watch?v=bmSAYlu0NcY Robert Martin (Uncle Bob) http://2017.agilesummit.gr/speakers/uncle-bob/

Slide 87

Slide 87 text

DEEP VS SHALLOW MODULES

Slide 88

Slide 88 text

DEEP VS SHALLOW MODULES Shallow Module Deep module Interface Functionality

Slide 89

Slide 89 text

DEEP VS SHALLOW MODULES Shallow Module “The best modules are deep: they allow a lot of functionality to be accessed through a simple interface. A shallow module is one with a relatively complex interface, but not much functionality: it doesn’t hide much complexity.”   – From A Philosophy of Software Design, John Ousterhout Deep module Interface Functionality

Slide 90

Slide 90 text

EXAMPLE OF A DEEP MODULE Meme Image Generation Package “meme”, image.go: Simple interface, a lot of functionality.

Slide 91

Slide 91 text

PASS-THROUGH CODE TENDS TO BE SHALLOW – BUT HOW MUCH SEPARATION?

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

Separate the outside from the inside Kind of onion, just fewer layers

Slide 94

Slide 94 text

Separate the outside from the inside Kind of onion, just fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers

Slide 95

Slide 95 text

Separate the outside from the inside Kind of onion, just fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers Inside (package meme) Core, your actual (business) functionality

Slide 96

Slide 96 text

WHAT ABOUT INTERFACES?

Slide 97

Slide 97 text

IMPLICIT INTERFACE CONFORMANCE

Slide 98

Slide 98 text

IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…

Slide 99

Slide 99 text

IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…

Slide 100

Slide 100 text

IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…

Slide 101

Slide 101 text

IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here… To be able to inject a different implementation, you would need to either create an interface that mirrors the datastore api or create a generic interface -> shallow wrapper

Slide 102

Slide 102 text

Slide 103

Slide 103 text

STRONGER ARGUMENT FOR AN INTERFACE?

Slide 104

Slide 104 text

STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But Go’s default logging has limitations.

Slide 105

Slide 105 text

STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But Go’s default logging has limitations. Even better: Structured Logging Implicit conformance to uber-go/zap for structured logging

Slide 106

Slide 106 text

Slide 107

Slide 107 text

HOW TO HANDLE CROSS- CUTTING CONCERNS…?

Slide 108

Slide 108 text

CONTEXT OBJECT PATTERN

Slide 109

Slide 109 text

CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware.

Slide 110

Slide 110 text

CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware.

Slide 111

Slide 111 text

CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware. Still not ideal, but good enough? Also see: https://medium.com/@gosamv/using-gos-context-library-for-logging-4a8feea26690

Slide 112

Slide 112 text

FINALLY, BEING CONSISTENT…

Slide 113

Slide 113 text

FINALLY, BEING CONSISTENT…

Slide 114

Slide 114 text

FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should see the most recent list of meme reminders   (strong consistency supported by Cloud Datastore)

Slide 115

Slide 115 text

FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should see the most recent list of meme reminders   (strong consistency supported by Cloud Datastore) Eventual Consistency Meme detail view might not have been generated and stored yet –   but you can always apologise to the user. Eventual consistency is often ﬁne.

Slide 116

Slide 116 text

CONCLUSION

Slide 117

Slide 117 text

CONCLUSION Keep it simple (but not stupid!) Reduce the number of “things” you need to deal with – both from an architecture and a product perspective.

Slide 118

Slide 118 text

CONCLUSION Keep it simple (but not stupid!) Reduce the number of “things” you need to deal with – both from an architecture and a product perspective. Challenge existing “best practices” Have you been brainwashed by cargo cults in the past? The “best” practices always depend on the context and scope.