Building a full-stack app with Golang and Google Cloud Platform in one week

The talk will cover how to effectively build a production-ready, full-stack app with Golang and GCP under time constraints. I'll discuss how to approach making quick and sound technical decisions and how to apply modern software engineering practices for end-to-end apps. The presentation shows, in an opinionated and "meme-ful" way, various lessons learned, tools, and key takeaways for cloud environments.

Transcript

1. Memes in the Cloud – Building a full-stack app with Golang and Google Cloud Platform in one week Dr. Felix Raab | Head of Engineering @ KI labs | ki-labs.com f.raab@kigroup.de | medium.com/@fe9lix
2. SCOPE OF TALK
3. SCOPE OF TALK Case Study Project idea and journey for an MVP finished within one week
4. SCOPE OF TALK Case Study Project idea and journey for an MVP finished within one week Software Engineering for Cloud Apps The “full” stack: Frontend, Backend, Build, Deployment, Logging… Towards the end of the talk: Software Engineering practices in Go
5. SCOPE OF TALK Case Study Project idea and journey for an MVP finished within one week Software Engineering for Cloud Apps The “full” stack: Frontend, Backend, Build, Deployment, Logging… Towards the end of the talk: Software Engineering practices in Go Memes At the heart of the app itself – consequently, Memes to illustrate some opinionated views
6. BACK AT MY PREVIOUS EMPLOYER… Hours claiming – trying not get “shitlisted”… because the boss would be notified!
9. Idea – Every Friday… Schedule an hours claim reminder mail containing a meme, to myself.
12. FIRST: HOW TO CHOOSE A CLOUD? …when you can start from scratch – no corporate constraints, no restrictions, no Frankfurt.
13. LOOK AT THE HISTORY OF CLOUD COMPANIES
14. LOOK AT THE HISTORY OF CLOUD COMPANIES
15. LOOK AT THE HISTORY OF CLOUD COMPANIES
16. LOOK AT THE HISTORY OF CLOUD COMPANIES
17. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE
18. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES
19. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM
20. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise
21. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops
22. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops S(R)E
23. –Brave New Geek “Multi-Cloud Is a Trap” https:/ /bravenewgeek.com/multi-cloud-is-a-trap/
24. WORKS LIKE YOU WOULD EXPECT IT…
25. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail
26. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later…
27. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… API access, permissions, config…
28. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… CI/CD API access, permissions, config…
29. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) API access, permissions, config…
30. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) API access, permissions, config…
31. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) Task Queue (with scheduling!) API access, permissions, config…
32. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) Task Queue (with scheduling!) Logging… API access, permissions, config…
33. The Frontend – Diving into the world of JavaScript? https://exceptionnotfound.net/big-ball-of-mud-the-daily-software-anti-pattern/
34. THE FULLSTACK IN GO – SO HOW ABOUT WEBASSEMBLY (WASM)?
35. NO! BECAUSE…
36. NO! BECAUSE…
37. NO! BECAUSE… Readability, Debugging, Size…
 (but maybe in the future)
38. NO WEBASSEMBLY. MAYBE ELM THEN?
39. NO! BECAUSE… Learning curve, different paradigm, transpiling to JS, libraries… – remember, we have one week to finish!
40. VANILLA JS (ES 6)! 
 BUT WHICH FRAMEWORK?
41. RESIST THE TEMPTATION TO…
42. RESIST THE TEMPTATION TO…
43. RESIST THE TEMPTATION TO… Also, do not use Github Stars as your only criterion! Popularity is one thing…
44. HOW TO CHOOSE
45. HOW TO CHOOSE 1) Does it solve your problem in a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base.
46. HOW TO CHOOSE 1) Does it solve your problem in a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base. 2) Look at the quality of the documentation! Because you need to read it and be able to understand it – quickly!
47. VUE.JS SEEMS TO MATCH. BUT WHICH ARCHITECTURE?
48. SO MANY APPROACHES… …MVVM, Flux, Redux, SAM, ELM Architecture…? What is the essence?
50. STATE MUTATION
51. STATE MUTATION Take notice when somebody tells you that you should or can entirely avoid mutating state.
52. MUTATING STATE. HERE’S HOW…
53. MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth.
54. MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads.
55. MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads. 3) Accept or reject the state mutation in your model At the end of your (async.) action, “commit” / “propose” a new value to the model.
56. MUTATING STATE. HERE’S HOW… 1) Put your entire app state into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads. 3) Accept or reject the state mutation in your model At the end of your (async.) action, “commit” / “propose” a new value to the model. 4) Update the UI when the state changes Derive the state to render from your model, e.g. by UI bindings.
57. VUEX.JS KIND OF SUPPORTS THAT PATTERN
58. –Ryan Dahl “[...] if you’re building a server, I can’t imagine using anything other than ” THE BACKEND.
59. HOW TO BUILD?
60. HOW TO BUILD? Go 1.11 Because Go Modules!
61. HOW TO BUILD? Go 1.11 Because Go Modules! Docker To be flexible with Go versions
 Multi-stage build!
62. HOW TO BUILD? Go 1.11 Because Go Modules! Docker To be flexible with Go versions
 Multi-stage build! entr + Makefile No Rake, Grunt, Gulp, NPM, Yarn – Makefile + Bash Script fine! Check entrproject.org
63. APP ENGINE – START SIMPLE
64. APP ENGINE – START SIMPLE
65. APP ENGINE – START SIMPLE App Engine Flex You can migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster.
66. APP ENGINE – START SIMPLE App Engine Flex You can migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster. Use defaults Start with App Engine default options and auto-scaling. Nobody knows how much traffic you will get. Watch and see what happens.
67. SEPARATE THE FRONTEND AND BACKEND?
68. YES AND NO!
69. YES AND NO! Expose your services as API to the frontend Frontend is rendered via a Go template. On that rendered page, a Single Page App can run in memory and call your backend API. Use Go templates for other static pages. Makes routing easier.
70. YES AND NO! Expose your services as API to the frontend Frontend is rendered via a Go template. On that rendered page, a Single Page App can run in memory and call your backend API. Use Go templates for other static pages. Makes routing easier. But no need to create two deployment units No need for premature optimisation (scaling). One Go binary is simpler than two Go binaries! 
 Static assets are loaded from the file system in your container.
71. CI/CD
72. CI/CD Google Cloud Build Will trigger and deploy when you push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think.
73. CI/CD Google Cloud Build Will trigger and deploy when you push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think. App Engine Deployment Straightforward versioning and service concept via DNS – version.service.appname Promoting app to production, traffic splitting etc. – 
 supported out of the box
76. The cloud can be mean and slow.
77. PLAYGROUNDS SAVE TIME Create a GCP “Playground” project Got stuck with Cloud Datastore (vs. Cloud Firestore) and 
 Cloud Tasks (only certain EU region). Led me to test service offerings first via playground.
78. WHICH ARCHITECTURE – ONIONS AND HEXAGONS?
79. SO MANY LAYERS AND CONCEPTS Also, ask yourself : Is there a rich domain? Or are you really building a CRUD app?
82. Take notice when nobody tells you to separate functionality into lots and lots of really small methods and delegating objects.
83. A QUESTION OF PHILOSOPHIES?
84. A QUESTION OF PHILOSOPHIES?
85. A QUESTION OF PHILOSOPHIES? Robert Martin (Uncle Bob) http://2017.agilesummit.gr/speakers/uncle-bob/
86. A QUESTION OF PHILOSOPHIES? John Ousterhout https://www.youtube.com/watch?v=bmSAYlu0NcY Robert Martin (Uncle Bob) http://2017.agilesummit.gr/speakers/uncle-bob/
87. DEEP VS SHALLOW MODULES
88. DEEP VS SHALLOW MODULES Shallow Module Deep module Interface Functionality
89. DEEP VS SHALLOW MODULES Shallow Module “The best modules are deep: they allow a lot of functionality to be accessed through a simple interface. A shallow module is one with a relatively complex interface, but not much functionality: it doesn’t hide much complexity.” 
 – From A Philosophy of Software Design, John Ousterhout Deep module Interface Functionality
90. EXAMPLE OF A DEEP MODULE Meme Image Generation Package “meme”, image.go: Simple interface, a lot of functionality.
91. PASS-THROUGH CODE TENDS TO BE SHALLOW – BUT HOW MUCH SEPARATION?
93. Separate the outside from the inside Kind of onion, just fewer layers
94. Separate the outside from the inside Kind of onion, just fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers
95. Separate the outside from the inside Kind of onion, just fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers Inside (package meme) Core, your actual (business) functionality
96. WHAT ABOUT INTERFACES?
97. IMPLICIT INTERFACE CONFORMANCE
98. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…
99. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…
100. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here…
101. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here… To be able to inject a different implementation, you would need to either create an interface that mirrors the datastore api or create a generic interface -> shallow wrapper
102. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here… To be able to inject a different implementation, you would need to either create an interface that mirrors the datastore api or create a generic interface -> shallow wrapper Have a look at google/go-cloud for unstructured binary storage (supports GCP and AWS)
103. STRONGER ARGUMENT FOR AN INTERFACE?
104. STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But Go’s default logging has limitations.
105. STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But Go’s default logging has limitations. Even better: Structured Logging Implicit conformance to uber-go/zap for structured logging
106. STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But Go’s default logging has limitations. Even better: Structured Logging Implicit conformance to uber-go/zap for structured logging
107. HOW TO HANDLE CROSS- CUTTING CONCERNS…?
108. CONTEXT OBJECT PATTERN
109. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware.
110. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware.
111. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of passing the logger through method parameters, pass a Go context object through middleware. Still not ideal, but good enough? Also see: https://medium.com/@gosamv/using-gos-context-library-for-logging-4a8feea26690
112. FINALLY, BEING CONSISTENT…
113. FINALLY, BEING CONSISTENT…
114. FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should see the most recent list of meme reminders 
 (strong consistency supported by Cloud Datastore)
115. FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should see the most recent list of meme reminders 
 (strong consistency supported by Cloud Datastore) Eventual Consistency Meme detail view might not have been generated and stored yet – 
 but you can always apologise to the user. Eventual consistency is often fine.
116. CONCLUSION
117. CONCLUSION Keep it simple (but not stupid!) Reduce the number of “things” you need to deal with – both from an architecture and a product perspective.
118. CONCLUSION Keep it simple (but not stupid!) Reduce the number of “things” you need to deal with – both from an architecture and a product perspective. Challenge existing “best practices” Have you been brainwashed by cargo cults in the past? The “best” practices always depend on the context and scope.
119. CONCLUSION Keep it simple (but not stupid!) Reduce the number of “things” you need to deal with – both from an architecture and a product perspective. Challenge existing “best practices” Have you been brainwashed by cargo cults in the past? The “best” practices always depend on the context and scope. Develop end-to-end Allows you to make better decisions for the entire system – and to iterate quickly for your MVP!