Building Instruqt, a gamified learning platform for IT

Bas is one of the founders of Instruqt, a startup building a hands-on learning platform for DevOps tooling and Cloud technology. With Instruqt, every participants gets his own personal infrastructure, that is being created in seconds. Bas will talk about how they leveraged Terraform, Kubernetes and Google Cloud to create a scalable and cost-effective learning platform. He will discuss the choices they've made, the problems they encountered and the lessons learned.

Transcript

1. Building instruqt, a scalable learning platform DevOps Amsterdam meetup March 14th 2018
2. Bas Tichelaar Start new things Writing code Entrepreneur I love skiing! DevOps (Google) cloud (pre-)sales Xebia Instruqt Inspire Challenge Connector Consultant Co-founder Startups Automation Solving problems Marketing Drumming
3. Audience check
4. What is your way of learning new stuff?
5. (official) classroom training
6. Online learning courses
7. Reading documentation
8. Learning on the job
9. What is instruqt?
10. Gamified learning platform for IT.
11. Build skills by doing.
12. Think for yourself. No more monkey see, monkey do.
13. Unlock challenges to complete tracks and earn badges
14. Create tracks yourself using the open SDK
15. Other use-cases are assessments, guided demo’s and sandbox environments.
16. This is the 4th version of the platform
17. Iteration #1
18. Iteration #2
19. Iteration #3
20. Current version
21. Technologies we use. Golang React Terraform Google Cloud Functions Bash Docker Firebase Google Cloud Container Builder Google Datastore Google Storage Kubernetes Google Kubernetes Engine Google Container Registry Google PubSub Redux
22. Technologies we use. Golang React Terraform Google Cloud Functions Bash Docker Firebase Google Cloud Container Builder Google Datastore Google Storage Google Kubernetes Engine Google Kubernetes Engine Google Container Registry Google PubSub Redux
23. What is being created by the platform? Kubernetes namespace Shell pod SSH service Gotty service SSH keys Alpine container
24. User namespace Exposed via ingress & proxy Proxy JWT token validation Gotty service play.instruqt.com Firebase
25. Make it scale.
26. Google Cloud Platform does the heavy lifting for us.
27. Everything runs in containers
28. Our containers are managed by Kubernetes
29. We don’t manage Kubernetes ourselves, we use Google Kubernetes Engine (GKE).
30. Benefits of using GKE Automatic scaling Automatic upgrades Separate node-pools Natively supported by Terraform Fast release cycle Integration with GCP IAM Google created Kubernetes Google Cloud networking
31. Continuous Delivery the GCP way
32. Google Cloud Container Builder builds our containers. No more build-servers!
33. Our build pipeline Track directory • config.yml • track.yml • challenges Instruqt Trackbuilder Google Cloud Storage bucket Google Cloud Container Builder POST instruqt track build upload trigger Google Container Registry Image
34. Automated deployment Google Container Registry Image Google Cloud PubSub Google Cloud Container Builder Google Cloud Function Generate Kubernetes YAML file Kubernetes New container Old container POST
35. Benefits of using a pipeline like this No central buildserver No more Jenkins (jay!) Deployment across projects Integration with GCP IAM No external services required, all Google Cloud Functions are small and simple Automatic build logs
36. Tips when creating a similar pipeline
37. Annotate pods with a timestamp, so it will always update the old pod.
38. PubSub has a slow-start mechanism. Be prepared for some waiting time!
39. PubSub cannot be used cross project. Use a cloud function that publishes the messages across projects. PubSub Cloud Function Cloud Function PubSub Project 1 Project 2
40. Terraform infrastructure as code done right
41. Terraform creates, updates and destroys our infrastructure
42. We use Terraform to create our own infrastructure as well as infrastructure for users
43. Track container Google Cloud Storage bucket Terraform: create user infrastructure config.yaml Terraform template (Golang template) Rendered template terraform apply Infrastructure Terraform state
44. Track container Google Storage bucket Terraform: destroy user infrastructure terraform destroy Terraform state
45. Benefits of using Terraform Immutable infrastructure Support for all our resources Integration with Google IAM Declarative Golang template + Terraform = powerful! Stored state of infrastructure Agentless (API + SSH) Dry-run: terraform plan
46. How do we scale?
47. Open platform Create and play content for free
48. The SDK makes creating content easy. Curation will become a challenge!
49. Pain points we encountered
50. Custom built containers didn’t scale. We inject our tools now, so you can use unmodified containers.
51. GKE auto upgrades are great. But make sure your critical services have at least 3 replicas running :)
52. Our infrastructure is extremely dynamic. Housekeeping is crucial to keep the bill low!
53. Some tools have a looong startup time, but we need it to start instantly. Our solution: VM pooling,
54. VM pool VM pooling Kubernetes VM Kubernetes VM Kubernetes VM Label: available=true Kubernetes VM Label: available=false A new VM is being created by terraform terraform apply
55. Key takeaways
56. Standing on the shoulders of giants
57. Leverage cloud services where possible. You probably won’t switch cloud providers anyway.
58. Cloud providers are better in security (Spectre, Meltdown) and scalability than us.
59. Our mission is to make learning IT more hands-on and fun. You can help us by creating tracks yourself!
60. Thank you. Go to play.instruqt.com to create an account! Bas Tichelaar bas@instruqt.com @bastichelaar linkedin.com/in/bastichelaar