Building a modern SaaS in 2020

Building a modern Software as a Service platform brings a lot of interesting engineering challenges. During this talk, I’m going to share my team’s journey of building a SaaS from scratch in 2020. First, we are going to start with the technologies and the architecture we picked. Then, we’ll go over the interesting challenge of implementing multitenancy. And we'll see how we benchmarked three different options and picked one. And last but not least, we’ll explore how every startup can use open source technologies to build observability infrastructure. And how to run their SaaS in production.

Transcript

1. Building a modern SaaS in 2020
   Nikolay Stoitsev, Engineering Manager @ Halo DX
   

   View Slide

2. View Slide

3. View Slide

4. View Slide

5. Language
   Architecture
   Repo Structure
   Builds
   Running in Production
   Observability
   Multitenancy
   

   View Slide

6. Language
   Architecture
   Repo Structure
   Builds
   Running in Production
   Observability
   Multitenancy
   

   View Slide

7. Language evaluation
   Python
   JavaScript/Node.js
   Java
   C#/.NET Core
   

   View Slide

8. Domain Specifics
   FIPS 140-2
   

   View Slide

9. Node.js
   Can build with special OpenSSL
   

   View Slide

10. Java
    Package with verified implementation of cryptographic algorithms
    

    View Slide

11. View Slide

12. View Slide

13. DICOM
    

    View Slide

14. Imaging
    Center
    

    View Slide

15. Imaging
    Center Hospital
    

    View Slide

16. HL7
    Imaging
    Center Hospital
    

    View Slide

17. HL7 DICOM
    

    View Slide

18. Java and Spring Boot
    

    View Slide

19. Angular
    

    View Slide

20. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

21. Microservices
    Discovery
    

    View Slide

22. Microservices
    Discovery
    Load Balancing
    

    View Slide

23. Microservices
    Discovery
    Load Balancing
    Circuit Breaking
    

    View Slide

24. Microservices
    Discovery
    Load Balancing
    Circuit Breaking
    Testing
    

    View Slide

25. Microservices
    Discovery
    Load Balancing
    Circuit Breaking
    Testing
    Debugging
    

    View Slide

26. Monolith
    

    View Slide

27. Failure Isolation
    

    View Slide

28. 6 services
    

    View Slide

29. Hexagonal Architecture
    Web
    Adapter
    Messaging
    Adapter
    Input Port
    Input Port
    Biz Logic
    Output Port
    Output Port
    Persistence
    Adapter
    Messaging
    

    View Slide

30. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

31. Multi Repo
    vs
    Mono Repo
    

    View Slide

32. Mono Repo
    Easily introduce and refactor
    global features
    Ease code sharing and reuse
    Simple dependency management
    Flexible team boundaries and code
    ownership
    Collaboration across teams
    Atomic commits across services
    

    View Slide

33. Build System
    Bazel
    ● Fast and incremental builds
    ● One tool to build multiple
    languages
    ● Extensible
    ● Google, Pinterest, Uber,
    Stripe
    Buck
    ● Fast and incremental builds
    ● One tool to build multiple
    languages
    ● Helps you better understand
    your dependencies
    ● Facebook, Airbnb
    

    View Slide

34. Multi-Project Builds in
    Gradle
    

    View Slide

35. Multi-Project Builds in Gradle
    ● One root project
    ● One subproject for each service
    ● One subproject for shared libs
    ● https://docs.gradle.org/current/userguide/multi_project_builds.html
    

    View Slide

36. Multi-Project Builds in Gradle
    

    View Slide

37. Nx for front-end
    monorepo
    https://github.com/nrwl/nx
    

    View Slide

38. Gradle wrapper for front-end builds and running tests
    

    View Slide

39. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

40. Separate Dockerfile
    for each service
    

    View Slide

41. Can have one Docker Compose for everything
    

    View Slide

42. Nexus
    https://www.sonatype.com/nexus/repository-oss
    

    View Slide

43. Why Nexus?
    Can store and distribute
    Maven/Java, npm, Helm, Docker
    Widely adopted
    Single source of truth for every
    component
    Easy to work with
    

    View Slide

44. Build Process
    CI
    Code
    

    View Slide

45. Build Process
    CI CD
    Code
    

    View Slide

46. Build Process
    CI CD
    Build
    Docker
    Image
    Code
    

    View Slide

47. Build Process
    CI CD
    Nexus
    Build
    Docker
    Image
    Code
    Docker
    Image
    

    View Slide

48. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

49. Kubernetes
    https://kubernetes.io
    

    View Slide

50. Why Kubernetes?
    Deployment
    Scaling
    Self-healing
    Inter-service communication
    Storage orchestration
    Scalable and battle tested
    

    View Slide

51. Great for engineers
    ● Each service is accessed via a single domain
    ● Logs are easy to read
    ● A lot of materials on how to learn it
    

    View Slide

52. https://github.com/rancher/rancher
    

    View Slide

53. Monitoring and Grafana out of the box
    

    View Slide

54. Configuring a service
    in Kubernetes
    ● Deployment
    ● Service
    ● Config Map
    ● Service Account
    ● RBAC
    ● Pod disruption budget
    ● ....
    

    View Slide

55. Helm
    https://github.com/helm/helm
    

    View Slide

56. Helm
    ● Abstraction over k8s that simplifies working with it
    ● The building block in helm is called a helm chart
    ● Help you define, install, and upgrade applications in k8s
    

    View Slide

57. How to work with Helm
    ● Search for “{technology} helm chart”
    ● Download helm chart
    ● Update values.yaml
    ● $ helm install {release-name} {chart-name} --set {var}={val}
    

    View Slide

58. Deploy Process
    CI CD
    Nexus
    Build
    Docker
    Image
    Docker
    Image
    

    View Slide

59. Deploy Process
    CI CD
    Nexus
    Kubernetes
    Kubernetes
    Kubernetes
    

    View Slide

60. Deploy Process
    CI CD
    Nexus
    Helm
    Update
    Kubernetes
    Kubernetes
    Kubernetes
    

    View Slide

61. Deploy Process
    CI CD
    Nexus
    Helm
    Update
    Kubernetes
    Kubernetes
    Kubernetes
    Pull Image
    

    View Slide

62. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

63. SLF4J backed by Log4j2
    https://github.com/qos-ch/slf4j
    

    View Slide

64. Backend logging
    EFK - Elasticsearch, Fluentd, Kibana
    Pod
    Pod
    Pod
    

    View Slide

65. Backend logging
    EFK - Elasticsearch, Fluentd, Kibana
    Pod
    Pod
    Pod
    Fluentd
    Fluentd
    Fluentd
    

    View Slide

66. Backend logging
    EFK - Elasticsearch, Fluentd, Kibana
    Pod
    Pod
    Pod
    Fluentd
    Fluentd
    Fluentd
    Elasticsearch
    

    View Slide

67. Backend logging
    EFK - Elasticsearch, Fluentd, Kibana
    Pod
    Pod
    Pod
    Fluentd
    Fluentd
    Fluentd
    Elasticsearch Kibana
    

    View Slide

68. Front-end logging
    

    View Slide

69. Front End Logging Library
    

    View Slide

70. Front End Logging Library
    

    View Slide

71. Front End Logging
    front end lib
    front end lib
    

    View Slide

72. Front End Logging
    Backend
    Service
    Backend
    Service
    front end lib
    front end lib
    

    View Slide

73. Front End Logging
    Backend
    Service
    Backend
    Service
    Fluentd
    Fluentd
    front end lib
    front end lib
    

    View Slide

74. Front End Logging
    Backend
    Service
    Backend
    Service
    Fluentd
    Fluentd
    Elasticsearch
    Kibana
    front end lib
    front end lib
    

    View Slide

75. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    

    View Slide

76. Subtenancy
    Org
    East West
    Sales
    Front
    Desk
    Sales
    Front
    Desk
    

    View Slide

77. Materialized path
    (1) Org
    (2) East (3) West
    (4) Sales
    (5) Front
    Desk
    (6) Sales
    (7) Front
    Desk
    ID Path Name
    1 1/ Org
    2 1/2/ East
    3 1/3/ West
    4 1/2/4 Sales East
    5 1/2/5 Front Desk East
    6 1/3/6 Sales West
    7 1/3/7 Front Desk West
    

    View Slide

78. Closure table
    (1) Org
    (2) East (3) West
    (4) Sales
    (5) Front
    Desk
    (6) Sales
    (7) Front
    Desk
    ID Name
    1 Org
    2 East
    3 West
    4 Sales East
    5 Front Desk East
    6 Sales West
    7 Front Desk West
    Ancestor Descendant
    1 2
    1 3
    1 4
    1 5
    1 6
    1 7
    2 4
    2 5
    3 6
    3 7
    4 Null
    

    View Slide

79. Keeping hierarchy and data separated
    (1) Org
    (2) East (3) West
    (4) Sales
    (5) Front
    Desk
    (6) Sales
    (7) Front
    Desk
    ● Each records get a tag - 1, 2, 3, etc.
    ● Each tag points to a node in the tree
    ● Tag hierarchy is loaded separately
    ● Queries are decorated with OR
    statements to match by tag
    

    View Slide

80. Benchmark
    

    View Slide

81. Benchmark
    

    View Slide

82. Language Java and Spring Boot
    

    View Slide

83. Language
    Architecture
    Java and Spring Boot
    Microservices
    

    View Slide

84. Language
    Architecture
    Repo Structure
    Java and Spring Boot
    Microservices
    Monorepo
    

    View Slide

85. Language
    Architecture
    Repo Structure
    Builds
    Java and Spring Boot
    Microservices
    Monorepo
    Gradle and Docker
    

    View Slide

86. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Java and Spring Boot
    Microservices
    Monorepo
    Gradle and Docker
    Kubernetes
    

    View Slide

87. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Java and Spring Boot
    Microservices
    Monorepo
    Gradle and Docker
    Kubernetes
    EFK stack
    

    View Slide

88. Language
    Architecture
    Repo Structure
    Builds
    Running in Production
    Observability
    Multitenancy
    Java and Spring Boot
    Microservices
    Monorepo
    Gradle and Docker
    Kubernetes
    EFK stack
    Separate hierarchy
    and data
    

    View Slide

89. Thank you! Q&A
    Nikolay Stoitsev
    Engineering Manager at Halo DX
    

    View Slide