Slide 1

Slide 1 text

HOW LINEDATA STREAMLINED CI/CD AND OPTIMIZED AWS CLOUD SPEND Andrey Budzar DevOps Lisbon 10/2020

Slide 2

Slide 2 text

DEVOPS LISBON CONTENTS • 01 – About Linedata • 02 – DevOps at Linedata • 04 – Continuous Infrastructure as Code: Harness + Terraform • 05 – GitOps • 06 – Summary • 07 – GitOps Demo

Slide 3

Slide 3 text

DEVOPS LISBON 01 ABOUT LINEDATA

Slide 4

Slide 4 text

WHAT WE DO : ENABLE YOUR FIRM’S EVOLUTION Alternative Managers Institutional Managers Wealth Managers Management Companies Private Equity Administrators TPA/Fund Administrators Asset Owners Portfolio Management Order Management & Trading Middle Office Oversight NAV Solution Compliance Risk Management Fund Accounting & Valuation ReportingTransfer Agency Operational Analytics Outsourcing We build with powerful new technologies and deliver cloud-based, integrated software, data and services to help you adapt and scale your business, embrace digitization and position for the future. We help our clients around the world evolve and operate at the highest levels. 10 of the world’s 30 largest asset management companies trust our investment solutions. Whatever your business Whatever your needs We create smart solutions Linedata’s technology solutions combine software, services, data and 20 years of experience, to position our global clients to innovate, streamline and optimize investment performance and operations for growth and success. SOFTWARE, SERVICES, DATA & ANALYTICS FOR ASSET MANAGERS DEVOPS LISBON

Slide 5

Slide 5 text

Seattle New York Boston Toronto Tunis Madrid Paris Dubli n London Luxembour g Edinburgh Riga Chennai Hong Kong Casablanc a Greenwic h Ahmedabad Mumbai Oxford Chicago GLOBAL DEVOPS AT LINEDATA EMPOWERING YOUR BUSINESS WITH TECHNOLOGY, SERVICES AND DATA SOLUTIONS DEVOPS LISBON 1,300+ people 700 clients 20 offices 200m revenue US$12.5T managed by AM clients

Slide 6

Slide 6 text

DEVOPS LISBON 02 DEVOPS AT LINEDATA

Slide 7

Slide 7 text

TYPE 5 DEVOPS TOPOLOGY + SCRIPTING Dev Ops DevOps What organizations try to implement Type 5: DevOps Team with an expiry date Dev Ops DevOps What usually happens Added operational overhead Anti-Type B: Permanent DevOps Team Silo Dev Ops DevOps DEVOPS LISBON

Slide 8

Slide 8 text

DEVOPS METHODOLOGY AT LINEDATA Smart Automation (Continuous Deployment) Secure Infrastructure-as-Code Cost Management-as-Code Self Service Continuous Verification Continuous Security DEVOPS LISBON

Slide 9

Slide 9 text

LINEDATA DEVOPS TOPOLOGY – SRE MODEL Group Level Platform Engineering DEVOPS LISBON Secure Infrastructure-as-Code Business Unit SRE DevOps Dev Self Service

Slide 10

Slide 10 text

DEVOPS LISBON 03 CONTINUOUS INFRASTRUCTURE-AS-CODE

Slide 11

Slide 11 text

AUTOMATION MATURITY DEVOPS LISBON LEVEL 0 Ad hoc • Minimal automation • Individually driven • Script based • Some tools adopted LEVEL 1 Opportunistic • Targeted Automation • Team or project driven • Reactive in nature • Platform and tools evaluated and adopted LEVEL 2 Systematic • Automation defined with metrics • Expert driven • Proactive in nature • Roadmap specified LEVEL 3 Institutionalized • Roadmap in action across the organization • Organization driven • Roadmap specified • Automation becomes way of life LEVEL 4 Adaptive • Automation becomes adaptive to process being automated • Self-learning, self-healing, with optimization methods in place • Widespread use of machine learning and AI • Automation inherently becomes smart Up to 5% 5%-10% 10%-25% 25%-50% >50% Productivity Gains ADOPTION • Automation realized with portfolio of platforms and tools

Slide 12

Slide 12 text

DEVOPS LISBON DEPLOYMENT SCRIPTING != CONTINUOUS DELIVERY Environments are static and run 24/7 10-20 scripts per pipeline Secrets stored in config files Terraform manually triggered by Developers Jenkins manually triggered by Developers and takes hours to go from Dev to Production.

Slide 13

Slide 13 text

DEVOPS LISBON CONTINUOUS INFRASTRUCTURE AS CODE • Layered Modules (outputs.tf) • Dynamic • Re-usable • Embedded Security • Auditable • Streamlined Tagging • Automated Approvals • Change Control

Slide 14

Slide 14 text

DEVOPS LISBON CD WITH HARNESS + TERRAFORM On-Demand Environments reduce cloud costs Terraform auto-provisions envs/infra on pipeline execution and auto-destroys on pipeline completion Harness pipeline auto-triggered on new build/artifact and takes mins to go from Dev to Production Single Harness pipeline template Pull request Developers create and templatize pipelines in minutes

Slide 15

Slide 15 text

DEVOPS LISBON 05 GITOPS

Slide 16

Slide 16 text

DEVOPS LISBON DEFINITION Describes the desired state of the whole system using a declarative specification for each environment A git repo is the single source of truth for the desired state of the whole system All changes to the desired state are approved Git commits When the desired and observed states are not the same then: There is a convergence mechanism to bring the desired and observed states in sync both eventually, and automatically This is triggered immediately or manually with an approved “change committed” After a configurable interval, an alert “diff” may also be sent if the states are divergent All Git commits cause verifiable and idempotent updates to the infrastructure Rollback is: “convergence to an earlier desired state” (Revert commit) https://www.weave.works/technologies/gitops/

Slide 17

Slide 17 text

DEVOPS LISBON GITOPS OPERATING MODEL Git as the single source of truth of a system’s desired state GitOps Diffs compare desired state with observed state (Terraform Plan) ALL intended operations are committed by pull requests, for all environments ALL diffs between GIT and observed state lead to convergence (Terraform Apply) ALL changes are observable, verifiable, audited indisputably, with rollback https://www.weave.works/technologies/gitops/

Slide 18

Slide 18 text

DEVOPS LISBON 06 SUMMARY

Slide 19

Slide 19 text

SELF SERVICE QA SRE Developer Sales/Product Feedback ~60 min on average Feedback DEVOPS LISBON Feedback Platform Pull request

Slide 20

Slide 20 text

IMPACT OF HARNESS + TERRAFORM POWERED DEVOPS AT LINEDATA Economic Benefits ~50% Savings on Entire AWS bill Granular visibility into Cloud spend Agility = Velocity of Revenue Efficiency = Increased DevOps ROI Cultural Benefits Improved Quality of Life across the business Value-driven DevOps instead of day-to-day task automation Template Library decreases handoff friction and empowers innovation Scalability through a frictionless automation framework DevOps enabled self-service supporting global collaboration and innovation DEVOPS LISBON

Slide 21

Slide 21 text

DEVOPS LISBON BENEFITS AND CHALLENGES OF GITOPS BENEFITS Self-Service IT Transparency (inherently documented) Collaboration between Dev and Ops (learning opportunities) Scalability Security Increased ROI CHALLENGES Design and implementation Steep Learning curve Discipline

Slide 22

Slide 22 text

DEVOPS LISBON DEMO

Slide 23

Slide 23 text

DEVOPS LISBON GITPS + IAC ECOSYSTEM Git Orgs Pull Requests Orchestrator Delegates Infrastructure-as-code Modules … … AWS … …

Slide 24

Slide 24 text

DEVOPS LISBON AWS INFRA GITOPS (HARNESS + TF) VPC, IAM, KMS, SECRETS, SSM, ETC 1a. Pull Request devops- Harness Role Assume role Assume role Assume role Harness Role Harness Role Harness Role Amazon EC2 2a. Approve PR 2b. Pull Request if request for change 2c. Approve from 2b Plan Apply 4. Approval ss h 3. Execute Change (RBAC + Delegate Scoping) 1b. KEN ticket for PR or request for change AWS CloudTrail AWS Organizations CPT != terraform

Slide 25

Slide 25 text

DEVOPS LISBON GIT PATTERNS MAKING CHANGES All orgs, repos auto-protect «live» branches (Harness execute limited to protected branches) Changes via PRs 1. Create a patch branch 2. Raise PR 3. Auto-plan (deltas) 4. Wait for PR approval & merge 5. Auto-apply with pending approval 6. No self-approvals, enforce admins RBAC controlled Full change audit trail (commits & PRs) Bi-directional link between Harness execution and PR

Slide 26

Slide 26 text

DEVOPS LISBON Q&A