Slide 1

Slide 1 text

Migrating Hundreds of Legacy Applications to Josef Adersberger, CTO, QAware
 @adersberger
 }THE GOOD, THE BAD, THE UGLY proud CNCF member

Slide 2

Slide 2 text

HYPERSCALE RESILIENT OPEX SAVINGS SPEED

Slide 3

Slide 3 text

CIO Let’s bring 
 all our web 
 applications
 into the cloud! I’ve already 
 canceled our
 current data center 
 contract.

Slide 4

Slide 4 text

Chief Architect Great goal! But we’ll need 
 our time.

Slide 5

Slide 5 text

Excellent! You’ve got one year.

Slide 6

Slide 6 text

CIO My priorities: (1) Security level (2) Time (3) OpEx savings (4) Migration costs

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

8 WE WERE BRAVE

Slide 9

Slide 9 text

AND WE HAD … IMPEDIMENTS

Slide 10

Slide 10 text

THE GOOD

Slide 11

Slide 11 text

11 Visibility

Slide 12

Slide 12 text

Let’s architect the cloud!

Slide 13

Slide 13 text

PACKAGED AND DISTRIBUTED AS CONTAINERS BUILD AND COMPOSED AS MICROSERVICES DYNAMICALLY EXECUTED IN THE CLOUD CLOUD NATIVE APPLICATIONS 3 KEY PRINCIPLES

Slide 14

Slide 14 text

OMG - no greenfield
 approach!? Hundreds of legacy systems!?

Slide 15

Slide 15 text

Questionnaire: Typical Questions And Their Motivation 1. Technology stack (e.g. OS, appserver, jvm) 2. Required resources (memory, CPU cores) 3. Writes to storage (local/remote storage, write mode, data volume) 4. Special requirements (native libs, special hardware) 5. Inbound and outbound protocols (protocol stack, TLS, multicast, dynamic ports) 6. Ability to execute (regression/load tests, business owner, dev knowhow, release cycle, end of life) 7. Client authentication (e.g. SSO, login, certificates) What images to provide? How many applications will be hard/ inefficient to schedule (>3 GB RAM, > 2 cores)? What storage solutions to provide? What applications will be hard to impossible to be containerized? Are there any non cloud-friendly protocols? How risky is the migration? Is the migration maybe not needed? What IAM and security mechanisms have to be ported to the cloud?

Slide 16

Slide 16 text

16 QAVALIDATOR SONARQUBE EAM TOOL QUESTIONNNAIRES JIRA XLS STATIC ANALYSIS IBM MIGRATION TOOL … MIGRATION TASKS BASIC TOUR-DE-MIGRATION SYSTEM PROPERTIES dashboard for information radiator freestyle analysis with Tableau MIGRATION DATABASE CLOUDALYZER

Slide 17

Slide 17 text

17 Emergent design of software landscapes

Slide 18

Slide 18 text

OLD APPLICATIONS (~200) MORE MODERN APPLICATIONS (~200) ALL WEB APPLICATIONS (~400) Re-architect to run on k8s on AWS lift & shift VMs to AWS EC2

Slide 19

Slide 19 text

APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token

Slide 20

Slide 20 text

APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token

Slide 21

Slide 21 text

2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure CLOUD FRIENDLY CLOUD NATIVE CLOUD ALIEN

Slide 22

Slide 22 text

2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure Sweetspot
 time and value (security, opex) Put the monolith into a container … and enhance the
 application according the 12 factors CLOUD FRIENDLY

Slide 23

Slide 23 text

Edge Service to the Rescue

Slide 24

Slide 24 text

MONOLITH MONOLITH EDGE SERVICE BACKEND CLIENTS API GATEWAY BACKEND CLIENTS BEFORE AFTER

Slide 25

Slide 25 text

MONOLITH EDGE SERVICE CLIENTS TOKEN PROVIDER TLS 1.2 + diverse user contexts (SAML, Cookie, Header, Certificate, User/Pwd, …) Change user context to OIDC identity token IAM SYSTEMS API GATEWAY TLS 1.2 + identity token TLS 1.2 + identity token Redirect to (S)SO if user context not set / valid but required user context > 10

Slide 26

Slide 26 text

Edge Service <> Edge Service <> Edge Service Chassis <> Spring Framework Spring Boot Netflix Zuul / Spring Cloud Zuul Edge Service Filters Redis Edge Library Token Service Token Cache SPI Token SPI Token API Token Handling Filter <
>
Static Content Filter <>
Security Filter <
>
Auth Plugins
Token
Provider
Web Layer
API Gateway

Slide 27

Slide 27 text

Sidecars to the Rescue

Slide 28

Slide 28 text

Container Patterns Applied • Log extraction / re-formatting (fluentd) • Scheduling (Quartz) Sidecar: Enhance container behaviour Ambassador: Proxy communication Adapter: Provide standardized interface • Configuration (ConfigMaps & Secrets to files) • TLS tunnel (Stunnel, ghosttunnel) • Circuit Breaking (linkerd) • Request monitoring (linkerd) Pod Application Container Pattern Container Other Container “Design patterns for container-based distributed systems”. Brendan Burns, David Oppenheimer. 2016

Slide 29

Slide 29 text

Kubernets 
 Constraints Initially we thought we’ll run into k8s restrictions on our infrastructure like: ‣ No support for multicast ‣ No RWX PVC available We did. But cutting these
 application requirements lead to a better architecture in each and every case.

Slide 30

Slide 30 text

THE BAD

Slide 31

Slide 31 text

STATE

Slide 32

Slide 32 text

Databases ONPREM DATA CENTER AWS DATABASES MONOLITH Databases stay in onprem data center • Advantages: • onprem/cloud version of the application can run in parallel • privacy • Disadvantages: • latency (no real problem) Activate TLS for all database connections
 (low effort on application-side but separate project on the database side)

Slide 33

Slide 33 text

Files File persistence is very restricted to keep persistent state completely out of AWS (privacy). ‣ No file writes allowed into container ‣ No RWX PVC available ‣ Files with application data written to PVC must be deleted after 15mins ‣ No NAS mounts from onprem data centers into containers allowed Migration tasks for affected applications ‣ Store files as BLOB in database or use FTPS

Slide 34

Slide 34 text

Session State 90% OF THE APPLICATIONS HAVE SESSION STATE 100% OF THE APPLICATIONS HAVE MORE THAN 1 INSTANCE Session Stickyness: not within the cloud! Session Persistence: ‣ Within existing application database: performance impact too high for most ‣ Redis: no transport encryption out-of-the-box and separate infrastructure required Session Synchronization: ‣ Application server: nope, no dynamic peer lookup within k8s ‣ In-memory data grid: Hazelcast. Nope. $$$ for TLS. ‣ In-memory data grid: Apache Ignite. Done. ‣ Embedded within application or standalone in a separate container. ‣ Little bit cumbersome but working k8s peer lookup ‣ Runs into JIT bug on IBM JVM (some methods have to be excluded from JIT)

Slide 35

Slide 35 text

DIAGNOS-
 ABILITY need a DevOps?

Slide 36

Slide 36 text

Metrics Events / Logs Traces Diagnosability

Slide 37

Slide 37 text

Metrics Events / Logs Traces Diagnosability

Slide 38

Slide 38 text

Metrics Events / Logs Traces • High effort to instrument for valuable insights •Scalability unclear for hundreds of applications •Applications have no time to run their own Prometheus instance •Scalability unclear for hundreds of applications (Jaeger & ZipKin) •Applications have no time to run their own instance •Scalability unclear (a lot of events lost) •Applications have no time to run their own EFK instance •Non-standardized log format requires custom log rewrite adapter but no fluentd DaemonSet Diagnosability

Slide 39

Slide 39 text

Metrics Events / Logs Traces

Slide 40

Slide 40 text

SECURITY need a SecDevOps?

Slide 41

Slide 41 text

We Came Far BACKEND MONOLITH EDGE SERVICE TLS 1.2 + user context API GATEWAY Two-way TLS 1.2 + identity token Two-way TLS 1.2 + identity token TOKEN CHECK TLS 1.2 + user context secrets injected at container startup security filters included CLIENTS CLIENTCERT ACL EGRESS RULES

Slide 42

Slide 42 text

The Bad: Certificate Management VAR 1: CLOUD NATIVE STYLE CERTIFICATE MANAGEMENT 
 (SPIFFE-BASED, AT SERVICE MESH OR APPLICATION LEVEL) VAR 2: REPLACE BY POLICIES (AT NETWORKING LEVEL) e.g. e.g. SPIRE

Slide 43

Slide 43 text

THE UGLY String hostRequest = new HostRequest().hostusPokus(message);

Slide 44

Slide 44 text

44 CLOUD ENABLING 
 CLOUD ALIENS

Slide 45

Slide 45 text

TOXIC
 TECH

Slide 46

Slide 46 text

The Almighty Legacy Framework • “worry-free package framework” from the early 2000s with about 500kLOC and 0% test coverage • Migration tasks • from J2EE 1.4 to JEE 7 and Java 6 to 8 • add identity token check and token relay • modify session handling (synchronization) • modify logging (to STDOUT) • modify configuration (overwrite from ConfigMap) • enforce TLS 1.2 • place circuit breakers • predefined liveness and readiness probes • Strategies: • the hard way: migrate manually and increase coverage • decorate with ambassadors, sidekicks and adapters • do not migrate parts and replace that API within the applications APPLICATION ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6

Slide 47

Slide 47 text

47 ~ 100 systems live on target by end of this year (after 8mo) ~ 200 systems live on target by end of first quarter 2018 other ~200 systems migrated by end of first quarter 2018 via virtual lift & shift. They will be migrated onto Kubernetes afterwards

Slide 48

Slide 48 text

That’s what we’ve learned from migrating hundreds
 of J2EE legacy apps
 onto Kubernetes?


Slide 49

Slide 49 text

No stupid “as-is into 
 containers” approach!

Slide 50

Slide 50 text

Getting as close as 
 possible to cloud friendly 
 application principles. Increasing the security 
 level by an order of 
 magnitude!

Slide 51

Slide 51 text

Q -> A proud member of the CNCF

Slide 52

Slide 52 text

TWITTER.COM/QAWARE - SLIDESHARE.NET/QAWARE Thank you! [email protected] @adersberger

Slide 53

Slide 53 text

BONUS SLIDES

Slide 54

Slide 54 text

54 Industrialization

Slide 55

Slide 55 text

ARCHITECTURE TEAM (~2 FTE) SUPPORT TEAM (~10 FTE) DOZENS OF MIGRATION PROJECTS RUNNING IN PARALLEL (UP TO ~80) ‣ Training sessions ‣ Support sessions ‣ Guidance (cookbook, sample application) ‣ Development environment (SEU-as-Code) ‣ Standard images ‣ Automated Refactorings (RefactoBot) ‣ Pre-migrated frameworks ‣ Solutions: Edge service, ambassadors INDUSTRIALIZATION TEAM (~6 FTE) ‣ Application blueprint (target architecture, migration rules) ‣ Migration database

Slide 56

Slide 56 text

NO SPEAK 
 CLOUD

Slide 57

Slide 57 text

DMS System APPLICATION DMS Proprietary (binary)
 TCP-level on fixed ports
 Non-encrypted
 Multiple target servers DMS ADAPTER Hard-coded DNS names as reference to other servers in response APPLICATION DMS ADAPTER STUNNEL STUNNEL DMS ONPREM KUBERNETES ON AWS POD STUNNEL STUNNEL PODS SVC.HOST1 SVC.HOST2 SVC.HOST3 lbl:host1 lbl:host2 lbl:host3 ONPREM dnshostname1.com.myapp.svc.cluster.local dnshostname1.com

Slide 58

Slide 58 text

8 PLAN BUILD RUN GRASP Continuous
 Delivery Continuous
 Feedback

Slide 59

Slide 59 text

IDE (ECLIPSE, INTELLIJ) ARTIFACTORY Kubernetes Cluster RUNNING TESTED APPLICATIONS INFORMATION RADIATOR GITHUB ENTERPRISE ACCOUNTABILITY PLUGIN JENKINS JARs
 Docker image Docker image deploy Trigger
 Code Code Status

Slide 60

Slide 60 text

No content