Detecting risks in unstructured data by scaling the execution of Data Science

How Deloitte has detected “the signal in the noise” to solve client regulatory and risk challenges by creating a flexible API platform (“Infinity Architecture”) for executing Machine Learning Models and Risk engines at scale. Leveraging the elastic scalability of AWS along with Open Source and custom solutions to process audio, documents and other unstructured data to provide insights and detect risks and issues for clients.

Transcript

1. Detecting risks in unstructured data by scaling the execution of Data Science Speakers: Oliver Cronk – Chief Architect Risk Advisory, Deloitte LLP Allan Smeyatsky – Solution Architect Risk Advisory, Deloitte LLP
2. 2 Introductions Oliver Cronk Chief Architect - Risk Advisory – Deloitte LLP Oliver is an MBCS Chartered IT Professional and AWS Cloud Practitioner certified. Oliver leads on Architecture, Engineering and Digital strategy for risk management products and services offered by Deloitte Risk Advisory. Platform owner for “Infinity Architecture”. @cronky https://www.linkedin.com/in/cronky/ Allan Smeyatsky Cloud Solutions Architect – Risk Advisory - Deloitte LLP Allan is a multi-cloud certified Solutions Architect with over 25 years’ experience dealing with contact centre and audio solutions. Allan is the Lead Architect on the “Infinity Architecture” platform. More recently Allan has become certified across all 3 major cloud platforms including AWS SA Professional level certification. https://www.linkedin.com/in/allansmeyatsky/
3. 3 Tax Consulting Audit and Assurance Risk Advisory Internal Client Services Financial Advisory Technology Strategy & Architecture Financial Services Applications Digital Systems Integration Technology Strategy & Architecture Analytics & Information Management Enterprise Applications Human Capital Strategy & Operations Technology Oliver & Allan (building client facing solutions powered by AWS) Internal IT (also AWS users) AWS Consulting services to clients
4. 4 Our Cloud offerings provide us the ability to help our clients at every stage of the cloud journey and at scale allowing our clients to focus on business outcomes O U R C L O U D O F F E R I N G S Digital DNA Organizational & Transformation Change Management Cloud Leadership Development & Teaming Cloud Workforce Strategy & Planning Marketing & Communications Support Risk Powers Performance Governance & Regulatory Compliance Business Continuity & Resilience Cyber Risk Data Security & Regulation Extended Enterprise & 3rd-party Risk Operational Enablement Cloud Impact to Facilities & Assets Strategic Sourcing Vendor Ecosystem Governance Tax Integration Services Cloud Strategy & Readiness Business Model Disruption  Cloud Strategy & Operating Model  Cloud Economics  Organizational Alignment & Readiness  Cyber Risk Assessment  Cloud Suitability & Planning SaaS Implementation Enterprise Resource Planning (ERP)  Customer Relationship Management (CRM)  Human Resources Transformation  Application Security  Other SaaS Domains Custom Implementation Cloud Native Application Architecture & Development  Orchestration & Automation  Application Development  Micro-Services Cloud Enabled Solutions API Management & Integration  Software Defined Data Center & Networking  DevOps  DevSecOps  Deloitte OpenCloud Cloud Migration Cloud Migration Services  Cloud Platform Engineering & Implementation  Cloud Transition & Integration Management  Application Modernization  Cyber Risk Transformation Cloud Managed Services Cloud Managed Services  Deloitte Managed Analytics Platform  BPaaS Services I m a g i n e D e l i v e r R u n E n t e r p r i s e S e r v i c e s
5. 5 Context Productising / codifying Risk Advisory Intellectual Property (traditionally delivered as people based services) Compliance Rules, Regulation Analysis, Risk Models, Risk Engines, Algorithms …
6. 6 Conceptual Architecture Enabler for risk products and services Deloitte Intellectual Property Secured Scaled Protect Service Enabled Risk Advisory Assets: models / algorithms / AI & ML Wrapped, scaled and protected by a Software Engineering & Operations Platform Queues Client Direct (API) Requests Workers Workers Deloitte Products & Apps (via API & S3) Service Security, Orchestration & Load Balancing Response back to Clients / Deloitte Apps (API / S3) Workers “Auto-scaling” Queues / Output
7. 7 Roadmap  x Infinity Architecture Infinity Core API Management License Management & Security Orchestration & Configurable Workflows Elastically Scaled Workers (Microservices) Billing Enabling pay per usage models AI/ML Model Execution Restrictions on geographic jurisdiction of processing Billing events captured Near real time (Async) & Real time* (Sync) Message Queues Secure storage areas for processing duration only Secure storage areas for models & reference data
8. 8 So what do these workers do? The following workers have been created for the Infinity Architecture: Feature Extraction Worker Scoring Worker Callback Worker Anti-Virus Worker Delete Worker Orchestrator Preprocessing Worker Audio Worker Text Worker Transcription Worker External API Worker R Wrapper Worker OCR Worker
9. 9 Infinity API Deloitte have developed a generic API that will be used by all clients There are four endpoints that have been developed for the API:  Upload Destination (POST)  Job (POST)  Job (GET)  Immediate-task (POST) In addition there is an API to send results out of the solution:  Callback
10. 10 Architecture Orchestrator Amazon S3 S3 Amazon ElastiCache Redis Cluster 4.0: - Celery Broker Auto Scaling Amazon API Gateway API Gateway Internet Subscription Client Subscription Client AWS cloud Amazon VPC Elastic Load Balancing Private Network Load Balancer Lambda: API call Headers modification, authorization Service Cluster Mode : ASYNC/SYNC Orchestrator Worker Service Preprocessing Worker Service Callback Worker Service Lambda: Queue Monitoring And Cloudwatch Alarm management Amazon CloudWatch Amazon CloudFront Application Load Balancer DynamoDB <env> - configurations DynamoDB <env> billing-events Private Subnet Temporary Data Store Infinity Configuration Manager Service Cluster DynamoDB <env> - clients DynamoDB <env> - workflows <env> -upload-destination Lambda: Temporary Upload Location Amazon VPC Deloitterisk.cloud VPC peering VPC NAT gateway Private Subnets (Multi AZ) <env> -cf-secrets Amazon ECR Docker Repository Jenkins Code Repository Amazon SQS Subscription Client Lambda: events consumer Audio Worker Service Text Worker Service Transcription Worker Service Scoring Worker Service Feature Extraction Worker <env> -spdra-working <env> -cf-templates <env> -S3-secrets <env> -S3-monitoring -trail <env> -S3-scoring -worker-files <env> -S3-config-man -files <env> Installers <default> <env> Outcomes <default> Deployment Lambda : AutoScale Anti-Virus Worker (ASYNC) Delete Worker S3 - Callback Anti-Virus Worker (SYNC) VPN Gateway OpenVPN DRAWN BY SMEYATSKY, ALLAN (UK - LONDON) 10/05/2019 Customer Management Platform AWS WAF OCR Worker External API Worker
11. 11 AWS Services & Custom Components Amazon VPC Amazon API Gateway Amazon ECS Redis Lambda function Things we needed to build to complement AWS services: + Orchestrator and Queue based auto-scaling scaling solution + Poison Pill to kill off superfluous workers + Python Worker Framework + Anti Virus Scanning for S3 Compute Network Database ECS SQS AWS CloudFormation Amazon CloudWatch Amazon DynamoDB API Gateway VPC Lambda Function CloudFormation DynamoDB CloudWatch Application Load Balancer Redis Amazon S3 Storage S3
12. 12 Continuous Integration / Deployment DEV TEST PROD UAT
13. 13 Summary What we have learnt?  Massive amount about AWS, Serverless and Containers and where the limitations are  Good collaboration with Data Scientists is key  Data Scientists complemented with cloud architecture and engineering is a powerful scalable combination  Cost Optimisation – use of Spot and Reserved instances vs on demand  AWS services move quickly – be prepared to re-evaluate choices What would we do differently now?  Leverage SageMaker, Fargate or AKS (these weren’t available or mature at the start of the project)  Create Sandbox environments earlier What Next?  “Infinity Connect” - Managed data connection and management  “Infinity Experience” – next generation of our scalable Serverless front end leveraging GraphQL & AppSync  AWS SageMaker, AWS Batch and AWS Step Functions
14. This publication has been written in general terms and we recommend that you obtain professional advice before acting or refraining from action on any of the contents of this publication. Deloitte LLP accepts no liability for any loss occasioned to any person acting or refraining from action as a result of any material in this publication. Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 1 New Street Square, London, EC4A 3HQ, United Kingdom. Deloitte LLP is the United Kingdom affiliate of Deloitte NWE LLP, a member firm of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”). DTTL and each of its member firms are legally separate and independent entities. DTTL and Deloitte NWE LLP do not provide services to clients. Please see www.deloitte.com/about to learn more about our global network of member firms. © 2019 Deloitte LLP. All rights reserved.