Teradata Data Expo 2024

Transcript

1. © 2024 Teradata. All rights reserved. Delivering Trusted AI at Scale to drive Business Value Dr Chris Hillman Director of AI & Data Science International
2. © 2024 Teradata. All rights reserved. Seller in the era of AI Three things that are top of mind in the era of AI Generative AI Where’s the Value? Risk and Regulation
3. © 2024 Teradata. All rights reserved. Companies need to execute AI/ML projects at scale that deliver sustainable value and predictive outcomes Critical capability #1 Data 80% of AI project time is spent on data preparation. Reducing data movement and reusing data are both critical to improving efficiency. Critical capability #2 Scale With ubiquitous machine learning on the rise, AI/ML adoption will increase, requiring a 100x increase in the number of models and queries Critical capability #3 Deployment Companies need the ability to productionise models in a system that requires little to no manual intervention from data scientists or governance teams. A successful AI/ML solution will enable these 3 critical capabilities:
4. © 2024 Teradata. All rights reserved. © 2024 Teradata. All rights reserved. 4 Strategic Transformation with AI/ML What business leaders want from AI… Optimisation of Internal Operations New and improved products and services Disrupt whole industries and marketplaces
5. © 2024 Teradata. All rights reserved. Internal use only 5 …and what they often get ❌ Lab projects ❌ Departmental solutions ❌ Narrow use-cases that are useful, but not transformative ❌ Hallucinations, failures - and bacon on your ice cream
6. © 2024 Teradata. All rights reserved. 6 Internal use only Getting to Trusted AI Trusted AI starts with trusted data - because the foundation of AI is clean, reliable, trustworthy data New tools and technologies that generate signal from huge volumes of transaction and interaction data Insight that is embedded in operational business processes to drive outcomes and results
7. © 2024 Teradata. All rights reserved. How enterprises can solve the challenges of Trust Define Trusted AI A clear definition of trusted AI helps in setting precise expectations for what AI systems should achieve. Align Trusted AI principles When organizations explicitly state their trusted AI principles, it helps build trust and confidence among users, stakeholders, and customers. Implement a Trusted AI Capabilities Framework A Framework that translates high-level principles into technical capabilities. TRUSTED AI
8. © 2024 Teradata. All rights reserved. © 2024 Teradata. All rights reserved. Bias in, Bias Out
9. © 2024 Teradata. All rights reserved. A Trusted AI Capabilities framework Integrated , harmonized data Analysis for bias and fairness Curated for AI Data Protection Accuracy Explainability and Traceability Governed Secured DATA SIGNAL PREDICTIVE AI GENERATIVE AI Operationalization High performance Least cost to scale Operational Resiliency Workflows and Applications
10. © 2024 Teradata. All rights reserved. 10 Operationalizing Signals: the art and science of data in motion Operationalizing signal involves extracting actionable information derived from insights (signal), and seamless delivery into application workflows with low latency, high scalability, and robustness. Signal. • Represents the true, consistent patterns in data, distinct from irrelevant noise, and is the main target of analytical models. • Derived from well-crafted features are pivotal for isolating a salient signal, driving model accuracy and robust, actionable analytical insights • Advanced AI refines intricate multi-dimensional data to extract salient signal vital for accurate, strategic data-driven decision making. Operationalization • Accessibility of Advanced AI: Democratize access to sophisticated AI predictions by encapsulating these capabilities in services and publishing across diverse business functions • Scalability and Efficiency : Efficient resource allocation in response to demand to manage fluctuating workloads and enabling real-time analysis in critical applications • Rapid Integration and Iteration : Agile approach for embedding analytics and machine learning within business operations accelerating deployment, analysis, and refinement.
11. © 2024 Teradata. All rights reserved. © 2024 Teradata. All rights reserved. Garbage in Garbage out
12. © 2024 Teradata. All rights reserved. A bank using signals to operationalize fraud prevention A bank scores models on VantageCloud to predict the probability whether a payment is fraud or not. Based on this score, signals are generated for what action needs to be taken. The conversion of a model score to a signal requires integration with operational data such as customer master data 70% - 95% probability of fraud payment >95% probability of fraud payment 50% - 70% probability of fraud payment SMS Intervention to verify identity Block the payment Manual Authentication by Fraud-Ops Model Outputs Signal for action to take
13. © 2024 Teradata. All rights reserved. 13 The one pipeline per model approach The feature store approach Source data Data prep pipeline Model specific ADS Model training and scoring Integrated data foundation Enterprise Feature Store Model specific ADS Model training and scoring ¿ ¿ ¿ ¿ ¿ ¿ ¿ Write back of new features and model outputs is possible One pipeline per model Redundant infrastructure, processing & effort Limited re-use of pipeline or features DSs functioning mostly as data janitors Long data prep cycles & poor time to market High TCO Poor productivity and data silos Inefficient allocation of resources “Off the peg” features dramatically improve analytic cycle time and time-to-market Extensive re-use reduces TCO and improves analytic data quality and predictive model accuracy ADS layer enables model-specific customization, whilst eliminating analytic data silos Separation of duties & improved productivity Enterprise Feature Store – Trusted Data for AI
14. © 2024 Teradata. All rights reserved. Freedom to use preferred tools to train an LLM Scale trusted and cost-effective AI Capabilities to operationalize external models in a single environment Model Training API for model training Microsoft Azure ML Amazon SageMaker Export to ONNX API for model scoring Sample data Data Preparation Model Deployment Sample data Prepare data set from text data Model deployed in-database Access scores through API 14
15. © 2024 Teradata. All rights reserved.  Simplify model lifecycle management by enabling reusability and explainability – a must have to trusted AI  Enable robust governance that ensures transparency, traceability, and compliance throughout the model lifecycle  Automatically monitor model performance and data drift with zero configuration alerts ModelOps provides end-to-end management and explainability of AI models
16. © 2024 Teradata. All rights reserved. Highly optimized in-database analytic functions Geospatial/Temporal Hypothesis testing Machine learning Multivariate statistics Descriptive statistics Data preprocessing/ transformation Feature engineering Pathing analytics Digital signal processing Time series forecasting Leverage languages and tools of choice Operationalized at scale to drive transformative results Deploy models at scale Easily operationalize all models, even externally developed ones Integrated ModelOps Comprehensive machine learning model lifecycle management and governance *Not all logos are represented Access REST, SQL, SAS, PYTHON, R, Java Orchestration Teradata QueryGrid, NOS Replication TPT, DSA, SFTP, native APIs ClearScape Analytics™ designed to solve the problems of data, scale, and deployment with trusted AI Bring Your Own Analytics Bring Your Own Model Partner integrations* Seamlessly integrate to the AI/ML ecosystem Languages Use preferred language Open Analytics Framework in VantageCloud Lake only Model Deployment Model Monitoring Model Governance Feature Store
17. © 2024 Teradata. All rights reserved. Pre-trained model fine- tuned using on-demand GPU instance Inference and Integration done in Teradata Vantage using BYOM and ONNX format. This analysis required integration of data from various sources: • Telemetry data coming from a race car • GPS data coming from 3rd party provider • Video stream coming from cameras on the car 17 AI for Feature Extraction and Integration Joining Telemetry and Video data Data Loaded into S3 storage • The engine considered important contexts, like which items were out of stock or on special promotion, before making a recommendation. • The retailer worked with Teradata to create a recommendation engine built into digital shopping carts. • Teradata VantageCloud customers can power AI innovation with our complete cloud analytics and data platform for AI.
18. © 2024 Teradata. All rights reserved. © 2024 Teradata. All rights reserved. AI For Feature Extraction https://metro.co.uk/2021/10/18/surrey-writing-on-womans-jumper-landed-couple-with-fine-when-she-walked-in-bus-lane-15439916/
19. © 2024 Teradata. All rights reserved. 19 Managing AI Risk The widespread deployment of AI can have significant and far-reaching implications. o Transparency and Explainability o Guidelines and Standards o Impact Assessment o AI Design patterns and standards o Contextual Policy Layers o Continuous Monitoring o Legal and Regulatory
20. © 2024 Teradata. All rights reserved. Voice of the Customer using GenAI with zero movement of sensitive data Detecting the signal in customer communications using task- specific language models The Bank wanted to understand the Voice of the customer in more detail using free text communication from multiple channels. Task specific Language Models were deployed directly in-database from the Hugging Face repository to perform text summarization, embedding generation and semantic search. The results are collated by customer and used in an augmented intelligence application. OPEN SOURCE LLM Can use an LLM of choice using repositories such as hugging face or CSP marketplaces ABLE TO RUN ON- PREMISE Available in on-premise, hybrid or cloud only architectures. REDUCTION IN TIME Allows the bank to efficiently focus on the important areas of customer communications across multiple channels
21. © 2024 Teradata. All rights reserved. © 2024 Teradata. All rights reserved. Rise of the Machines
22. © 2024 Teradata. All rights reserved. ClearScape Analytics Experience Demo Overview 22 → Get started Classification: Use of LLM to predict if any incoming communication is a complaint or not Sentiment Analysis: Predict sentiment of complaint as negative, neutral, positive. Also see trend of sentiment over time by product categories Clustering: Use LLM embedding, to cluster similar complaints together Summarization: Create shorter summarized version of complaints. Visuals shows original text length vs summarized text length Topic Modelling: Identify complaint topics within complaint text Speech Analysis: Analyse customer voice data to understand sentiment and complaint topic Customer 360°: Augment customer 360° with Generative AI outputs Try it yourself: visit clearscape.Teradata.com and try out the examples for yourself free of charge DEMO