Share
Embed
How to Update Your Data Management Program in the Age of AI
Karen Lopez
May 23, 2025
Transcript
47 slides
-
How to Update Your Data Governance and Management Strategies for
the AI Era Karen Lopez InfoAdvisors 1 -
Karen Lopez Microsoft MVP, Data Platform Microsoft Certified Trainer, vExpert
Data management expert, space enthusiast, and #TeamData evangelist www.datamodel.com @datachick.bksy.social -
ANY TIME Ask questions Raise issues Leave kudos Don’t hold
them until the end. -
Nature of Data Management Data management principles haven’t changed much
in the last 30 years Advance in technology and business innovation have increased the need for stronger data management requirements AI is no different, but does bring new emphasis on certain aspect. -
Two-phased AI Requirements • Preparing for AI uses • Protecting
against AI abuses -
Key Takeaways AI as a Copilot, Not an Autopilot AI-Powered
Query Optimization Managing Data Complexity Mastering Multi- Platform Complexity Governance at Scale The Importance of Synthetic Data Generation Data Architect Productivity Managing Risk Importance of Laziness -
Biggest Mistake in Data Complexity? Not knowing what data we
have, where it is, how it is protected, when it is updated, is it trusted, how old it is, what does is mean, is it named correctly, who governs it, why it’s being collected… -
Data Inventory and Discovery Finding and identifying relevant data within
an organization Identifying data that is usable for a specific purpose Helping organizations find new insights, make better decisions, and meet compliance targets -
Scaling Data Governance Roles and responsibilities Usable standards Robust tools
& automation Collaboration incentives Classifications and grouping Monitoring & alerting -
How AI Will Help Us With Complexity We aren’t all
SQL Developers It’s not just about SQL syntax A good data pro is a lazy data pro We need help understanding data in other practices -
Karen’s Directives on AI & ML Never autonomous Paired “programming”
Responsible AI must be foremost Understand bias Understand security and privacy -
Where AI Can be Used in Data Governance • Data
Quality • Data Classification • Data Preparation • Data Security • Data Observation and Monitoring • Data Support and Literacy • Auditing and Compliance • Anomaly Detection • Data Profiling • Data Growth and Capacity Planning • Generative use cases – test data • ..and more -
Data Capacity Planning • Predictive Analytics, but more. • Cost
savings • Outage protection • Auto scaling -
Test Data Generation Synthetic Data NO MORE PRODUCTION TEST DATA
More realistic test data Faster, cheaper, better -
Give it a try Ask a bot to generate data
for your use case. Refine, validate, refine -
Data Literacy Understanding data sources and types Data exploration Meta
data collaboration Meta data “translations” Data visualization and communication Interpreting data analysis -
Give it a try Ask a bot to generate a
simple table, then to generate a not so simple query about it. -
Data Preparation • Identifying data sources • Validating data relevance
• Data cleansing • Data transformation • Data integration • Data loading -
Data Quality • Data Types • Error Detection • Data
Deduplication • Data Validation • Data Standardization • Anomaly Dection -
Data Quality • Data Cleansing • Data Matching • Monitoring
• Automation • Alerting -
Analyze this data for data quality and data privacy issues
1.Missing values – Certain fields (e.g., in the Year column and possibly in Date or Duration fields) have missing or invalid entries (such as a non-date value or 'nat’). 2.Inconsistent formatting – The Duration column is in text format while a derived numerical column (Duration_minutes) exists. It is important to ensure that time-related values are consistently converted and validated. 3.Data integrity – There could be duplicate entries or inconsistencies between related fields (for instance, discrepancies between the Duration and Duration_minutes columns). 4.Data parsing – Special care is needed in handling non-standard or erroneous entries (for example, a 'nat' entry in place of a valid date). -
Data Classification Data profiling vs. data meta data inspection Natural
Language Processing Pattern recognition Anti-patterns in profiling Data Exceptions 80/20 for now -
Give it a try Ask a bot to generate a
simple table with sensitive data as columns, then ask it to classify the data. -
Where Else Can AI be Used in Data Governance &
Data Management? -
Data Management AI Readiness Invest in Data Governance Integrate Data
Governance with AI Governance Improve Data Quality processes Strengthen Data Integration Governance Integrate AI with Data Analytics -
Coffee Talk | Expert View: How To Update Your Data
Management Strategy for the AI Era Data Management AI Readiness Refocus on metadata programs Strengthen security measures Do more automation Build a data-driven culture Build ethical AI skills, knowledge, and methods -
More Data- Related AI Threats Cybersecurity Bias and Discrimination Jobs?
Compliance breaches Flirting and bullying Ethical Abuses IP issues -
Data-related AI Protection • Defensive AI • Employee training •
Data Governance • More auditing • AI-driven data protection • Data risk assessments -
Questions to Answer Before AI What are our real goals
for AI? Do we have the right data infrastructure? Do we understand the legal and ethical uses of our data? How will we protect the data, models, and systems? How will we monitor and assess the outcomes? How does AI integrate with our existing systems and tools? Do we have resources to do this responsibly? -
10 Tips 1. Do personal learning and education on AI
topics 2. Do personal learning and education AI ethics 3. Build up your Data Governance programs 4. Bring meta data methods back to the forefront 5. Communicate, often, the importance of monitoring and auditing -
10 Tips 6. Leverage AI, smartly and ethically to build
these programs 7. Enhance all data management segments with AI 8. Ensure data professionals are part of the strategic planning 9. Evangelize the importance of cross-group collaboration 10.Build Data Literacy programs -
Frameworks for Ethical and Responsible AI 1.Asilomar AI Principles: Developed
during the 2017 Asilomar conference, these principles focus on research issues, ethics and values, and longer-term issues 2.IEEE’s Ethically Aligned Design: A set of guidelines that prioritize human rights and well-being in the development of autonomous and intelligent systems 3.EU’s Ethics Guidelines for Trustworthy AI: Created by the High-Level Expert Group on AI, these guidelines emphasize lawful, ethical, and robust AI 4.Montreal Declaration for Responsible AI: A framework that outlines principles for responsible AI development, including well-being, respect for autonomy, and democratic participation 5.AI4People’s Ethical Framework for a Good AI Society: Offers recommendations and outlines principles for the ethical implementation of AI in society -
Frameworks for Ethical and Responsible AI 6. Google’s AI Principles:
Google’s own set of ethical principles for AI development, which includes being socially beneficial, avoiding creating or reinforcing unfair bias, and being accountable 7.Microsoft’s AI Principles: Microsoft’s framework focuses on fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability 8.The Toronto Declaration: A declaration focusing on protecting the right to equality and non-discrimination in machine learning system 9.The Beijing AI Principles: Developed by the Beijing Academy of Artificial Intelligence, these principles focus on harmony and friendliness, fairness and justice, and inclusivity and sharing 10.The Public Voice’s Universal Guidelines for AI: A set of principles that include the right to transparency, the right to human determination, and the right to redress2.