IBM

Transcript

1. Data Fabric ↓ The foundation for a data-driven enterprise ↓ From Data Fabric to Data Observability
2. The rise of the data-driven enterprise 2 Data Fabric / © 2023 IBM Corporation
3. To be competitive, you need to use data as your differentiator! Leverage your data to create differentiated experiences for customers and employees 3 Sales Companies that are using data-driven B2B sales-growth engines report above-market growth and EBITDA increases in the range of 15 to 25 percent1 Marketing Discover fast changing customer behavior for 17% increase in purchase consideration.2 Operations By 2025, nearly all employees naturally and regularly leverage data to support their work. Rather than defaulting to solving problems by developing lengthy roadmaps.3 Finance Set reliable forecasts in volatile conditions for 90%+ improvement in time to decision.4 HR Attract and retain top talent by engaging 600+ applicants a month.5 IT Innovate and modernize apps and infrastructure for 25% more time to focus on strategic projects.6 Data Fabric / © 2023 IBM Corporation
4. 21% 8% However, evolving to become data-driven is challenging 4 use data-driven decisions to realize business value “all the time” have data and advanced analytics fully embedded into company processes and modernized in the cloud Source: Data-Driven Business Transformation, Corinium 2022 Data Fabric / © 2023 IBM Corporation
5. As a result, enterprises are missing out on data-driven opportunities 52% 5 Data Fabric / © 2023 IBM Corporation Revenue loss or cost over-runs 43% Inconsistency in decisions 43% Slow time to market 41% Poor customer experience
6. Because adoption of hybrid and multi-cloud environments requires a new approach 6 – Attempt to centralize everything fizzles – Cloud and cloud data warehouses add complexity – Higher compliance, security, governance risks – Increased focus on regulatory requirements – Multiple environments still result in data silos – Challenges related to scale of: • Discovering everything you have • Processing all your data in disparate locations • Data engineering Data Fabric / © 2023 IBM Corporation
7. That new approach is Data Fabric 7 Data Fabric (n): A data architecture with an integrated set of technologies and services designed to democratize data access across the enterprise at scale. Data Fabric / © 2023 IBM Corporation
8. 1. Connect 2. Profile 3. Classify and enrich 5. Publish Establish connectivity to physical data sources. Assess the quality of data assets. Classify data assets, assign data policies and rules, and enrich with semantics. Publish data products and enable subscription to data products based on data contracts. 4. Prepare / transform Engineer data assets into trusted data products. Lifecycle and governance Implement DataOps principles throughout the lifecycle and enforce governance end-to-end. Data products Data assets Data Fabric / © 2023 IBM Corporation Data Fabric enables a “trusted factory” approach for innovation
9. AI governance 9 IBM’s approach for Data Fabric and AI lifecycles Data governance Data integration Data science and MLOps Data Fabric / © 2023 IBM Corporation
10. Deliver readily consumable and reliable data to your teams anytime and anywhere 03 Data integration Data Fabric / © 2023 IBM Corporation
11. What is data observability? 02 Data observability Data Fabric / © 2023 IBM Corporation “Data observability” is the blanket term for understanding the health and the state of data in your system. Essentially, data observability covers an umbrella of activities and technologies that, when combined, allow you to identify, troubleshoot, and resolve data issues in near real time. By encompassing a basket of activities, observability is much more useful for engineers. Unlike the data quality frameworks and tools that came out along with the concept of the data warehouse, it doesn’t stop at describing the problem. It provides enough context to enable the engineer to resolve the problem and start conversations to prevent that type of error from occurring again. The way to achieve this is to pull best practices from DevOps and apply them to Data Operations. All of that to say, data observability is the natural evolution of the data quality movement, and it’s making DataOps as a practice possible. And to best define what data observability means, you need to know where DataOps stands today and where it’s going. For more information, please goto https://databand.ai/data-observability/
12. Data Fabric / © 2023 IBM Corporation
13. Data monitoring vs data observability 02 Data observability Data Fabric / © 2023 IBM Corporation
14. Why is data observability becoming a hot topic? 02 Data observability Data Fabric / © 2023 IBM Corporation Multi-cloud organizations The field is changing rapidly… Digitization leads to more data integrations Data is becoming client-facing by default … leading to increased complexity Reduced efficiency Large dependency on experts Higher business risk with data issues Monitoring & act real-time Data in motion & 24/7 delivery
15. 02 Data observability Data Fabric / © 2023 IBM Corporation Big FMCG client “ ” One of our largest challenges is maintaining and tracking the data quality that is streaming through our platforms as it is every increasing. We want to make sure our data scientists can focus on the work they love instead of monitoring and spending a lot of time in debugging
16. What can data observability deliver? 02 Data observability Data Fabric / © 2023 IBM Corporation DETECT EARLIER RESOLVE FASTER DELIVER TRUST Pinpoint unknown data incidents and reduce mean time to detection (MTTD) from days to minutes Improve mean time to resolution (MTTR) with incidents alerts and routing from weeks to hours Enhance reliability and data delivery SLAs, provide visibility into pipeline quality issues.
17. Building a central data observability center to achieve optimal data solutions 02 Data observability Data Fabric / © 2023 IBM Corporation IBM Databand Data Observability Incident management Anomaly detection Custom alerting Alert centralization Data pipeline monitoring Data quality monitoring Impact analysis and lineage Root cause analysis Ingestion, orchestration, transformation Data warehouses Data lakes / lake houses Dashboard & reporting Data Teams Real time alerts and SLAs dashboards for rapid resolution. Notifications Data engineering Data platform Analytics Automated metadata collection and error logging.
18. The Databand solution 02 Data observability Data Fabric / © 2023 IBM Corporation 1. Collect Automatically collect metadata. From all key solutions in the modern data stack. 4. Resolve Automatically collect metadata. Create smart communication workflows to resolve data quality issues and meet SLAs. 2. Profile Build historical baseline. Based on common data pipeline behavior. 3. Alert Alert on anomalies and rules. Based on deviations or breaches. Data Fabric / © 2023 IBM Corporation
19. How does a standard implementation flow look like? 02 Data observability Data Fabric / © 2023 IBM Corporation Design Translate Test Scale 1. Map current architecture 2. Determine adjustments 3. Identify major pain points 4. Decide on area for PoC 5. Translate into Tech & Functional doc
20. Data Fabric / © 2023 IBM Corporation Take care of your data observability and leverage the gained time to expand your business!
21. Any questions? For more information you can find us at the following stands at the data expo Implementation partner Standnr. 5 Standnr. 136