Replacing Data Analysts with AI - Building Data Pipelines and Automating Analysis with Generative AI

MCPサーバーを活用して、複雑だった多段階パイプラインのデータ分析ワークフローを革新する方法をご紹介します。単純にAIを利用してデータ分析を試みる際に失敗する原因を把握し、その解決策としてsemantic layeringをご紹介し、実際の実装戦略をデモンストレーションします。データアナリストの役割が消失するのではなく、どのように進化しているか、どのような能力が必要になるかについて議論いたします。

Transcript

1. Replacing Data Analyst Jobs with AI LINE+, ML Product Strategy Minseo Park, Shinae Choi
2. Familiar Dashboard Insights The dashboards you create every day
3. Plot Twist: AI Created This • Created with a single simple prompt Created via In-house MCP Server Same insights, revolutionary approach • Took 5 minutes to create, saving days of metric creation
4. The Data Analysis Challenge Complex workflows we all recognize event_log subscription_log ua_daily user_segmentation model_cost kpi_user_count log_summary kpi_function kpi_session usage_flow Data source OASIS In-house Data Platform Visualization (Excel) Insight derivation Reporting Request
5. The MCP Server Promise From complex pipelines to natural language event_log subscription_log Data source In-House MCP Server In-house Data Platform Request LLM
6. Part 1. MCP Servers & The Context Problem Part 2. The Semantic Layer Solution Part 3. Solution in Action Part 4. What’s Next? Part 5. Closing Thoughts from Setup to Strategy The roadmap for integrating LLMs into your data work
7. Understanding the power and the context challenge MCP Servers & The Context Problem
8. Introducing MCP Servers Bridging LLMs and databases User Client Do this ! What is this ? LLM !
9. Introducing MCP Servers Bridging LLMs and databases Analyze this ! Create this metric ! User Client DB Database X LLM ?
10. Introducing MCP Servers Bridging LLMs and databases Analyze this ! Create this metric ! User MCP Client DB Database LLM ! MCP Server MCP Server
11. AI-Powered Analysis Workflow From natural language to database results User MCP Client MongoDB MCP server MongoDB Request task Analyze request Decide on tools/resource to use Call tool or request resources Execute tool/resource Send result Send result Format and interpret result Present formatted result
12. Basic Implementation Issues Expectations vs. actual outcomes User Expectation What is the churn rate for this month? Event log Subscription log Event log Subscription log Active users last month Inactive OR Canceled users Extract active user data Extract subscription data Extract active user data Active users last month Inactive users Actual outcome
13. The Root Problem Why basic MCP servers produce wrong results Because …
14. • … understand what the user wants • … cannot map natural language to queries • … understand the business context • … know the data structure • … recognize business terms • … The Root Problem Why basic MCP servers produce wrong results Because LLM doesn’t …
15. • … understand what the user wants • … cannot map natural language to queries • … understand the business context • … know the data structure • … recognize business terms • … The Root Problem: Missing Context Why basic MCP servers produce wrong results Because LLM doesn’t … → NO CONTEXT
16. • … understand what the user wants • … cannot map natural language to queries • … understand the business context • … know the data structure • … recognize business terms • … The Root Problem: Missing Context Why basic MCP servers produce wrong results Because LLM doesn’t … • Provide request interpretation • Provide query mapping • Provide business contexts • Provide data schema and metadata • Provide definitions for business terms • … Provide LLM with context data → HAS CONTEXT
17. Three approaches to add business intelligence to AI The Semantic Layer Solution
18. • An intermediary layer that translates natural language questions into precise business logic Introducing the Semantic Layer How to add business intelligence to AI analysis What is semantic layering? Natural language question Precise business logic • Provides business and data context to the model • Adds meaning & context to the raw tools and resources in MCP What role does it play? Semantic Layer CONTEXT →
19. • LINE Official Account • An AI chat bot that uses offers various AI- based services, such as web search, translation, OCR, and more • Uses freemium business model Case Study: AI Assistant Service A freemium LINE AI chatbot AI Assistant
20. Defining Business Context Terms, metrics, and data relationships Example context type 1 - Business terms
21. • Friend: User that has added AI Assistant as a friend • Free User: User that has agreed to the terms of use of AI Assistant • DAU: Number of unique users that have at least one event log that is not unsubscribe and block Defining Business Context Terms, metrics, and data relationships Example context type 1 - Business terms
22. • Friend: User that has added AI Assistant as a friend • Free User: User that has agreed to the terms of use of AI Assistant • DAU: Number of unique users that have at least one event log that is not unsubscribe and block Defining Business Context Terms, metrics, and data relationships Example context type 1 - Business terms Example context type 2 – Data schema
23. • Friend: User that has added AI Assistant as a friend • Free User: User that has agreed to the terms of use of AI Assistant • DAU: Number of unique users that have at least one event log that is not unsubscribe and block Defining Business Context Terms, metrics, and data relationships Example context type 1 - Business terms -- Users Table users ( user_id: VARCHAR, reg_date: DATE, status: CHAR(1), -- 'A'=Active, 'I'=Inactive msg_count: INTEGER ) Example context type 2 – Data schema -- Events Table events ( event_id: VARCHAR, user_id: VARCHAR, date: DATE, type: CHAR(1) -- 'T'=Text, 'M'=Media, 'V'=Voice )
24. Three Solution Approaches Client, server, and database-level semantic integration User MCP Client MongoDB MCP server MongoDB
25. Three Solution Approaches Client, server, and database-level semantic integration User MCP Client MongoDB MCP server MongoDB Context Semantic Layering + Semantic Layering + Semantic Layering +
26. Client level Semantic Layering System prompt enhancement approach Semantic Layering + User MCP Client MongoDB MCP server MongoDB
27. Client level Semantic Layering System prompt enhancement approach Semantic Layering + User MCP Client MongoDB MCP server MongoDB System Prompts
28. Basic vs. Enhanced Prompting You are a data assistant. Help users query the users and events tables. Bad system prompt (w/o Semantic Layering) The difference context makes
29. Basic vs. Enhanced Prompting You are a messenger platform analyst with access to our messaging database. DATA SCHEMA: - users table: user_id, reg_date, status ('A'=Active, 'I'=Inactive), msg_count - events table: event_id, user_id, date, type ('T'=Text, 'M'=Media, 'V'=Voice) BUSINESS TERMS: • Friend: User that has added AI Assistant as a friend • Free User: User that has agreed to the terms of use of AI Assistant • DAU: Number of unique users that have at least one event log that is not unsubscribe and block Always translate technical codes into business terms and provide strategic insights. Good system prompt (w/ Semantic Layering) The difference context make
30. Server-level Semantic Layering Configuration-driven approach Semantic Layering + User MCP Client MongoDB MCP server MongoDB
31. Server-level Semantic Layering Configuration-driven approach Semantic Layering + User MCP Client MongoDB MCP server MongoDB Configuration files
32. YAML Configuration Files business_terms: metrics: friend: description: "User that has added AI Assistant as a friend” definition: ”events.type=‘add_friend’ exists and events.type=‘agree’ doesn’t” business_value: "12% of user base generating 67% of platform engagement” ... Business terms→ business_terms.yaml Structured business logic definition
33. YAML Configuration Files data_schema: tables: users: description: "Platform user accounts and activity metrics” fields: user_id: type: "VARCHAR” description: "Unique user identifier” primary_key: true ... Data Schema → schema.yaml Structured business logic definition
34. Server Integration Example messenger_mcp_server/ ├── main.py ├── server.py ├── semantics/ │ ├── schema.yaml │ ├── business_terms.yaml │ └── semantic_orchestrator.py └── requirements.txt Add semantics and layers to server code Code implementation with semantic tools Server now uses semantics for query processing and response enhancement
35. DB-level Semantic Layering Automated metadata approach Semantic Layering User MCP Client MongoDB MCP server MongoDB +
36. DB-level Semantic Layering Automated metadata approach Semantic Layering User MCP Client MongoDB MCP server MongoDB Metadata collections +
37. Triggers and View Generation messenger_platform_db/ ├── users ├── events ├── schema_metadata ├── business_terms ├── business_intelligence │ ├── daily_metrics │ └── user_segments └── semantic_views ├── dau_view └── engagement_summary Add collections & triggers to DB Real-time business intelligence updates
38. Triggers and View Generation messenger_platform_db/ ├── users ├── events ├── schema_metadata ├── business_terms ├── business_intelligence │ ├── daily_metrics │ └── user_segments └── semantic_views ├── dau_view └── engagement_summary Add collections & triggers to DB Real-time business intelligence updates
39. Triggers and View Generation messenger_platform_db/ ├── users ├── events ├── schema_metadata ├── business_terms ├── business_intelligence │ ├── daily_metrics │ └── user_segments └── semantic_views ├── dau_view └── engagement_summary Add collections & triggers to DB Real-time business intelligence updates
40. Triggers and View Generation messenger_platform_db/ ├── users ├── events ├── schema_metadata ├── business_terms ├── business_intelligence │ ├── daily_metrics │ └── user_segments └── semantic_views ├── dau_view └── engagement_summary Add collections & triggers to DB Real-time business intelligence updates
41. Triggers and View Generation messenger_platform_db/ ├── users ├── events ├── schema_metadata ├── business_terms ├── business_intelligence │ ├── daily_metrics │ └── user_segments └── semantic_views ├── dau_view └── engagement_summary Add collections & triggers to DB Real-time business intelligence updates messenger_mcp_server/ ├── main.py ├── server.py └── requirements.txt Modify server code
42. Semantic Layering: Summary Key Differences Between Levels Level Client Server Database Semantic storage System prompt YAML config file Metadata collection Implementation Request enhancement MCP server code integration Schema-embedded semantics Automation Manual prompting Semi-automated Fully-automated Performance Real-time translation Cached processing Pre-computed view
43. Real improvements and practical query capabilities Solution in Action
44. Why Semantic Layering Matters A hands-on exploration with actual analysis questions 𝐴𝑝𝑟𝑖𝑙 𝑐ℎ𝑢𝑟𝑛 𝑟𝑎𝑡𝑒 = 𝐴𝑝𝑟𝑖𝑙 𝐼𝑛𝑎𝑐𝑡𝑖𝑣𝑒 𝑂𝑅 𝐶𝑎𝑛𝑐𝑒𝑙𝑒𝑑 𝑢𝑠𝑒𝑟𝑠 𝑀𝑎𝑟𝑐ℎ 𝐴𝑐𝑡𝑖𝑣𝑒 𝑢𝑠𝑒𝑟𝑠 What’s the churn rate for April 25?
45. Pipeline With Semantic Layering - Recap User MCP Client MongoDB MCP server MongoDB Semantic Layering + Semantic Layering + Semantic Layering +
46. User MCP Client MongoDB MCP server MongoDB Pipeline With Semantic Layering - Recap
47. Basic Implementation & Limitations Stage 1: Direct LLM query processing via MongoDB MCP server The Question: “What’s the churn rate for April 2025?” The Result • Calculated Result: 83.0% • Calculation Logic: (April 25 Inactive Users / March 25 Active Users) × 100 = (15,520 / 18,695) × 100 • Problem: Missing cancelled user data • LLM found the data but lacked business context understanding • Churn = Inactive + Cancelled → Only calculated inactive • Raw data analysis without domain knowledge leads to incomplete analysis Core Issue
48. Stage 2: 4-Step Business Logic Integration Server-level Semantic Processing “What is the churn rate for April?” ① Natural Language Interpretation {intent: “metric_query”, subject: “churn_rate”} Business term lookup ② Query Mapping Metric_id: “customer_churn_rate” Business context clarification ③ Metric Definition Calculation: ”prev_month_mau vs current_month_inactive_or_canceled” Domain-specific execution ④ Business Logic Execution Result: 83.6% of churn rate
49. Enhanced Implementation Results Stage 2: Business context integration outcomes The Result • Inactive Users: 15,520 • Subscription Cancellation: 112 (newly discovered) • Total Churned Customers: 15,632 • Improved Result: 83.6% (15,632 / 18,695) March Active Users April Inactive Users The Question: “What’s the churn rate for April 2025?”
50. Enhanced Implementation Results Stage 2: Business context integration outcomes • Inactive Users: 15,520 • Subscription Cancellation: 112 (newly discovered) • Total Churned Customers: 15,632 • Improved Result: 83.6% (15,632 / 18,695) April Canceled Users The Result The Question: “What’s the churn rate for April 2025?” March Active Users April Inactive Users
51. Enhanced Implementation Results Stage 2: Business context integration outcomes • Inactive Users: 15,520 • Subscription Cancellation: 112 (newly discovered) • Total Churned Users: 15,632 • Improved Result: 83.6% (15,632 / 18,695) The Result The Question: “What’s the churn rate for April 2025?” April Canceled Users March Active Users April Inactive Users
52. Enhanced Implementation Limitations Logical correctness achieved, but lacking execution efficiency Logic vs Execution • Designed: Complete churn calculation (inactive + cancelled) • Reality: LLM simplifies when facing complexity • Approach: Full collection scanning • Strategy: No selective filtering or indexing • Design: Missing query optimization patterns Root Cause Key Insights LLM automatically reduces logic complexity when execution becomes inefficient → Next: Database-level optimization to support original semantic design
53. Stage 3: Metadata optimization for semantic query execution DB-level Semantic Processing Metadata Strategy Example // Strategic index design {order_type: 1, dt: 1, emid: 1} // Churn calculation requirements: // - Find who cancelled (order_type filter) // - Within target month (dt filter) // - From previous active users (emid filter) • Before: • Full collection scan • Processing millions of records • After: • Targeted data access • Focus only on the cancelled users in specific month Execution Enhancement Benefits Metadata optimization enables original semantic design execution
54. • Inactive Users: 15,520 • Subscription Cancellation: 112 • Total Churned Customers: 15,632 • Improved Result: 83.6% (15,632 / 18,695) The Result Complete Implementation Results Stage 3: Metadata optimization for semantic query execution The Question: “What’s the churn rate for April 2025?” April Canceled Users March Active Users April Inactive Users
55. Complete Implementation Results Stage 3: Metadata optimization for semantic query execution The Question: “What’s the churn rate for April 2025?” • Inactive Users: 15,520 • Subscription Cancellation: 76 (corrected from 112) • Total Churned Users: 15,596 • Final Result: 83.4% (15,596 / 18,695) The Result April Canceled Users March Active Users April Inactive Users
56. Implementation Summary Layer-by-Layer Problem Resolution 18,695 83.0% 15,520 ? 18,695 83.6% 15,520 112 18,695 83.4% 15,520 76 DB-level Optimization Server-level Business Logic
57. Examples of Natural Language Queries in Action What You Could Ask Q1. Monthly Active Users What was the number of active users in June? “18,695 active users” Q2. Historical Churn Analysis What was the customer churn rate in February 2025? “83.4%” ③ Relative Time Queries How many users were active last month? “18,695 active users in May 2025” ④ Regional Distribution Can I see a breakdown of users by region for this month? “JP 45%, TW 30%, TH 25%”
58. Multi-system automation What’s Next?
59. MCP Integration Connecting multiple systems through a unified hub MCP Client Database MCP Server MongoDB MongoDB MCP Server
60. MCP Integration Connecting multiple systems through a unified hub MongoDB MCP Server MongoDB PostgreSQL MCP Client System MCP Server Tableau Slack
61. MCP Integration Connecting multiple systems through a unified hub MongoDB MCP Server MongoDB PostgreSQL MCP Client System MCP Server Tableau Slack PostgreSQL MCP Server Tableau MCP Server Slack MCP Server
62. Work Tools Working Together Single Platform, Multiple Systems MCP Client Hub Tableau MongoDB PostgreSQL Slack Github Confluence …
63. Working Together: Use Cases (1/3) Business Analysis MongoDB Individual user service usage frequency + revisit patterns Result: “60 % of users with 3+ monthly usage willing to convert to subscription” Request: “Would converting our service to a subscription model improve profitability?” PostgreSQL Current users’ purchase cycles + Avg. purchase amounts Confluence User interview results + subscription intention surveys
64. Working Together: Use Cases (2/3) Real-time Monitoring Tableau Real-time user count Result: “1,200+ users, performance metrics, 2 alerts detected” Request: “Check our service real-time status.” MongoDB Last hour error logs + user churn patterns Slack Incident-related conversations in channels
65. Working Together: Use Cases (3/3) Daily Workflow Github Today’s deadline Pull Requests + pending code reviews Result: Prioritized to-do list for each individual Request: “Organize today’s team tasks.” Confluence Documents needing updates + action items + today’s meeting agendas
66. Adapting to the age of AI-driven analytics Closing Thoughts
67. The Future Analyst New Partnership: Human + AI What LLMs Excel at • Large-scale data processing and pattern recognition • Simultaneous multi-system query execution • Repetitive analysis task automation • 24/7 real-time monitoring and alerting • Standardized report generation What Only Humans Can Do • Designing logic systems for LLMs to understand business context • Building semantic layers and context systems • Strategic question design and framing • Structuring domain knowledge for AI utilization • Analysis result integrity verification
68. Multi-Platform analysis Conduct comprehensive analysis across MongoDB user behavior, PostgreSQL transaction data, and Tableau dashboards simultaneously - beyond individual analytical capacity The Future Analyst New Partnership: Human + AI Context-Aware Workflow Automation Automate analytical workflows that adapt to data patterns: anomaly detection triggers deeper investigation, seasonal trends adjust forecasting models, and outliers prompt validation processes Advanced Analytics Implementation Deploy models for customer segmentation, demand forecasting, and A/B testing — without requiring deep technical implementation, whether statistical, rule-based, or machine learning.
69. Multi-Platform analysis Conduct comprehensive analysis across MongoDB user behavior, PostgreSQL transaction data, and Tableau dashboards simultaneously - beyond individual analytical capacity The Future Analyst New Partnership: Human + AI Context-Aware Workflow Automation Automate analytical workflows that adapt to data patterns: anomaly detection triggers deeper investigation, seasonal trends adjust forecasting models, and outliers prompt validation processes Advanced Analytics Implementation Deploy models for customer segmentation, demand forecasting, and A/B testing — without requiring deep technical implementation, whether statistical, rule-based, or machine learning. “LLMs are powerful partners that expand our analytical capabilities”
70. If you have any questions, feel free to find us in the QA section Thank You