Vector Database Ingestion with PyFlink on Decodable @ Flink Forward Berlin 2024

Abstract:

Join us for a live demo featuring a real-time vector database ingestion use case. We demonstrate how to run Apache Flink jobs written in Python using Decodable's latest managed PyFlink offering. The PyFlink job reads records from an operational database (MySQL), transforms them into vector embeddings on the fly, and streams them into MongoDB used as a vector database to support RAG architectures with fresh data.

Repository: https://github.com/decodableco/examples/tree/main/pyflink-vector-embeddings

Transcript

1. Vector Database Ingestion with PyFlink on Decodable Hans-Peter Grahsl @hpgrahsl Staff Developer Advocate
4. Real-time ETL Powered by Apache Flink® and Debezium
5. Resource Management Job Control Connector Library Platform and Job Observability State Management CLI UI dbt Unified API Flink and Debezium Runtime Cloud Infrastructure Java SQL Python Job Tuning and Optimization Security Engineering Data Catalog Decodable Provided Customer Provided Our focus today! Real-time ETL Powered by Apache Flink® and Debezium
6. DEMO TIME
7. Powerful Stream Processing In Minutes Not Months. https://decodable.co