Manticore Search - IT Press Tour #59 Dec. 2024

Transcript

1. Bridging Simplicity and Power Dec 3rd 2024 Sergey Nikolaev, CEO and co-founder
2. About me • CEO, Co-founder of Manticore. • Previously: ◦ QA ◦ Developer ◦ Team Lead ◦ CTO
3. Manticore Search at a Glance Manticore Search: High-Performance Open-Source Search Engine • Fast and Accurate: Built for speed and precision. • Highly Scalable: Handles everything from small datasets to distributed architectures. • Feature-Rich: ◦ Full-text search, vector search, highlighting, and geo search. ◦ Advanced features like replication, percolate queries, and distributed nodes. ◦ Flexible options: SQL/JSON, row-wise and columnar storages, JOIN functionality. ◦ Real-time data insertion and secondary indexes. ◦ Seamless integrations: sync with MySQL, Postgres, ODBC, XML, CSV. ◦ Backup and restore tools for reliability. ◦ And many more.
4. The Story Behind Manticore Search • 2001: Sphinx Search created. • Early 2000s: Sphinx Gained popularity.
5. Why We Created Manticore Search • Sphinx ceased development in 2016-2017. • Critical to our work: fast, reliable, and essential. • Determination to modernize and improve Sphinx for new challenges.
6. What were our goals back then • Keep the project alive and open source. • Support for new technologies. • Easier configuration and use. • Enhanced performance.
7. Co-founders Peter Zaitsev (former CEO of Percona) Mindaugas Žukas (COO of Altinity)
8. Team ILYA C++ Ex-Sphinx STAS C++ Ex-Sphinx SERGEY CEO KLIM PHP Developer NICK PHP Developer ALEXEY C++ Ex-Sphinx PAVEL QA GLORIA SMM Ex-Sphinx DMITRII PHP, Rust Developer
9. • Mission: ◦ Precise, fast, and scalable search. ◦ Focus on performance and cost-efficiency. ◦ Enables use of affordable hardware to reduce costs. ◦ Ensures powerful search at any data scale. • Vision: ◦ Leading solution for general search and log analytics. ◦ Versatile tool for standard and complex search needs. ◦ Drive innovation and adaptability in the search space. What Drives Us at Manticore Search
10. • Slow Queries • High Resource Consumption • Scalability Issues • Complex Queries, no SQL support. • Setup and Maintenance complications. The Pain Points We Solve
11. • Easy to use. • SQL and JSON support. • Multi-model storage: row-wise and columnar. • High performance with minimal resources. • Advanced search features. • Highly Scalable. • Seamless integration with popular tools and databases. • Fully open source. Why Manticore Search?
12. • Intuitive setup and configuration. • Works seamlessly with default settings. • Empowers developers and DevOps teams to focus on their goals. Ease of Use
13. • SQL: Universal, simple, and powerful. • JSON: Structured. • MySQL protocol support for easy client use. SQL & JSON support
14. • Row-wise: Ideal for small datasets in RAM. • Columnar: for large datasets beyond RAM capacity. • Flexible configuration modes: ◦ Declarative: Uses a configuration file. ◦ Imperative: Create, modify, or drop tables with SQL commands. Storages and Configuration Modes
15. • Written in C++ for speed and resource optimization. • Minimal RAM usage, maximized CPU efficiency. • Like squeezing every drop of juice from your CPU. Performance and Resource Efficiency
16. • Over 20 operators and ranking factors: ◦ AND, OR, Phrase, Quorum, Proximity, Words order, Exact Form … • A number of built-in rankers and a custom ranker. • Morphology: stemming, synonyms, Chinese segmentation. • Text highlighting. Full-Text Capabilities
17. • Full-text search. • Faceted search. • Grouping and aggregations. • Boolean search. • Fuzzy search. • Geo search. • Vector search. • Autocomplete, query suggestions. • Highlighting. • Percolate queries. Advanced Search Features
18. • From small sites to billions of log records. • Replication and load distribution made simple. Scalability
19. • Supports MySQL, Postgres, ODBC, XML, CSV, and more. • Works with tools like Logstash, Grafana, and Apache Superset. • Clients for popular programing languages Seamless Integrations
20. • Fully open source under OSI-approved licenses. • Accessible on GitHub for everyone. Open Source Commitment
21. Github Search Demo github.mnt.cr
22. Demo: Autocomplete
23. Demo: Corrected Suggestions
24. Demo: Fuzzy Search
25. Demo: Faceted Search
26. Demo: Morphology
27. Demo: Semantic Search
28. Image Search Demo image.mnt.cr
29. Demo: Text-to-Image Search
30. Demo: Reverse Image Search
31. • Classifieds: Craigslist • Marketplaces: Rozetka • Data Intelligence: Socialgist, Clarivate, Statista, Priceshape, Clausebase • Job Search: Learn4good • Travel: Europarcs, Hotelplan • Science: PubChem • Real Estate: Huispedia Real-World Applications
32. What our users say
33. “With Manticore we are able to deliver a powerful search solution to our users to find their real estate. Manticore provides many great features, high performance, and low (memory) resource consumption.” Ramon Noordeloos, Founder & CTO at Huispedia
34. “Manticore Search impresses us with its lightweight build and lightning-fast performance. The degree of customization and configurability it provides sets it apart and allows us to fine-tune our search system to our exact needs. Overall, our experience with Manticore has been extremely satisfying.” Henrik Steffen. CEO at cgrd GmbH in Hamburg
35. “Manticore delivers impressive performance when working with MariaDB, notably enhancing response times for many of our Geographic, and Full Text Search calls. With its outstanding functionality, I'd rate my overall experience a perfect 10 out of 10.” Dave Minogue, VP of Technology at New Spark Media
36. “Comparing Manticore with Elasticsearch, Manticore is significantly simpler to install and operate, making it far more user-friendly. It also appears to be more resource-efficient, even during idle periods, unlike Elasticsearch, which consumed considerable resources even when idle.” Constantin Tsukanov, CTO, Botmother
37. “The PostgreSQL and Pgvector combination became noticeably slow with this volume of data, taking several seconds to return results. It was at this point that ClauseBase decided to switch to Manticore Search.” Maarten Truyens, Founder & CEO at ClauseBase
39. How Manticore Search is different Search Engine Small Data Big Data Functionality Richness Ease of Use Manticore Search ✅ ✅ ✅ ✅ Elasticsearch ✅ ⚠ ✅ ❌ Meilisearch ✅ ❌ ❌ ✅ Typesense ✅ ❌ ❌ ✅ Quickwit ❌ ✅ ❌ ❌ Vespa ❌ ✅ ✅ ❌
40. Benchmarks based on db-benchmarks.com
41. Elasticsearch
42. Quickwit
43. Meilisearch and Typesense
44. MySQL and Postgres
45. Conclusions from the benchmarks
46. • Auto-Sharding. • Authentication. • Integration with Kibana. • Auto-embeddings for vector search. Roadmap
47. • Feature-rich and cost-efficient general search solution. • Recognized for: ◦ High performance: Faster than competitors. ◦ SQL support: Easier to use than JSON DSL. ◦ Feature-richness: Includes full-text, vector, geo, faceted search, and more. ◦ Cost-efficiency: Handles large datasets without requiring expensive infrastructure. Positioning: Today
48. • Cost-effective alternative to Elasticsearch in the ELK stack: ◦ Supports Logstash, Beats,vector.dev, elasticdump for data ingestion. ◦ Actively working on Kibana integration. • Investing in AI capabilities: ◦ Enhancing AI search features to meet growing demand. ◦ Strengthening Manticore’s position as a comprehensive general search solution. Positioning: Future
49. • Professional services around the open-source software: ◦ Support subscription. ◦ New features engineering. How Manticore Stays Sustainable Today
50. Monetization and SaaS Plans • From Services to Product: ◦ Committed to keeping the core open-source. ◦ Insights from helping hundreds of users. • SaaS Goals: ◦ UI as a Service: Simplified deployment, predictable fees. ◦ Fully Managed Cloud: We handle infrastructure; users focus on search.
51. • Self-sustaining and growing: ◦ Founder-funded and supported by services. ◦ SaaS monetization plans in progress. • Ready to accelerate: ◦ Faster product and SaaS development. ◦ Broader market reach. • Seeking the right partner: ◦ Aligned with open-source vision. ◦ Strategic expertise to fuel growth. Bootstrapping or VC?
52. Thank you! Contact: sergey@manticoresearch.com