Apache Ignite

This presentation gives an overview of the Apache Ignite project. It explains Ignite in relation to its architecture, scaleability, caching, datagrid and machine learning abilities.

Links for further information and connecting

http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/

https://nz.linkedin.com/pub/mike-frampton/20/630/385

https://open-source-systems.blogspot.com/

Transcript

1. What Is Apache Ignite ? • An in memory distributed database • A cluster based processing platform • Open source / Apache 2.0 license • A key / value store • Written in Java, C#, C++, SQL • Multiple API's available • Multiple deployment options • Developed by GridGain Systems
2. What Is Apache Ignite ? • Persistence is turned off – A distributed in-memory database – In memory data grid – Use SQL or key-value APIs • Persistence is turned on – A distributed, horizontally scalable database – Guarantees full data consistency – Is resilient to full cluster failures
3. How Does Ignite Work ? • Persistence can be toggled • Small data sets can be stored in memory • Large datasets can use disk • Memory can be used as a caching mechanism • RDBMS or NoSQL integration • Partitioning or replication of data across cluster • Horizontally scaleable
4. How Does Ignite Work ? • Use Ignite as a distributed cache • Implements JCache specification (JSR 107) • Supports ACID transactions • Supports SQL ( with limitations ) • Supports Java, C++, and .NET • Has a feature rich key-value API • Offers two user interface ( UI ) options • Supports collocated processing
5. How Does Ignite Work ? • Ignite has client and server nodes in it's cluster • Server nodes are for storage and computation • Client nodes are connection points to the database • They may be embedded in client applications • Application code written in Java, C# or C++ • Ignite supports ODBC, JDBC and REST
6. Ignite DataGrid
7. Ignite DataGrid • An in-memory distributed key-value store • Horizontal scaleable • Add nodes on demand • Scale to hundreds of nodes • View as a distributed partitioned hash map • Every cluster node owns a portion of the data • As cluster (server) nodes are added more data is cached
8. Ignite Console
9. GridGain Console
10. Ignite Machine Learning
11. Ignite Machine Learning • Multiple development API's • A set of simple / efficient ML functions • Massively scaleable • Co-location of data and processing • Minimised the need for costly data transfers • Minimised ETL as ML functions act on distributed data • Algorithms support – Classification, Regression, Clustering – Recommendation, Preprocessing
12. Ignite Deployment Options • Multiple deployment options – Docker – Amazon AWS – Google Cloud – Mesos – YARN – VMWare – Microsoft Azure – Kubernetes
13. Available Books • See “Big Data Made Easy” – Apress Jan 2015 • See “Mastering Apache Spark” – Packt Oct 2015 • See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” • Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ • Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020
14. Connect • Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 • See my open source blog at – open-source-systems.blogspot.com/ • I am always interested in – New technology – Opportunities – Technology based issues – Big data integration