Nuances of Machine Learning with Ignite ML

Transcript

1. Nuances of Machine Learning with Ignite ML Alexey Zinoviev, Java/BigData Trainer, Apache Ignite Contributor
2. E-mail : zaleslaw.sin@gmail.com Twitter : @zaleslaw @BigDataRussia vk.com/big_data_russia Big Data Russia + Telegram @bigdatarussia vk.com/java_jvm Java & JVM langs + Telegram @javajvmlangs Follow me
3. What is Apache Ignite?
4. Release 2.7
5. SQL ACID (β) MVCC (β) Spring Data 2.0 New features in 2.5 - 2.7
6. SQL ACID (β) MVCC (β) Spring Data 2.0 New features in 2.5 - 2.7 Thin clients Java .NET C++ + PHP + Python + Node.js
7. SQL ACID (β) MVCC (β) Spring Data 2.0 New features in 2.5 - 2.7 Thin clients Java .NET C++ + PHP + Python + Node.js Topology Star topology with Zookeeper
8. Integration
9. Read from Ignite to DataFrame Write from DataFrame to Ignite IgniteSparkSession + IgniteCatalog SQL Optimization Spark integration spark .read .format(FORMAT_IGNITE) .option(OPTION_CONFIG_FILE, TEST_CONFIG_FILE) .option(OPTION_TABLE, "person") .load() tbl .write .format(FORMAT_IGNITE). option(OPTION_CONFIG_FILE, CFG_PATH). option(OPTION_TABLE, tableName). option(OPTION_CRT_TBL_PRIMARY_KEY_FIELDS, pk). save()
10. Spark integration
11. Kafka Streamer Flink Streamer JMS Streamer Storm Streamer Flume Streamer ... Streaming integrations KafkaStreamer<String, String, String> ks = new KafkaStreamer<>(); IgniteDataStreamer<String, String> stmr = ignite.dataStreamer("myCache")); ks.setIgnite(ignite); ks.setStreamer(stmr); ks.setTopic(someKafkaTopic); ks.setConsumerConfig(kafkaConsumerConfig); ks.setSingleTupleExtractor(strExtractor); ks.start(); ks.stop(); strm.close();
12. Kafka & Flink integrations
13. Hibernate L2 Cache
14. ML Frameworks
15. ML/DL Most Popular Frameworks
16. Could it be trained directly on Ignite data?
17. Could it be trained directly on Ignite data?
18. Ignite ML Overview
19. Classification Regression Clustering Neural Networks Multiclass and multilabel algorithms scikit-learn Preprocessing NLP Dimensionality reduction Pipelines Imputation of missing values Model selection and evaluation Model persistence Ensemble methods Tuning the hyper-parameters
20. ML module in 2.4 - 2.6 releases
21. ML module in 2.7 release
22. Abstraction layer on top of Ignite storage and computation Partitioned-Based Dataset
23. Abstraction layer on top of Ignite storage and computation MapReduce using Compute Grid Partitioned-Based Dataset
24. Abstraction layer on top of Ignite storage and computation MapReduce using Compute Grid Partition data (can be recovered from another node) Partitioned-Based Dataset
25. Abstraction layer on top of Ignite storage and computation MapReduce using Compute Grid Partition data (can be recovered from another node) Partition context (ML algorithms are iterative and require context) Partitioned-Based Dataset
26. Recovery after Node Failure P = Partition C = Partition Context D = Partition Data D* = Local ETL
27. ML Algorithms
28. Logistic Regression SVM KNN ANN Decision trees Random Forest Classification algorithms
29. KNN Regression Linear Regression Decision tree regression Random forest regression Gradient-boosted tree regression Regression algorithms
30. Multilayer Perceptron Neural Network
31. Preprocessors
32. Normalization
33. Scaling
34. One-Hot Encoding
35. How to get the best model?
36. Pipeline API Test-Train Split Parameter Grid Binary Evaluator Binary Classification Metrics Tuning Hyperparameters Model Evaluation with K-fold cross validation
37. Ensemble as a Mean value of predictions Majority-based Ensemble Ensemble as a weighted sum of predictions Machine Learning Ensemble Model Averaging
38. Superpower Features
39. Online mini-batch learning
40. TensorFlow on Apache Ignite Ignite Dataset IGFS Plugin Distributed Training
41. TensorFlow on Apache Ignite Ignite Dataset IGFS Plugin Distributed Training >>> import tensorflow as tf >>> from tensorflow.contrib.ignite import IgniteDataset >>> >>> dataset = IgniteDataset(cache_name="SQL_PUBLIC_KITTEN_CACHE") >>> iterator = dataset.make_one_shot_iterator() >>> next_obj = iterator.get_next() >>> >>> with tf.Session() as sess: >>> for _ in range(3): >>> print(sess.run(next_obj)) {'key': 1, 'val': {'NAME': b'WARM KITTY'}} {'key': 2, 'val': {'NAME': b'SOFT KITTY'}} {'key': 3, 'val': {'NAME': b'LITTLE BALL OF FUR'}}
42. ISSUES
43. Data Parallelism
44. Amdahl's law for Distributed Programming
45. Find or prepare something locally Repeat it a few times (locIterations) Reduce results Make next step (globalIterations++) Check convergence The iterative-convergent nature of ML programs
46. Assume, n the sample size and p the number of features The complexity of ML algorithms Algorithm Training complexity Prediction complexity Naive Bayes O(n*p) O(p) kNN O(1) O(n*p) ANN O(n*p) + KMeans Complexity O(p) Decision Tree O(n^2*p) O(p) Random Forest O(n^2*p*amount of trees) O(p*amount of trees) SVM O(n^2*p + n^3) O(amount of sup.vec * p) Multi - SVM O(O(SVM) * amount of classes) O(O(SVM) * amount of classes * O(sort(classes)))
47. Use <?> or avoid them? How to make enough good Java API?
48. Use <?> or avoid them? Use Stream as a return type? How to make enough good Java API?
49. How to make enough good Java API? Use <?> or avoid them? Use Stream as a return type? Be coherent with other parts of Ignite
50. How to make enough good Java API? Use <?> or avoid them? Use Stream as a return type? Be coherent with other parts of Ignite Naming conventions
51. How to make enough good Java API? Use <?> or avoid them? Use Stream as a return type? Be coherent with other parts of Ignite Naming conventions Lambda everywhere
52. How to make enough good Java API? Use <?> or avoid them? Use Stream as a return type? Be coherent with other parts of Ignite Naming conventions Lambda everywhere Method chaining vs setters
53. How to make enough good Java API? Use <?> or avoid them? Use Stream as a return type? Be coherent with other parts of Ignite Naming conventions Lambda everywhere Method chaining vs setters Chain method naming for hyperparameters tuning
54. How to contribute?
55. > 180 contributors totally 8 contributors to ML module VK Group Blog posts Ignite Documentation ML Documentation Apache Ignite Community
56. NLP (TF-IDF, Word2Vec) More integration with TF Clustering: LDA, Bisecting K-Means Naive Bayes and Statistical package Dimensionality reduction … a lot of tasks for beginners:) Roadmap for Ignite 3.0
57. DEMO
58. E-mail : zaleslaw.sin@gmail.com Twitter : @zaleslaw @BigDataRussia vk.com/big_data_russia Big Data Russia + Telegram @bigdatarussia vk.com/java_jvm Java & JVM langs + Telegram @javajvmlangs Follow me