Building AI units using Rust - Devfest Yangon

Learn to build an end-to-end, high-performance artificial intelligence algorithm in Rust language capable of solving classification problems.

Code - https://github.com/vigneshwerd/DevfestYangon/

Transcript

1. Building Artificial Intelligence Units in Rust
2. About Me. 2 I am Vigneshwer Author of Rust CookBook moz://a tech speakers
4. Objective 4 Demystifying Artificial Intelligence - Fundamental units for AI algos - Learn the math behind AI Rust is great for building AI - Properties of Rust - Overview of different AI crates Learn to build real world AI applications - Code snippets for Mathematical models - Implementing web services
5. Workshop Overview Hacker’s guide to Rust Programming 1. Ownership & Borrowing Concept 2. Rust tools, library & community support ecosystem 3. Packaging and shipping Rust applications 5 Implementing ML Apps in Rust 1. Building Web services in Rust 2. Training your first ML model in Rust The Math behind Machine Learning 1. Gradient descent optimization technique 2. Vectors & Matrix Operations 3. Neural Networks 4. Hyperparameter Optimization
6. Hacker's guide to Rust Programming
7. What is Rust? System programming language that has great control like C/C++, delivers productivity like in Python, and is super safe
8. What is Rust? “Systems programming without fear
10. where are we with Rust?
12. 2016 -17 Rust is the Most Loved Language by Developers
13. Friends of Rust Organizations running Rust in production. (https://www.rust-lang.org/en-US/friends.html)
14. Why should one consider Rust? ➔ State of art programming language ➔ Solves a lot of common system programming bugs ➔ Cargo : Rust Package manager ➔ Improving your toolkit ➔ Self learning ➔ It's FUN ...
15. Basic Terminologies ➔ Low and high level language ➔ System programming ➔ Stack and heap ➔ Concurrency and parallelism ➔ Compile time and run time ➔ Type system ➔ Garbage collector ➔ Mutability ➔ Scope
16. Segmentation Fault ➔ Dereference a null pointer ➔ Try to write to a portion of memory that was marked as read-only
17. Buffer OverFlow ➔ Writing and reading the past end of buffer
18. Sample Error Outputs ➔ Segmentation fault ➔ Buffer OverFlow
20. Hack without Fear ➔ Strong type system ◆ Reduces a lot of common bugs ➔ Borrowing and Ownership ◆ Memory safety ◆ Freedom from data races ➔ Abstraction without overhead ➔ Stability without stagnation ➔ Libraries & tools ecosystem
21. Installing Rust Ubuntu / MacOS • Open your terminal (Ctrl + Alt +T) • curl -sSf https://static.rust-lang.org/rustup.sh | sh
22. Installing Rust rustc --version cargo --version Windows : • Go to https://win.rustup.rs/ ◦ This will download rustup-init.exe • Double click and start the installation
23. Features of rustup tool -> Update to latest version: rustup update stable -> Update the rustup tool to the latest version rustup self update -> Install the nightly toolkit version of the Rust compiler: rustup install nightly -> Change the default version of the Rust compiler to nightly version: rustup default nightly
24. Type System
25. Hello World // Execution starts here fn main() { let greet = “world”; println!("Hello {}!”, greet); }
26. Variable Bindings let x = 5; let (x, y) = (1, 2); // patterns let x: i32 = 5; // Type annotations let x = 5; // By default, bindings are immutable. x = 10; let mut x = 5; // mut x: i32 x = 10;
27. Function in Rust fn main() { print_sum(5, 6); } fn print_sum(x: i32, y: i32) { println!("sum is: {}", x + y); }
28. Identify the error fn main() { print_sum(5, 6); } fn print_sum(x , y ) { println!("sum is: {}", x + y); }
29. Returning a value fn add_one(x: i32) -> i32 { x + 1 } fn add_one(x: i32) -> i32 { x + 1; }
30. Expressions vs Statements x = y = 5 let x = (let y = 5); // Expected identifier, found keyword `let`. Rust : Expression -based language
31. Primitive Types
32. bool let bool_val: bool = true; println!("Bool value is {}", bool_val); let bool_val: bool = false;
33. char let x_char: char = 'a'; // Printing the character println!("x char is {}", x_char);
34. i8/i16/i32/i64/isize let num =10; println!("Num is {}", num); let age: i32 =40; println!("Age is {}", age); println!("Max i32 {}",i32::MAX); println!("Max i32 {}",i32::MIN);
35. Arrays let name: [type; size] = [elem1, elem2, elem3, elem4]; let array: [i32; 5] = [0, 1, 2, 3, 4]; let rand_array = [1,2,3]; // Defining an array println!("random array {:?}",rand_array ); println!("random array 1st element {}",rand_array[0] ); // indexing starts with 0 println!("random array length {}",rand_array.len() );
36. Tuples // Declaring a tuple let rand_tuple = ("DevFest Siberia", 2017); let rand_tuple2 : (&str, i8) = ("Viki",4); // tuple operations println!(" Name : {}", rand_tuple2.0); println!(" Lucky no : {}", rand_tuple2.1);
37. slice let array: [i32; 5] = [0, 1, 2, 3, 4]; println!("random array {:?}",&rand_array[0..3] ); // last three elements
38. String let rand_string = "Devfest Yangon 2017"; // declaring a random string println!("length of the string is {}",rand_string.len() ); // printing the length of the string let (first,second) = rand_string.split_at(7); // Splits in string let count = rand_string.chars().count(); // Count using iterator count println!(rand_string)
39. Ownership In Rust, every value has an “owning scope” and passing or returning a value means transferring ownership (“moving” it) to a new scope
40. Example 1 fn foo{ let v = vec![1,2,3]; let x = v; println!(“{:?}”,v); // ERROR : use of moved value: “v” }
41. Example 2 fn print(v : Vec<u32>) { println!(“{:?}”, v); } fn make_vec() { let v = vec![1,2,3]; print(v); print(v); // ERROR : use of moved value: “v” }
42. Example 3
43. Aliasing Aliasing -> More than one pointer to the same memory The key problem to most memory problems out there is when mutation and aliasing both happens at the same time. Ownership concepts avoids Aliasing
44. Borrowing If you have access to a value in Rust, you can lend out that access to the functions you call
45. Types of Borrowing There is two type of borrowing in Rust, both the cases aliasing and mutation do not happen simultaneously • Shared Borrowing (&T) • Mutable Borrow (&mut T)
46. &mut T fn add_one(v: &mut Vec<u32> ) { v.push(1) } fn foo() { let mut v = Vec![1,2,3]; add_one(&mut v); }
47. Lifetimes let outer; { let v = 1; outer = &v; // ERROR: ‘v’ doesn’t live long } println!(“{}”, outer);
48. Mutability Rules
49. Mutability Rules All variables are immutable by default Only one mutable reference at a time But as many immutable &’s as you want Mutable references block all other access The &mut must go out of scope before using other &’s
50. A bit complex example fn avg(list: &[f64]) -> f64 { let mut total = 0; for el in list{ total += *el; } total/list.len() as f64 }
51. HLL version fn avg(list: &[f64]) -> f64 { list.iter().sum::<f64>() / list.len() as f64 }
52. Parallel Version (Rayon) fn avg(list: &[f64]) -> f64 { list.par_iter().sum::<f64>() / list.len() as f64 }
53. Demos ➔ Vectors, Pointers, closures, typecasting ➔ Complex Data Structures : Structs, enum, impl , trait ➔ Decision making and looping statements ➔ Crates and Modules ➔ Cargo features ➔ Introduction to Rust library ecosystem ➔ Error handling ➔ Understanding Macros ➔ FFI Examples ➔ Building REST API’s
54. Vectors & Std datatypes ➔ Sample arithmetic operations (arithmetic.rs) ➔ Arrays (array.rs) ➔ Assignment operations (assignment.rs) ➔ Boolean types (boolean.rs) ➔ Calculator example (calculator.rs) ➔ Decimal (decimal.rs) ➔ Mutability example (mutuable.rs) ➔ String operations (string.rs) ➔ Tuples (tuple.rs) ➔ Vector (vector.rs)
55. User defined structs, Pointers, closures, typecasting ➔ Closure (closures.rs) ➔ Conditions (condition.rs) ➔ Constants (constant.rs) ➔ Enum types (enum.rs) ➔ block example (expressions.rs) ➔ Looping (looping.rs) ➔ Pointer example (pointers.rs) ➔ struct operations (string.rs) ➔ Methods (implement.rs) ➔ Functionality (trait.rs) ➔ Typecasting (typecasting.rs)
56. Cargo Features ➔ Creating a project ➔ Building & Running a project ➔ Installing dependencies ➔ Deployment of projects ➔ Publishing a package to crates.io ➔ Testing and benchmarking ➔ CI using travis CI
57. Crates and Modules ➔ Defining a module in Rust (sample_mod.rs) ➔ Building a nested module (sample_nested.rs) ➔ Creating a module with struct (sample_struct.rs) ➔ Controlling modules (sample_control.rs) ➔ Accessing modules (sample_access.rs) ➔ Creating a file hierarchy (sample_split.rs) ➔ Building libraries in Rust (sample_lib.rs) ➔ Calling external crates (sample_exec.rs)
58. Deep Dive into Parallelism ➔ Creating a thread in Rust (sample_move.rs) ➔ Spawning multiple threads (sample_multiple_thread.rs) ➔ Holding threads in a vector (sample_thread_expt.rs) ➔ Sharing data between threads using channels (sample_channel.rs) ➔ Implementing safe mutable access (sample_lock.rs) ➔ Creating child processes (sample_child_process.rs) ➔ Waiting for a child process (sample_wait.rs) ➔ Making sequential code parallel (sample_rayon.rs)
59. Efficient Error Handling ➔ Implementing panic (sample_panic.rs) ➔ Implementing Option (sample_option.rs) ➔ Creating map combinator (sample_map.rs) ➔ Creating and_then combinator (sample_and_then.rs) ➔ Creating map for the Result type (sample_map_results.rs) ➔ Implementing aliases (sample_alias.rs) ➔ Handling multiple errors (sample_multi_err.rs) ➔ Implementing early returns (sample_erl_ret.rs) ➔ Implementing the try! Macro (sample_try.rs)
60. Hacking Macros ➔ Building macros in Rust (sample_macro.rs) ➔ Implementing matching in macros (sample_match.rs) ➔ Implementing designators (sample_designator.rs) ➔ Overloading macros (sample_overloading.rs)
61. Integrating Rust with other languages ➔ Calling C operations from Rust ➔ Calling Rust commands from C ➔ Calling Rust operations from Node.js apps ➔ Calling Rust operations from Python ➔ Writing a Python module in Rust
62. Web Development with Rust ➔ Setting up a web server (nickel-demo) ➔ Creating endpoints (nickel-routing) ➔ Handling JSONRequests (nickel-jsonhandling) ➔ Building a custom error handler (nickel-errorhandling) ➔ Hosting templates (nickel-template)
63. RESTful API web service ➔ Setting up the API (sample_app) ➔ Saving user data in MongoDB (sample_post) ➔ Fetching user data (sample_get) ➔ Deleting user data (sample_rest_api)
64. Get /users
65. Post /users/new
66. DELETE /users/object_id
67. Changes to Rust
68. Rust Trailheads Follow us on Twitter: @rustlang @ThisWeekInRust Join Rust IRC Channels: #rust #rust-community #rust-machine-learning #tensorflow-rust Join Rust Websites: rust-lang.org rustbyexample.com rustaceans.org reddit.com/r/rust/ 68
69. The Math behind Machine Intelligence
70. Predictive Maintenance
71. Complex Games
72. Computer Vision
73. Voice Search
75. 2.5 x 10^8 Bytes Data created daily
76. 90 % of the World’s Data is created 2 years ago! 80% of it is unstructured
77. Fundamental units of AI system Data/ Sensor Mathematical Model + Training Actuator Basic AI System
78. Maths Behind AI Without mathematics, there's nothing you can do. Everything around you is mathematics. Everything around you is numbers. - Shakuntala Devi
79. Classification in General
80. Challenges
81. Neural Networks ➔ Attempt to make a computer model of the brain
82. Learning Process
83. Activation Functions Activation functions add nonlinearity to our network’s function
84. Training Neural networks Loss is a function of the model’s parameters - Aim is to minimize loss and find the best parameters
85. Optimization ➔ Random search ➔ Random Local Search ➔ Following the Gradient
86. SVM vs Softmax
87. Evaluation metrics Accuracy: Overall, how often is the classifier correct? (TP+TN)/total = (100+50)/165 = 0.91 True Positive Rate (Recall) : When it's actually yes, how often does it predict yes? TP/actual yes = 100/105 = 0.95 Precision: When it predicts yes, how often is it correct? TP/predicted yes = 100/110 = 0.91
88. Inspection
89. Summarize ➔ Assign random weights ➔ Calculated the activation rates between the different layers ➔ Find the error rate at output node ➔ Re-calibrate the linkage weights by back propagating ➔ Continue process until convergence criteria
90. Implementing ML Apps in Rust
91. Preparing the Training Dataset
92. Preparing the Labels
93. Building a SVM model in Rustlearn
94. Building a ANN using juggernaut extern crate juggernaut; use juggernaut::nl::NeuralLayer; use juggernaut::nn::NeuralNetwork; use juggernaut::activation::Sigmoid; use juggernaut::sample::Sample; use juggernaut::matrix::MatrixTrait;
95. Creating a sample dataset let dataset = vec![ Sample::new(vec![0f64, 0f64, 1f64], vec![0f64]), Sample::new(vec![0f64, 1f64, 1f64], vec![0f64]), Sample::new(vec![1f64, 0f64, 1f64], vec![1f64]), Sample::new(vec![1f64, 1f64, 1f64], vec![1f64]), ];
96. Creating the ANN architecture let mut test = NeuralNetwork::new(); let sig_activation = Sigmoid::new(); // 1st layer = 2 neurons - 3 inputs test.add_layer(NeuralLayer::new(2, 3, sig_activation)); test.add_layer(NeuralLayer::new(1, 2, sig_activation));
97. Train & Test test.on_error(|err| { println!("error({})", err.to_string()); }); test.train(dataset, 1000, 0.1f64); let think = test.evaluate(&Sample::predict(vec![1f64, 0f64, 1f64])); println!("Evaluate [1, 0, 1] = {:?}", think.get(0, 0));
98. What’s next? - Leaf (https://github.com/autumnai/leaf) - Tensorflow(https://github.com/tensorflow/rust) - Rustml (https://github.com/daniel-e/rustml)
99. What’s next? Collenchyma: Portable, HPC-Framework on any hardware with CUDA, OpenCL, Rust https://github.com/autumnai/collenchyma
100. Thanks! 100 Twitter @dvigneshwer Instagram @dvigneshwer Email dvigneshwer@gmail.com Website dvigneshwer.github.io/