Node.js, GoLang and Concurrency

Node.js and GoLang are modern languages that are being used to build server-side systems. These languages were designed specifically to make it easier to write good, concurrent code, while ensuring that the final product is efficient when deployed at a large scale. Node.js uses a single-threaded, event-driven model. GoLang uses a construct similar to OS threads, but allowing the developer to work at a higher level of abstraction. This talk will explore the difference between these two approaches and what works best when developing scalable backend systems.

Speaker Bio:

Siddharth Kannan is a final year Mechanical engineering student at IIT Kharagpur. He has used Node.js at previous internships, including one at the Bangalore-based start-up Elanic, mainly working with frameworks to build REST APIs. He recently built Year in Twitter, a web-app that gives the user a report of their usage of Twitter in 2017, in GoLang. In the past, he has worked with Ruby, Python and PHP. He is active on GitHub, reads a lot of books, as is evident from his Goodreads profile and writes blog posts

This talk was presented on 21st January, 2018 at the Kharagpur Open Source Summit. The summit's keynote speaker was Harsh Gupta. The one-day event saw about 12 talks from people with vastly different backgrounds.

Talk page: http://archive.is/2018.01.16-063824/http://kwoc.kossiitkgp.in/summit/106

Transcript

1. Node.js, GoLang and Concurrency Node.js, GoLang and Concurrency 21st January, 2018 Siddharth Kannan
2. @icyflame
3. WHAT & WHY Act 1
4. Final outcome is independent of the order of execution Act 1, Scene 1
5. No correct order to execute the events in Act 1, Scene 1
6. Sending unrelated emails Act 1, Scene 2
7. Applying edits to different documents in a database Act 1, Scene 2
8. API requests Act 1, Scene 2
9. Horizontal scaling is cheaper than vertical scaling Act 1, Scene 2
10. Act 1, Recap • No correct order • Several web server tasks can be run concurrently • Concurrent code scales well
11. Act 2 How to write concurrent code?
12. Act 2, Scene 1 The Node.js Approach
13. Act 2, Scene 1 Node.js is a single threaded, non-blocking, asynchronous, concurrent language
14. Act 2, Scene 1 Node.js is a single threaded, non-blocking, asynchronous, concurrent language
15. Act 2, Scene 1 single threaded == one thing at a time CONCURRENT HOW?!
16. Act 2, Scene 1 JavaScript is weird Deal with it
17. Act 2, Scene 1 • Listen to Events • When events happen, push the handler into a queue • Subsequently, when the stack is empty, run the functions in the queue!
18. Act 2, Scene 1 require(“async”)
19. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
20. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
21. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
22. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
23. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
24. Act 2, Scene 1 $ node test.js Input Order: [ 0, 1, 2, 3, 6, 23 ] Output Order: [ 0, 1, 2, 3, 6, 23 ] $ node test.js Input Order: [ 0, 1, 2, 3, 6, 23 ] Output Order: [ 0, 1, 2, 3, 6, 23 ] $ node test.js Input Order: [ 0, 1, 2, 3, 6, 23 ] Output Order: [ 0, 1, 2, 3, 6, 23 ]
25. Act 2, Scene 1 var async = require("async"); var tweets = [ 0, 1, 2, 3, 6, 23 ]; var output = [ ]; var iterator = function (tweet, callback) { output.push(tweet); return callback(); }; async.each(tweets, iterator, function (err) { console.log(output) });
26. Act 2, Scene 1, Recap • Single-threaded • event loop mechanism • Events • Handler functions • Callback queue • Subsequently executed
27. Act 2, Scene 2 The GoLang approach
28. Act 2, Scene 2 Communicating Sequential Processes Model
29. Act 2, Scene 2 GoRoutines
30. Act 2, Scene 2 GoRoutines are not threads
31. Act 2, Scene 2 Channels
32. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
33. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
34. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
35. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
36. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
37. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
38. Act 2, Scene 2 func AsyncTask(argument int, channel chan int) { ... channel<-result } argument := ... chan1 := make(chan int) fmt.Printf("Execution started!") go AsyncTask(argument, chan1) <-chan1 fmt.Printf("Execution completed!")
39. Act 2, Scene 2 Retrieving from an empty channel will block the program
40. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
41. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
42. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
43. Act 2, Scene 2 func GetTweet(a int, tw_chan chan int) { tw_chan<-(a) }
44. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
45. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
46. Act 2, Scene 2 tweets := []int{ 0, 1, 2, 3, 6, 23 } tw_chan := make(chan int) for _, value := range tweets { go GetTweet(value, tw_chan) } output := []int{} for i := 0; i < len(tweets); i++ { req := <-tw_chan output = append(output[:], req) }
47. Act 2, Scene 2 $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 0 3 6 2 23] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [23 0 1 2 3 6] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 6 2 3 0 23] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 6 0 2 3 23]
48. Act 2, Scene 2 $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 0 3 6 2 23] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [23 0 1 2 3 6] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 6 2 3 0 23] $ go run test.go Input Order: [0 1 2 3 6 23] Output Order: [1 6 0 2 3 23]
49. Act 2, Recap • Node.js: single-threaded, event loop mechanism • Events • Handler functions • Callback queue • Subsequently executed
50. Act 2, Recap • GoLang: CSP model • GoRoutines • Channels • Easier to write • Easier to reason about
51. Act 3 What now?
52. Act 3 Threads are expensive
53. Act 3 GoLang I will handle threads for you Node.js No Threading Python, Java You handle everything Number of Threads
54. Act 3 Apparently Synchronous Code
55. Act 3 The async keyword
56. Act 3 Promises, Promises
57. Act 3 Verdict?
58. Act 3 The Times They Are A-Changin’
59. Act 3 I think Node is not the best system to build a massive web server. I would use Go for that. And honestly, that’s the reason why I left Node. It was the realization that: oh, actually, this is not the best server-side system ever. “
60. Act 3 I think Node is not the best system to build a massive web server. I would use Go for that. And honestly, that’s the reason why I left Node. It was the realization that: oh, actually, this is not the best server-side system ever. “
61. Act 3 I think Node is not the best system to build a massive web server. I would use Go for that. And honestly, that’s the reason why I left Node. It was the realization that: oh, actually, this is not the best server-side system ever. “ -- Ryan Dahl Created Node.js in 2009 Podcast link
62. Fin @icyflame