Golang For Scalable Systems

Transcript

1. Go for Scalable Systems Abu Ashraf Masnun
2. The myth about scalability Technology X doesn’t scale • I often hear “PHP / Python / Node doesn’t scale” - they do. • Scaling is more dependent on optimization in data structure and algorithms, good design and architecture. • Example: The case of image upload and horizontal scaling - Disk vs S3 • Go does some things well but Go is not the silver bullet or magic wand.
3. Scaling infrastructure When we talk about Scaling, we mostly mean scaling our servers • Horizontal ◦ Maintains a pool of servers ◦ Add more servers to scale up ◦ Multiple servers share distributed load • Vertical ◦ Add more power to the server to scale up ◦ Upgrade CPU / RAM / DISK etc ◦ One powerful server does it all
4. Scaling across Team • Does your team already know language X? • Will it scale nicely across your team? • Will they be able to learn it if it’s new to them? If your team is well versed in an existing technology, you should really really think through before introducing a new technology.
5. Let’s Go! How I introduce Golang to my friends: • Created at Google by some serious talents. Just look up their credentials. • It’s lead by a team at Google but accepts open source contributions. • Google needed a language that could take benefits of multi core cpu but easy to use • The language is small, quite easy to comprehend - (mostly) beginners friendly
6. Let’s Go! How I introduce Golang to my friends: • It has a very exciting and efficient approach to managing concurrency. • Compiled, quite fast, produces a single binary file, statically typed • Community is pretty decent, adequate resources, lots of open source packages • Lots of success stories of large orgs using it for handling tons of requests Go was intended for system programming, but here we are, building web apis and microservices
7. Big Names (except Google) • Facebook • Github • Uber • Twitch • Dailymotion • Soundcloud • Dropbox You can probably pick any large scale web company and you will find they use Go in some ways
8. Go at my work • Click tracking • Web hooks • Stats aggregation platform • Email verification system • Ad Serving platform (WIP) • Different internal APIs
9. Concurrency, you say? • What is concurrency? • Is it parallelism? • Taking advantage of multi core cpus • Concurrency, Parallelism and my friend’s restaurant
10. CPU Bound vs IO Bound • CPU Bound tasks keep using the CPU continuously, for example calculating a prime number. Parallelism is crucial for running operations simultaneously. • IO Bound operations execute some stuff and then wait for IO to finish, for example, reading a file or making a network requests. Concurrency without parallelism would be better suited.
11. Concurrency in Go • Goroutines and channels • Scheduling goroutines • Work stealing scheduler Do not communicate by sharing memory, share memory by communicating
12. Go vs Language X Python • Threads - GIL • Multi processing - good for CPU bound tasks, but what about network requests? • Asyncio - complex but good, but needs time for adaption Node • Event loop model works well but single threaded - there are ways to use multiple cores but I didn’t like them. • JavaScript, do I need more reasons? • I don’t like the npm eco system - too many modules
13. Does go scale better? Infrastructure: • Fast, low memory footprint, cpu efficient - handles a lot of operations • Superior concurrency - can handle a lot of concurrent requests • Single binary - easy to deploy Team: • Simple language, easy to learn • Statically typed nature makes code more comprehensible • There’s only one standard for code formatting • Fixed conventions make it easier to document code • Boring language - no fancy mumbo jumbo, no big brain oneliners - one way to do things
14. When should I need Go? Totally depends on your use cases. Ask yourself • Do we need the cost saving vs efforts? • Is it going to make the code cleaner / better in anyways? • Is the performance going to be significantly better? • Is the language / framework the bottleneck? (Or the database?)
15. Learning Resources Online: • Go Tour • Go By Example Book: • Go Programming Language • Go In Action Community: • Gophers Slack • Go Developers Network FB Group
16. Learning Solo
17. How I build APIs • Chi for routing • Logrus for logging • Custom handler for sending errors to Sentry • Redis for cache and message queue • Official MongoDB driver • sqlx + sql driver for mysql
18. Questions Further queries can be directed to: • @masnun • masnun@gmail.com