Let's build event-driven application in 15 minutes - introduction to Watermill

Building event-driven applications should be as easy as building HTTP API - this was our goal when we were starting to work on Watermill.

During the talk, I would like to tell you the story of how we created one of the most popular libraries for building event-driven applications in Go. I will not end with the theory - I will do a quick live coding to show you how to use Watermill. The application will be independent of any Pub/Sub implementation and will be able to work with RabbitMQ, Kafka or even MySQL without many changes.

Transcript

1. roblaszczak Introduction to Watermill let’s build event-driven application in 30 13 minutes Go Remote Fest 26.04.2020
2. roblaszczak About me • I’m Robert ;-) • Tech Lead @ • Blogging at https://threedots.tech/ • @roblaszczak at
3. roblaszczak Cascading failures
4. roblaszczak
5. roblaszczak
6. roblaszczak
7. roblaszczak PROBLEM?
8. roblaszczak Building applications based on Kafka is not easy • Consumer groups • Partitioning • At-least-once delivery • Message Ack and Nack support • Not losing any message
9. roblaszczak BUILDING EVENT-DRIVEN APPLICATIONS IS NOT EASY
10. roblaszczak How to make building event-driven applications easy in Go?
11. roblaszczak Watermill
12. roblaszczak 1.9k GitHub Stars
13. roblaszczak 20 Contributors
14. roblaszczak 8 Supported Pub/Subs
15. roblaszczak
16. roblaszczak Watermill is a library Watermill is not a framework
17. roblaszczak
18. roblaszczak HOW?
19. roblaszczak Unix philosophy (1978) • Write programs that do one thing and do it well. • Write programs to work together. • Write programs to handle text streams message, because that is a universal interface.
20. roblaszczak type Publisher interface { Publish(topic string, messages ...*Message) error Close() error } type Subscriber interface { Subscribe(ctx context.Context, topic string) (<-chan *Message, error) Close() error }
21. roblaszczak type HandlerFunc func(msg *Message) ([]*Message, error)
22. roblaszczak
23. roblaszczak
24. roblaszczak
25. roblaszczak
26. roblaszczak Testing
27. roblaszczak func TestPublishSubscribe(t *testing.T) { features := tests.Features{ ConsumerGroups: true, ExactlyOnceDelivery: false, GuaranteedOrder: false, Persistent: true, } tests.TestPubSub( t, features, createPubSub, createPubSubWithConsumerGrup, ) }
28. roblaszczak var stressTestTestsCount = 10 func TestPubSubStressTest( t *testing.T, features Features, pubSubConstructor PubSubConstructor, consumerGroupPubSubConstructor ConsumerGroupPubSubConstructor, ) { for i := 0; i < stressTestTestsCount; i++ { t.Run(fmt.Sprintf("%d", i), func(t *testing.T) { t.Parallel() TestPubSub(t, features, pubSubConstructor, consumerGroupConstructor) }) } }
29. roblaszczak
30. roblaszczak How fast is Watermill?
31. roblaszczak Pub/Sub Publish (messages / s)* Subscribe (messages / s)* Kafka 70,252 117,529 NATS 76,208 38,169 SQL (MySQL) 6,989 143 Google Cloud Pub/Sub 7,416 39,591 AMQP 2,408 10,608 *1 instance, 1 process, Docker Compose
32. roblaszczak Pub/Sub Publish (messages / s) Subscribe (messages / s) Kafka 70,252 117,529 NATS 76,208 38,169 SQL (MySQL) 6,989 143 Google Cloud Pub/Sub 7,416 39,591 AMQP 2,408 10,608
33. roblaszczak Pub/Sub Publish (messages / s) Subscribe (messages / s) Kafka 70,252 117,529 NATS 76,208 38,169 SQL (MySQL) 6,989 143 Google Cloud Pub/Sub 7,416 39,591 AMQP 2,408 10,608
34. roblaszczak Pub/Sub Publish (messages / s) Subscribe (messages / s) Kafka 70,252 117,529 NATS 76,208 38,169 SQL (MySQL) 6,989 143 Google Cloud Pub/Sub 7,416 39,591 AMQP 2,408 10,608
35. roblaszczak The first rule of live coding
36. roblaszczak Don’t do livecoding The first rule of live coding
37. roblaszczak Not covered • It’s hard to show everything in 13 minutes :) • Kafka and Google Cloud Pub/Sub internals • At-least-once delivery • CQRS component • Middlewares
38. roblaszczak
39. roblaszczak
40. roblaszczak Thanks for all contributors!
41. roblaszczak Small announcement: new articles on the blog • DDD, CQRS, Clean/Hexagonal Architecture/Event Storming • Focused on solving real issues instead of blindly applying patterns • Done by refactoring a real Go project!
42. roblaszczak roblaszczak https://threedots.tech/ https://watermill.io/ THANKS! NEW ARTICLES IN 2 WEEKS!
43. roblaszczak Photo by Micheile Henderson @micheile010 // Visual Stories [nl] on Unsplash Photo by Thomas Ehling on Unsplash