SymfonyCon 2022 Keynote: Webhooks

Transcript

1. Don't call us, we will call you Fabien Potencier
2. Don't call us, we will call you Fabien Potencier
3. Don't call us, we will call you Fabien Potencier
4. https://symfony.com/components | https://symfony.com/stats/downloads Let's make it 200? That's almost 15 billion dls ~240 dl/second Full-stack PHP compat Frontend 3rd party 
 providers Most of 
 the code is here!
5. Provide fl exible low-level abstractions 
 Build higher-level abstractions Still on-going I want more of these! 
 Productivity boosts
6. RateLimiter Lock Login Throttling Cache 3.3 3.1 5.2 5.2 security: firewalls: default: # default limits to 5 login attempts per minute, # the number can be configured via "max_attempts" login_throttling: ~ # define your own RateLimiter login_throttling: limiter: login Great high-level features Throttling Login Attempts
7. Twig Great high-level features Sending Emails Serializer 2.0 + Dotenv 3.3 + Messenger 4.1 + Mime 4.3 + HttpClient 4.3 + Mailer 4.3 =>
8. Con fi guration Event Dispatcher Messenger Twig Dotenv Web Pro fi ler Speci fi c PHPUnit constraints ... An Ecocsystem with a Shared Experience It's also about the processes: 
 BC promise 
 Release cycle Security management 
 Consistency across the board ...
9. Incremental Improvements from the Community Mailer Examples ✓ More providers ✓ DKIM ✓ Draft Emails ✓ mailer:test command ✓ Tags and Metadata Currently supported by 
 MailChimp, Mailgun, Mailpace*, Postmark, 
 Sendgrid, Sendinblue, Amazon SES* The power of the community Based on real-world needs
10. Building flexible high-level abstractions 
 on top of solid low-level ones What's next?
11. Clicks, Bounces, Spam, and more…
12. email_webhooks: resource: '@MailerBundle/Resources/config/routing/webhooks.xml' prefix: /_email Webhooks: Mount the “universal” controller
13. class WebhookFailureHandler implements MessageHandlerInterface { private $logger; public function __construct(LoggerInterface $logger) { $this->logger = $logger; } public function __invoke(FailureWebhook $failure) { $this->logger->error('APP: New email failure', ['type' => $failure->getType(), 'email' => $failure->getEmail()]); } } Webhooks: Do something…
14. routing: 'Symfony\Component\Mailer\Webhook\FailureWebhook': amqp Webhooks: … sync or async Webhooks were tied 
 to Mailer Very specialized 
 webhook events
15. Time to revisit?
16. Does Webhooks handling 
 deserve to become a 
 Symfony Component? Do I have a need myself? Do I feel some pain?
17. Our Symfony Corp 
 "vanity" Slack channel It leverages several components 
 like HTTP Client, Noti fi er And some custom code 
 for Stripe Webhooks Slack support here was key 
 to creating the Notifier component Second use case for Webhooks...
18. Does Webhooks handling deserve 
 to become a Symfony Component? Are there any best practices 
 that could be abstracted and enforced? Is it generic enough to build a 
 useful and reusable abstraction? Could Symfony leverage it 
 in some components? Would it be useful 
 for many of our users?
19. Does Webhooks handling deserve 
 to become a Symfony Component? ✓ Support Mailer and Noti fi er ✓ Consume and send ✓ Replay payloads in tests ✓ Prevent production replay ✓ Promote dataless noti fi cations ✓ Check payload signatures ✓ ... Abstract and reuse Implement 
 common "best practices" Fully featured
20. Webhook Will land in 6.3 
 as an experimental component
21. A Webhook 
 is a noti fi cation 
 from one system 
 to another system 
 of some state change A successful payment in Stripe Twilio sent an SMS successfully A SaaS provider 
 or an internal system Your application It allows to react to changes from external systems What are Webhooks?
22. A system makes 
 real-time HTTP requests 
 to other servers, 
 sending events as JSON payloads Very common, but not universal language agnostic What are Webhooks? Broadcast
23. Well understood universal concept with support from many providers But not standardized Payload is freeform What are Webhooks? Security is provider dependent
24. Symfony Webhooks 
 Support in Practice
25. A Single HTTP Entry Point 
 for Consuming Webhooks webhook: resource: ../vendor/symfony/webhook/Controller/ type: attribute prefix: /webhook Prefix of 
 your choice Similar to 
 HTTP fragments 
 Web Profiler ...
26. framework: webhook: routing: emails: service: mailer.webhook_request_parser.mailgun secret: '%env(MAILGUN_WEBHOOK_SECRET)%' Webhooks Routing Your provider Similar to 
 Messenger Routing https://somwhere.com/webhook/emails
27. #[AsRemoteEventConsumer(name: 'emails')] class MailerEventConsumer implements ConsumerInterface { /** * @param AbstractMailerEvent $event */ public function consume(Event $event): void { $email = $event->getRecipientEmail(); error_log(match ($event->getName()) { MailerDeliveryEvent::BOUNCE => sprintf('Email to %s bounced (%s)', $email, $event->getReason()), MailerEngagementEvent::UNSUBSCRIBE => sprintf('Unsubscribe from %s', $email), default => sprintf('Receive unhandled email event %s', $event->getName()), }); } } Mailer Webhook Consumer Logic independent 
 from the provider!
28. framework: messenger: routing: Symfony\Component\RemoteEvent\Messenger\ConsumeRemoteEventMessage: async 👌Webhook Best Practice 
 Fast HTTP Response For later...
29. protected function getRequestMatcher(): RequestMatcherInterface { return new ChainRequestMatcher([ new MethodRequestMatcher('POST'), new IsJsonRequestMatcher(), ]); } 1 Incoming HTTP Request Validation return new ChainRequestMatcher([ new MethodRequestMatcher('POST'), // https://postmarkapp.com/support/article/800-ips-for-firewalls#webhooks // localhost is added for testing new IpsRequestMatcher(['3.134.147.250', '50.31.156.6', '50.31.156.77', '18.217.206.57', '127.0.0.1']), new IsJsonRequestMatcher(), ]); Enforce provider security Reuses HttpFoundation infrastructure Used by Security
30. if ( !isset($content['signature']['timestamp']) || ... ) { throw new RejectWebhookException(406, 'Payload is malformed.'); } -- private function validateSignature(array $signature, string $secret): void { if (!hash_equals($signature['signature'], hash_hmac('sha256', $signature['timestamp'].$signature['token'], $secret)) throw new RejectWebhookException(406, 'Signature is wrong.'); } } 2 Signature & Payload Validation Enforce provider security
31. final class MailgunPayloadConverter implements PayloadConverterInterface { /** @return AbstractMailerEvent */ public function convert(array $payload): Event { if (in_array($payload['event'], ['failed', 'accepted', 'rejected', 'delivered'])) { // ... } else { $event = match ($payload['event']) { 'clicked' => MailerEngagementEvent::CLICK, // ... default => throw new RejectWebhookException(406, sprintf('Not supported event "%s".', $payload['event'])) }; $wh = new MailerEngagementEvent($event, $payload['id'], $payload); } if (!$date = \DateTimeImmutable::createFromFormat('U.u', $payload['timestamp'])) { throw new RejectWebhookException(406, sprintf('Invalid date "%s".', $date)); } $wh->setRecipientEmail($payload['recipient']); $wh->setTags($payload['tags']); // ... 3 Payload Parsing & Conversion Where the standardization happens!
32. namespace Symfony\Component\RemoteEvent\Event\Mailer; abstract class AbstractMailerEvent extends Event { private \DateTimeImmutable $date; private string $email = ''; private array $metadata = []; private array $tags = []; public function getDate(): \DateTimeImmutable { return $this->date; } public function getRecipientEmail(): string { return $this->email; } public function getMetadata(): array { return $this->metadata; } public function getTags(): array { return $this->tags; } } Mailer 
 Event Supports common 
 provider features
33. final class MailerDeliveryEvent extends AbstractMailerEvent { public const RECEIVED = 'received'; public const DROPPED = 'dropped'; public const DELIVERED = 'delivered'; public const DEFERRED = 'deferred'; public const BOUNCE = 'bounce'; private string $reason = ''; public function getReason(): string { return $this->reason; } } final class MailerEngagementEvent extends AbstractMailerEvent { public const OPEN = 'open'; public const CLICK = 'click'; public const SPAM = 'spam'; public const UNSUBSCRIBE = 'unsubscribe'; } Mailer 
 Event Fined grained 
 Provider agnostic
34. Webhooks for Noti fi er? Can we do the same 
 for SMS, Chat, ...?
35. namespace Symfony\Component\RemoteEvent\Event\Sms; final class SmsEvent extends Event { public const FAILED = 'failed'; public const DELIVERED = 'delivered'; private string $sms = ''; public function getRecipientSms(): string { return $this->sms; } } SMSEvent Provider agnostic
36. #[AsRemoteEventConsumer(name: 'sms')] class SmsEventConsumer implements ConsumerInterface { public function consume(Event $event): void { $phone = $event->getRecipientSms(); error_log(match ($event->getName()) { SmsEvent::DELIVERED => sprintf('SMS delivered to %s', $phone), SmsEvent::FAILED => sprintf('SMS failed for %s', $phone), default => sprintf('Receive unhandled SMS event %s', $event->getName()), }); } } $sms = new SmsMessage('+1411111111', 'A new login was detected!'); $sms->options((new TwilioOptions())->webhookUrl('https://somewhere.com/webhook/sms')); $texter->send($sms); SMS Webhook Consumer framework: webhook: routing: sms: service: Symfony\...\TwilioRequestParser Logic independent 
 from the provider! Routing /webhook/sms Plumbing specific 
 to provider
37. 🙋 I need your help ‣ Implement support for more providers ‣ Validate event names and common payload items ‣ Write tests ‣ Write documentation Next Steps? I will submit the Pull Request 
 after the 6.2 release
38. class Event { public function __construct( private string $name, private string $id, private array $payload, ){ } public function getName(): string { return $this->name; } public function getId(): string { return $this->id; } public function getPayload(): array { return $this->payload; } } name: user.new id: 1337 name: invoice.paid id: evt_1M2yqF2Gz5TJuMKBAlWnwTKe name: complained id: -Agny091SquKnsrW2NEKUA name: bounced id: 00000000-0000-0000-0000-000000000000 Your own internal 
 application Events Implement your own 
 Webhooks!
39. $subscriber = new Subscriber($userDefinedUrl, $userDefinedSecret); $event = new Event('user.new', '1', [ 'id' => 1, 'name' => 'Fabien Potencier', 'email' => 'fabien@symfony.com', ]); $this->bus->dispatch(new SendWebhookMessage($subscriber, $event)); Sends the Webhook via HTTP async What about providing Webhooks to other applications? Application defined Payload
40. > POST /webhook/users HTTP/1.1 Host: 127.0.0.1:8000 X-Event: user.new X-Id: 1 X-Signature: sha256=5331fa2b02cd43616c9d8c44cd... Content-Type: application/json Accept: */* Content-Length: 22 User-Agent: Symfony HttpClient/Curl Accept-Encoding: gzip {"id":1, "name":"Fabien Potencier", ...} POST HTTP request Payload signature JSON payload HTTP Transport Abstraction Default implementation, fully customizable
41. #[AsRemoteEventConsumer(name: 'users')] class SomeEventConsumer implements ConsumerInterface { public function consume(Event $event): void { error_log('Receive an internal and generic event '.var_export($event, true)); } } Some other 
 Symfony application(s) Consumers can be anything ... including Symfony itself for SOA/micro-services/multi-apps/...
42. class Event { public function __construct( private string $name, private string $id, private array $payload, ){ } public function getName(): string { return $this->name; } public function getId(): string { return $this->id; } public function getPayload(): array { return $this->payload; } } Alternative transports: Queues for internal systems APIs ... A Webhook is just 
 one possible transport for an event - via HTTP
43. Let's decouple 
 Remote Events and Webhooks vs local events from EventDispatcher Webhook events are Remote Events Like Mime & Mailer
44. RemoteEvent Will land in 6.3 
 as an experimental component
45. namespace Symfony\Component\RemoteEvent; class Event { public function __construct( private string $name, private string $id, private array $payload, ){ } public function getName(): string { return $this->name; } public function getId(): string { return $this->id; } public function getPayload(): array { return $this->payload; } } RemoteEvent 
 in Symfony
46. Webhooks vs API? RemoteEvent Component Shared infrastructure ‣ RemoteEvent class structure ‣ Payload converters ‣ Consumer code push Webhook: A system makes real-time HTTP requests 
 to other servers, the payload is an event API: A system API endpoint that can be polled on-demand; 
 It returns all events that have occurred since last request pull Only useful if the provider supports 
 Webhooks and an API for events
47. Event API Resource waste (polling) Not real-time Webhooks vs Event API: which one to use? Many providers provide both Good providers 
 mitigate those Webhooks Uncertain security Downtime might lose events No ordering guarantees Needs some tunnel to test locally Frequent HTTP rq 
 without new events Depends on provider retry strategy Expose a web server 
 Security depends on the provider Don't call us 
 We will call you Matters sometimes
48. Webhooks vs Event API? Use both? Subscribe to real-time noti fi cations via a webhook 
 Use the API to retrieve the event That's dataless notifications
49. Webhooks vs Event API? Depends on your use case... Vanity notifications 
 on Slack on paid 
 invoices from Stripe Stripe invoices 
 for accounting Event API Webhooks
50. 🙋 I need your help ‣ Payload replay for tests ‣ Logs / Audit trail ‣ Production Replay prevention ‣ Dataless noti fi cations ‣ Support for CRM, payment gateways, ...? ‣ ... Next Steps? The main concepts 
 and low-level infrastructure 
 are well-defined or wait a few years :)
51. https://symfony.com/sponsor Sponsor Symfony Thank you!