Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to code a SMTP service in Go

How to code a SMTP service in Go

Sometimes it is useful to receive data in the form of e-mails instead of having traditional HTTP(S) APIs.

A traditional case for it is handling customer replies to incident tickets, for which it is much easier for them to reply to an e-mail notification from their mailbox than it would be for them to access the ticketing system to reply.

Fortunately, Go makes it easy to write such SMTP servers, in a way which even resembles HTTP server. See how it is done.

Avatar for Frédéric G. MARAND

Frédéric G. MARAND

April 13, 2023
Tweet

More Decks by Frédéric G. MARAND

Other Decks in Programming

Transcript

  1. © 2023 OSInet How to automate inbound emails handling Use

    case for the idea: a business partner that does only exposes issues using emails instead of APIs. Other use case: handling ticket responses by email instead of forcing customers to go to the ticketing tool. Also, intellectual curiosity 👻 The problem to solve 3
  2. © 2023 OSInet 4 Solutions • Pull • POP3 •

    IMAP4 • Custom APIs (e.g. CDO, MAPI) • Push • (E)SMTP • Homegrown (this presentation) • AWS SES • Lambda • SNS topic • S3 • Autres : header, bounce, stop • Google AppEngine SMTP ➢ HTTP • inbound_services.mail in app.yaml
  3. © 2023 OSInet 5 Pros • Pull • Fully capable

    and secure SMTP server upstream • Push • SMTPD server code is incredibly simple • SES for push is also fully capable and secure • SES ➢ Lambda is simple and cheap at that scale • SES output is simpler than MIME • GAE Inbound mail is HTTP
  4. © 2023 OSInet 6 Cons • Pull • POP3/IMAP APIs

    are complex and brittle • CDO is limited to MS Exchange • MAPI is…. Better forgotten • Maintenance on mail servers actually requires work • Push • Needs dedicated subdomain with MX record. Even for SES • Homegrown code needs and FW/ WAF rules to allow inbound SMTP • Homegrown microservice will have to be strictly delimited to remain sage • SES API is proprietary • GAE does not parse like SES
  5. © 2023 OSInet • At demo/PoC scale, or if vendor

    lock-in is a real issue, a homegrown very limited server is better • Otherwise SES → Lambda is probably • Not much more complex • Very reliable • Observable • Still very cheap • GAE SMTP ➢ HTTP is a legacy service • Pull is more for personal, one-off or management actions (e.g. export, backup of existing service) 7 Choosing
  6. © 2023 OSInet SMTP push means : • Messages routed

    by an MX, so… • Need one or more MX records… • In a specific subdomain Inbound SMTP on platform (not SES/GAE) means: • Open flow to port 25, 587, 465, or 2525 • Configure firewall / WAF 9 Infra setup
  7. © 2023 OSInet 10 SMTP (very) simplified steps •HELO /

    EHLO <domain> •MAIL FROM:<mail> •RCPT TO:<mail> •DATA •Headers: From,To,Subject,others •Body •“.” •QUIT Note: no whitespace after “.” In MAIL FROM / RCPT TO.
  8. © 2023 OSInet 11 Bradfitz/go-smtpd •smtpd.Connection : actual TCP connection

    •smtpd.Server •Addr: listen •Hostname: announce •(Read|Write)Timeout: DoS defense •PlainAuth: for SSL connections •.OnNewConnection •.OnNewMail •smtpd.Envelope
  9. © 2023 OSInet 12 Smtp.Envelope contents • Unreliable network info

    • SmtpRemote • SmtpNetwork • Unreliable protocol addresses • MailFrom • RcptTo • Lines (data) • Headers • Body
  10. © 2023 OSInet 13 Hooking Envelope events Library emits events

    while parsing the inbound message • AddRecipient: check recipients existence/interest • BeginData: set up write destination • Write: write parsed body • Close: finish writes
  11. © 2023 OSInet 14 Handling headers All processing is provided

    by the Go stdlib: • Headers : 
 net/mail.Message.Header • Email address parsing : 
 net/mail.AddressParser • Reading body : net/mail.Message.Body 
 is an io.Reader
  12. © 2023 OSInet 15 In practice • Define an Envelope

    implementation • Suggestion: parse body on close • doesn't scale but enough for these needs • at that point, header and body are already parsed and ready • write them to a net/mail.Message • Add custom logic as needed
  13. © 2023 OSInet 23 How to run the demo •

    git clone https://code.osinet.fr/fgml/smtpd • cd smtp • In one terminal tab: • go run ./cmd • In another terminal tab: • nc localhost 25 < fixture/simple.eml