An Over-Engineering Disaster with Macaroons

Macaroons are a decentralized authorization credential designed for distributed systems. They allow services to grant authorization tokens for one another, with precisely limited scope and ability. Macaroons may very well be the future of authorization across services.

Several months ago, my team eagerly adopted them, using an implementation in Go. Replacing our old system was easy enough—but working with macaroons turned out to be a disaster. In this talk, I share how our decision to use macaroons negatively affected our user experience, our own developer experience, and even our system availability. We also lost hours of developer time arguing about the difference between a “macaroon” and a “macaron”.

So, what makes macaroons so dang cool? How do they work? Why should gophers get excited about them and why did my team get excited about them? Why are macaroons such a tempting fit for a project, especially a Go project, and where did my team go wrong? What should a team of gophers consider and where did my team ultimately land?

This is a story about cool new technology, over-engineering, and coconut cookies.

1. An Over-Engineering Disaster with Macaroons Tess Rinearson, Chain A U G U S T 2 0 1 8
2. Macaroons: maybe not? Tess Rinearson, Chain A U G U S T 2 0 1 8
3. Authentication and authorization 3 who are you? what can you do?
4. // Authenticate returns the request with added tokens in the context func (a *API) Authenticate(req *http.Request) (*http.Request, error) { ctx := req.Context() user, pw, ok := req.BasicAuth() if !ok { return "", errors.New("no token") } err = a.checkTokenAuthn(ctx, user, pw) if err != nil { return "", errors.New("unauthenticated") } if user != "" { // if request was successfully authn’d, pass the user along ctx = newContextWithUser(ctx, user) } return req.WithContext(ctx), nil }
5. // Authenticate returns the request with added tokens in the context func (a *API) Authenticate(req *http.Request) (*http.Request, error) { ctx := req.Context() user, pw, ok := req.BasicAuth() if !ok { return "", errors.New("no token") } err = a.checkTokenAuthn(ctx, user, pw) if err != nil { return "", errors.New("unauthenticated") } if user != "" { // if request was successfully authn’d, pass the user along ctx = newContextWithUser(ctx, user) } return req.WithContext(ctx), nil }
6. // Authenticate returns the request with added tokens in the context func (a *API) Authenticate(req *http.Request) (*http.Request, error) { ctx := req.Context() user, pw, ok := req.BasicAuth() if !ok { return "", errors.New("no token") } err = a.checkTokenAuthn(ctx, user, pw) if err != nil { return "", errors.New("unauthenticated") } if user != "" { // if request was successfully authn’d, pass the user along ctx = newContextWithUser(ctx, user) } return req.WithContext(ctx), nil }
7. // Authenticate returns the request with added tokens in the context func (a *API) Authenticate(req *http.Request) (*http.Request, error) { ctx := req.Context() user, pw, ok := req.BasicAuth() if !ok { return "", errors.New("no token") } err = a.checkTokenAuthn(ctx, user, pw) if err != nil { return "", errors.New("unauthenticated") } if user != "" { // if request was successfully authn’d, pass the user along ctx = newContextWithUser(ctx, user) } return req.WithContext(ctx), nil }
8. 5 func (a *Authorizer) Authorize(req *http.Request) error { policies := a.policyByRoute[req.RequestURI] if policies == nil { return errors.New("missing policy on this route") } grants, err := a.grantsByPolicies(policies) if err != nil { return errors.Wrap(err) } for _, g := range grants { if accessTokenData(g) == authn.Token(ctx) { return nil } } return ErrNotAuthorized }
9. 5 func (a *Authorizer) Authorize(req *http.Request) error { policies := a.policyByRoute[req.RequestURI] if policies == nil { return errors.New("missing policy on this route") } grants, err := a.grantsByPolicies(policies) if err != nil { return errors.Wrap(err) } for _, g := range grants { if accessTokenData(g) == authn.Token(ctx) { return nil } } return ErrNotAuthorized }
10. 5 func (a *Authorizer) Authorize(req *http.Request) error { policies := a.policyByRoute[req.RequestURI] if policies == nil { return errors.New("missing policy on this route") } grants, err := a.grantsByPolicies(policies) if err != nil { return errors.Wrap(err) } for _, g := range grants { if accessTokenData(g) == authn.Token(ctx) { return nil } } return ErrNotAuthorized }
11. 5 func (a *Authorizer) Authorize(req *http.Request) error { policies := a.policyByRoute[req.RequestURI] if policies == nil { return errors.New("missing policy on this route") } grants, err := a.grantsByPolicies(policies) if err != nil { return errors.Wrap(err) } for _, g := range grants { if accessTokenData(g) == authn.Token(ctx) { return nil } } return ErrNotAuthorized }
12. Capability vs Identity 6
13. Capability vs. Identity 7
14. Capability vs. Identity 7
15. Capability vs. Identity 7 (a capability!)
16. Capability vs. Identity 8 %
17. Capability vs. Identity 8 % who the heck are you?
18. Capability vs. Identity 8 % who the heck are you?
19. Capability vs. Identity 8 % who the heck are you?
20. Capability vs. Identity 8 % yeah ok I guess who the heck are you?
21. Capability vs. Identity 9 ( )
22. Capability vs. Identity 9 ( )
23. Capability vs. Identity 10 ( )
24. Capability vs. Identity 10 ( ) )
25. Capability vs. Identity 10 ( ) )
26. Capability vs. Identity 10 ( ) ) yeah ok I guess
27. “The ACL model is unable to make correct access decisions for interactions involving more than two principals, since required information is not retained across message sends.” 11 Tyler Close, ACLs don’t
28. 12 　 　 　 　 　 howdy, I’m the confused deputy
29. /transfer?amount=100&recipient=Alice Cross-Site Request Forgery 13
30. Cross-Site Request Forgery 14 /transfer?amount=100&recipient=Alice&token=xyz
31. A Middle Ground 15 ( )
32. A Middle Ground 15 ( )
33. A Middle Ground 15 ( ) ❌
34. / A Middle Ground 15 ( ) ❌ ✅
35. Macaroons 16
36. Basic Auth 17 func (r *Request) SetBasicAuth(username, password string) { r.Header.Set("Authorization", "Basic "+basicAuth(username, password)) } func basicAuth(username, password string) string { auth := username + ":" + password return base64.StdEncoding.EncodeToString([]byte(auth)) } (from the net/http package)
37. Problems with Basic Auth 18
38. Problems with Basic Auth 1. Username/password required on every request 18
39. Problems with Basic Auth 1. Username/password required on every request 2.Identity-based 18
40. username:pw Token Auth 19
41. Token Auth 19 abcdefg
42. Token Auth 19 abcdefg
43. Token Auth 19 abcdefg 1234670 1 abcdefg 2 bbbbbbb 3 hijklmn 4 314159 )
44. HMACs: keyed-Hash Message Authentication Codes 20 // compute hmac func GenerateMAC(message, key []byte) []byte { mac := hmac.New(sha256.New, key) mac.Write(message) return mac.Sum(nil) } // check hmac by recalculating server-side and then checking for equality func CheckMAC(message, messageMAC, key []byte) bool { expectedMAC := GenerateMac(message, key) return hmac.Equal(messageMAC, expectedMAC) }
45. HMACs: keyed-Hash Message Authentication Codes 21
46. 22 func New(h func() hash.Hash, key []byte) hash.Hash { hm := new(hmac) hm.outer = h() hm.inner = h() hm.size = hm.inner.Size() hm.size = hm.inner.BlockSize() hm.ipad = make([]byte, hm.blocksize) hm.opad = make([]byte, hm.blocksize) if len(key) > hm.blocksize { // If key is too big, hash it. hm.outer.Write(key) key = hm.outer.Sum(nil) } copy(hm.ipad, key) copy(hm.opad, key) for i := range hm.ipad { hm.ipad[i] ^= 0x36 } for i := range hm.opad { hm.opad[i] ^= 0x5c } hm.inner.Write(hm.ipad) return hm } func (h *hmac) Sum(in []byte) []byte { origLen := len(in) in = h.inner.Sum(in) h.outer.Reset() h.outer.Write(h.opad) h.outer.Write(in[origLen:]) return h.outer.Sum(in[:origLen]) } (from the crypto/hmac package)
47. 22 func New(h func() hash.Hash, key []byte) hash.Hash { hm := new(hmac) hm.outer = h() hm.inner = h() hm.size = hm.inner.Size() hm.size = hm.inner.BlockSize() hm.ipad = make([]byte, hm.blocksize) hm.opad = make([]byte, hm.blocksize) if len(key) > hm.blocksize { // If key is too big, hash it. hm.outer.Write(key) key = hm.outer.Sum(nil) } copy(hm.ipad, key) copy(hm.opad, key) for i := range hm.ipad { hm.ipad[i] ^= 0x36 } for i := range hm.opad { hm.opad[i] ^= 0x5c } hm.inner.Write(hm.ipad) return hm } func (h *hmac) Sum(in []byte) []byte { origLen := len(in) in = h.inner.Sum(in) h.outer.Reset() h.outer.Write(h.opad) h.outer.Write(in[origLen:]) return h.outer.Sum(in[:origLen]) } (from the crypto/hmac package)
48. Auth with HMACs 23 hmac(2,) hmac(1,) 1 hmac(2,) 2 hmac(3,) 3 hmac(4,) 4 hmac(),) )
49. Auth with HMACs 23 hmac(2,)
50. Auth with HMACs 23 hmac(2,)
51. Problems with Basic Auth 1. Username/password required on every request 2.Identity-based 24
52. Macaroons are cookies with layers 25 hmac(2,) 6
53. Macaroons are cookies with layers 25 hmac(2,) 6 “perms:read-only”
54. Macaroons are cookies with layers 25 hmac(2,) 6 hmac(“perms:read-only”,6) 7
55. Macaroons are cookies with layers 25 hmac(2,) 6 hmac(“perms:read-only”,6) 7
56. 26 func mintMacaroon(ctx context.Context, teamID string) (*macaroon.Macaroon, error) { secretKey := getSecretKeyFromParameterStore() // this is a fake function // Construct a unique identifier for the macaroon. id := &macaroons.Identifier{ Nonce: make([]byte, 16), KeyId: keyID, // random string IssuedAt: ptypes.TimestampNow(), } _, err := rand.Read(id.Nonce) idBytes, err := proto.Marshal(id) // Create an unattenuated macaroon for a team m, err := macaroon.New(secretKey, string(idBytes), "") // Attenuate the omnipotent macaroon with a caveat // limiting it to requests for the provided team ID. encodedCaveats, err := proto.Marshal(&ledgerpb.CaveatList{ Caveats: []*ledgerpb.Caveat{RequestIsForTeam: teamID}, }) err = m.AddFirstPartyCaveat(string(encodedCaveats)) return m, err }
57. 26 func mintMacaroon(ctx context.Context, teamID string) (*macaroon.Macaroon, error) { secretKey := getSecretKeyFromParameterStore() // this is a fake function // Construct a unique identifier for the macaroon. id := &macaroons.Identifier{ Nonce: make([]byte, 16), KeyId: keyID, // random string IssuedAt: ptypes.TimestampNow(), } _, err := rand.Read(id.Nonce) idBytes, err := proto.Marshal(id) // Create an unattenuated macaroon for a team m, err := macaroon.New(secretKey, string(idBytes), "") // Attenuate the omnipotent macaroon with a caveat // limiting it to requests for the provided team ID. encodedCaveats, err := proto.Marshal(&ledgerpb.CaveatList{ Caveats: []*ledgerpb.Caveat{RequestIsForTeam: teamID}, }) err = m.AddFirstPartyCaveat(string(encodedCaveats)) return m, err }
58. 26 func mintMacaroon(ctx context.Context, teamID string) (*macaroon.Macaroon, error) { secretKey := getSecretKeyFromParameterStore() // this is a fake function // Construct a unique identifier for the macaroon. id := &macaroons.Identifier{ Nonce: make([]byte, 16), KeyId: keyID, // random string IssuedAt: ptypes.TimestampNow(), } _, err := rand.Read(id.Nonce) idBytes, err := proto.Marshal(id) // Create an unattenuated macaroon for a team m, err := macaroon.New(secretKey, string(idBytes), "") // Attenuate the omnipotent macaroon with a caveat // limiting it to requests for the provided team ID. encodedCaveats, err := proto.Marshal(&ledgerpb.CaveatList{ Caveats: []*ledgerpb.Caveat{RequestIsForTeam: teamID}, }) err = m.AddFirstPartyCaveat(string(encodedCaveats)) return m, err }
59. 26 func mintMacaroon(ctx context.Context, teamID string) (*macaroon.Macaroon, error) { secretKey := getSecretKeyFromParameterStore() // this is a fake function // Construct a unique identifier for the macaroon. id := &macaroons.Identifier{ Nonce: make([]byte, 16), KeyId: keyID, // random string IssuedAt: ptypes.TimestampNow(), } _, err := rand.Read(id.Nonce) idBytes, err := proto.Marshal(id) // Create an unattenuated macaroon for a team m, err := macaroon.New(secretKey, string(idBytes), "") // Attenuate the omnipotent macaroon with a caveat // limiting it to requests for the provided team ID. encodedCaveats, err := proto.Marshal(&ledgerpb.CaveatList{ Caveats: []*ledgerpb.Caveat{RequestIsForTeam: teamID}, }) err = m.AddFirstPartyCaveat(string(encodedCaveats)) return m, err }
60. 27 type Macaroon struct { location string id []byte caveats []Caveat sig [hashLen]byte version Version } type Caveat struct { Id []byte VerificationId []byte Location string } Anatomy of a Macaroon (from https://github.com/go-macaroon/macaroon)
61. 27 type Macaroon struct { location string id []byte caveats []Caveat sig [hashLen]byte version Version } type Caveat struct { Id []byte VerificationId []byte Location string } Anatomy of a Macaroon (from https://github.com/go-macaroon/macaroon)
62. 27 type Macaroon struct { location string id []byte caveats []Caveat sig [hashLen]byte version Version } type Caveat struct { Id []byte VerificationId []byte Location string } Anatomy of a Macaroon (from https://github.com/go-macaroon/macaroon)
63. 27 type Macaroon struct { location string id []byte caveats []Caveat sig [hashLen]byte version Version } type Caveat struct { Id []byte VerificationId []byte Location string } Anatomy of a Macaroon (from https://github.com/go-macaroon/macaroon)
64. 27 type Macaroon struct { location string id []byte caveats []Caveat sig [hashLen]byte version Version } type Caveat struct { Id []byte VerificationId []byte Location string } Anatomy of a Macaroon (from https://github.com/go-macaroon/macaroon)
65. // AddFirstPartyCaveat adds a caveat that will be verified // by the target service. func (m *Macaroon) AddFirstPartyCaveat(condition []byte) error { m.addCaveat(condition, nil, "") return nil } func (m *Macaroon) addCaveat(caveatId, verificationId []byte, loc string) error { m.appendCaveat(caveatId, verificationId, loc) m.sig = *keyedHash(&m.sig, caveatId) return nil } 28 Adding a Caveat (from https://github.com/go-macaroon/macaroon)
66. // AddFirstPartyCaveat adds a caveat that will be verified // by the target service. func (m *Macaroon) AddFirstPartyCaveat(condition []byte) error { m.addCaveat(condition, nil, "") return nil } func (m *Macaroon) addCaveat(caveatId, verificationId []byte, loc string) error { m.appendCaveat(caveatId, verificationId, loc) m.sig = *keyedHash(&m.sig, caveatId) return nil } 28 Adding a Caveat (from https://github.com/go-macaroon/macaroon)
67. 29 Adding a Caveat func keyedHash(key *[hashLen]byte, text []byte) *[hashLen]byte { h := keyedHasher(key) h.Write([]byte(text)) var sum [hashLen]byte hashSum(h, &sum) return &sum } func keyedHasher(key *[hashLen]byte) hash.Hash { return hmac.New(sha256.New, key[:]) } (from https://github.com/go-macaroon/macaroon)
68. 29 Adding a Caveat func keyedHash(key *[hashLen]byte, text []byte) *[hashLen]byte { h := keyedHasher(key) h.Write([]byte(text)) var sum [hashLen]byte hashSum(h, &sum) return &sum } func keyedHasher(key *[hashLen]byte) hash.Hash { return hmac.New(sha256.New, key[:]) } (from https://github.com/go-macaroon/macaroon)
69. Verifying a Macaroon 30 func authorizeRequest(teamID string, req *http.Request) error { ctx := req.Context() mac := authn.Macaroon(ctx) discharges := authn.DischargeMacaroons(ctx) if mac == nil { return httperror.ErrNotAuthenticated } verifier := caveatVerifier(teamID, req.URL.Path) err := mac.Verify(authzKey, verifier, discharges) if err != nil { return httperror.ErrNotAuthorized } return nil }
70. Verifying a Macaroon 30 func authorizeRequest(teamID string, req *http.Request) error { ctx := req.Context() mac := authn.Macaroon(ctx) discharges := authn.DischargeMacaroons(ctx) if mac == nil { return httperror.ErrNotAuthenticated } verifier := caveatVerifier(teamID, req.URL.Path) err := mac.Verify(authzKey, verifier, discharges) if err != nil { return httperror.ErrNotAuthorized } return nil }
71. Verifying a Macaroon 30 func authorizeRequest(teamID string, req *http.Request) error { ctx := req.Context() mac := authn.Macaroon(ctx) discharges := authn.DischargeMacaroons(ctx) if mac == nil { return httperror.ErrNotAuthenticated } verifier := caveatVerifier(teamID, req.URL.Path) err := mac.Verify(authzKey, verifier, discharges) if err != nil { return httperror.ErrNotAuthorized } return nil }
72. Verifying a Macaroon 30 func authorizeRequest(teamID string, req *http.Request) error { ctx := req.Context() mac := authn.Macaroon(ctx) discharges := authn.DischargeMacaroons(ctx) if mac == nil { return httperror.ErrNotAuthenticated } verifier := caveatVerifier(teamID, req.URL.Path) err := mac.Verify(authzKey, verifier, discharges) if err != nil { return httperror.ErrNotAuthorized } return nil }
73. 31 func caveatVerifier(teamID, path string) func(string) error { return func(caveatID string) error { var caveatList ledgerpb.CaveatList err := proto.Unmarshal([]byte(caveatID), &caveatList) if err != nil { return err } for _, caveat := range caveatList.Caveats { switch c := caveat.Caveat.(type) { case *ledgerpb.Caveat_RequestIsForTeam: reqIsForTeam := strings.ToLower(c.RequestIsForTeam) if strings.ToLower(teamID) != reqIsForTeam { return fmt.Errorf("credential scoped to team %q", reqIsForTeam) } case *ledgerpb.Caveat_NotBefore: ts, err := ptypes.Timestamp(c.NotBefore) if err != nil { return err } if time.Now().Before(ts) { return fmt.Errorf("credential not valid before %s", ts.Format(time.RFC3339)) } case *ledgerpb.Caveat_NotAfter: ts, err := ptypes.Timestamp(c.NotAfter) if err != nil { return err } if ts.Before(time.Now()) { return fmt.Errorf("credential expired at %s", ts.Format(time.RFC3339)) } case *ledgerpb.Caveat_OperationIsWithinCoarsePolicy: var n int var routePolicy ledgerpb.CoarsePolicy for pattern, pol := range minimumPolicyByRoute { if !pathMatch(pattern, path) { continue
74. Different kinds of caveats 32 * * my actual sister
75. Third-party caveats 33 hmac(2,) 6 hmac(“perms:read-only”,6) 7 hmac([3rd party caveat],7) :
76. Third-party caveats 33 hmac(2,) 6 hmac(“perms:read-only”,6) 7 hmac([3rd party caveat],7) : type Caveat struct { Id []byte VerificationId []byte Location string }
77. Verifying a third-party macaroon 34
78. Verifying a third-party macaroon 34 d
79. Verifying a third-party macaroon 34 : d d
80. Macaroons at Chain 35
82. Sequence Architecture 37 ledgerd dashboard SDK <
83. Sequence Architecture 38 ledgerd dashboard SDK <
84. Sequence Architecture 38 ledgerd dashboard SDK <
85. Sequence Architecture 38 ledgerd dashboard SDK <
86. Sequence Architecture 38 ledgerd dashboard SDK <
87. Sequence Architecture 38 ledgerd dashboard SDK <
88. Sequence Architecture 38 ledgerd dashboard SDK <
89. Sequence Architecture 39 ledgerd dashboard SDK <
90. Sequence Architecture 39 ledgerd dashboard SDK <
91. Macaroons: The Bad Parts 40
92. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 41
93. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 42
94. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 3. Confusing behavior for users 43
95. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 3. Confusing behavior for users 4. Impossible to revoke immediately 44
96. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 3. Confusing behavior for users 4. Impossible to revoke immediately 5. Tricky format 45
97. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 3. Confusing behavior for users 4. Impossible to revoke immediately 5. Tricky format 6. Maybe misnamed? 46
100. Macaroons: The Bad Parts 1. New availability dependency on Dashboard 2. Difficult to use locally 3. Confusing behavior for users 4. Impossible to revoke immediately 5. Tricky format 6. Maybe misnamed? 49
102. Macaroons are exciting technology. 50
103. Macaroons are exciting technology. (We like boring technology.) 50
104. Further reading and watching • ACLs don’t http://waterken.sourceforge.net/aclsdont/ • Macaroons paper https://ai.google/research/pubs/pub41892 • Tony Arcieri’s talk on Macaroons https://www.youtube.com/watch?v=bFn-wjQtxZ0 • Ulfar Erlingsson, Macaroons co-author https://air.mozilla.org/macaroons-cookies-with- contextual-caveats-for-decentralized-authorization-in-the-cloud/ • Dan McKinley, Choose Boring Technology http://mcfunley.com/choose-boring-technology • Macarons images from https://www.danasbakery.com/collections/flavors 51 tess@chain.com // @_tessr