Security of Go Modules and Vulnerability Scanning in GoCenter

This was a 7 minute lightening talk presented at GopherCon EU!

Golang developers care a lot about security and as Go modules become more widely used, they need more ways to assure these publicly shared packages are safe.

One unique feature included with Golang version 1.13 is the foresight that went into authentication and security for Go modules. When a developer creates a new module or a new version of an existing module, a go.sum file included there creates a list of SHA-256 hashes that are unique to that module version. That go.sum file is then sent to Google’s official checksum database where it is stored and used to verify that modules haven’t been tampered with when accessed later by a GOPROXY. This helps keep the integrity of packages intact. In this talk, we’ll go over the behavior of the checksum database, how it protects Go modules, and how the Merkle Tree works.

Now, while the checksum authentication feature helps create trust among developers, it isn’t fully tamperproof. If a vulnerability is introduced in the original module’s files, the gosumdb will only be able to indicate that the module wasn’t changed later. This doesn’t solve the problem of malicious code being introduced in the very first commit.

Luckily, GoCenter can now tell you when any Go module has a known vulnerability. We’ve brought the power of JFrog Xray’s security scanning to this reliable repository of Go modules for the Golang developer community.

Transcript

1. SECURITY OF GO MODULES Lightning Talk By Deep Datta
2. DEPENDENCY MANAGEMENT IN GOLANG glide, gopath, dep, vendoring… These weren’t the right answer for Go... GO 1.11 INTRODUCED GO MODULES GO 1.13 GO MODULES BECOME THE STANDARD Basic data integrity features are introduced with the go.sum and go.mod Go module mirror and Go checksum database
3. GO.SUM FILE github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38 = github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ = github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4 = github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4 = github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE = github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME = github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs = golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc = golang.org/x/sys v0.0.0-20190422165155-953cdadca894 /go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs =
4. SECURE HASH ALGORITHM (SHA2) AS CHECKSUMS cryptographic hash algorithms produce irreversible and unique hashes Irreversible because you can’t use the hash to figure out what the original piece of data was Unique means that two different pieces of data can never produce the same hash
5. MERKLE TREE BASICS At its core, a Merkle Tree is a list of items representing the data that should be verified. data data data data hash hash hash hash hash hash hash go.sum data Each of these items is inserted into a leaf node and a tree of hashes is constructed. If you change the data, the hash will also change that branch all the through the tree. hash hash root
6. Module 1 (go.sum) go.sum go.sum go.sum List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. Cz4ceDQGXuKRnVBDTS23GTn/pU5 OE2C0WrNTOYK1Uuc= go.sum List of SHA-256 base64 hashes. Cz4ceDQGXuKRnVBDTS23GTn/pU5 OE2C0WrNTOYK1Uuc= https://sum.golang.org/ Module 2 (go.sum) CHECKSUMS
7. go.sum go.sum go.sum List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. Cz4ceDQGXuKRnVBDTS23GTn/pU5 OE2C0WrNTOYK1Uuc= https://sum.golang.org/ HOW HASHES PROTECT MODULE USERS Module 1 (go.sum) Module 1 (go.sum) Minor content change
8. Let’s say you create your first Go module. package main import { “encoding/json”, “io/ioutil”, “net/http”, “os”, “text/template” } type TodoPageData struct { PageTitle string Todos []Todo } ... You save it as mod1 go.mod go.sum main.go
9. main.go mod1 You can fix your files and create a new version for everyone called mod/v2 package main import { “encoding/json”, “io/ioutil”, “net/http”, “os”, “html/template” } type TodoPageData struct { PageTitle string Todos []Todo } ... Semantic import versioning
10. BOTH V1 and V2 ARE COMMITTED TO SUMDB go.sum go.sum go.sum List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. PIRibHv4MatM3XXNO2BJeFLZwZ2L vZgfQ5+UNI2im4= List of SHA-256 base64 hashes. Cz4ceDQGXuKRnVBDTS23GTn/pU5 OE2C0WrNTOYK1Uuc= go.sum List of SHA-256 base64 hashes. Cz4ceDQGXuKRnVBDTS23GTn/pU5 OE2C0WrNTOYK1Uuc= github.com/deep/ mod1 github.com/deep/ mod1/v2
11. ...so if someone imports…. package main import { “encoding/json”, “io/ioutil”, “net/http”, “os”, “text/template” } type TodoPageData struct { PageTitle string Todos []Todo } ... That original mod1 main.go They open themselves up to a XSS attack ...imagine if your app is a dependency for other projects...
12. FREE VULNERABILITY DATA IN GOCENTER.IO Set Me Up: export GOPROXY=https://gocenter.io
13. https://gocenter.io Twitter @DeepDattax deepd@jfrog.com