Sourcegraph at GoSF: Building and Using a Semantic Go Code Search Engine

Presented at GoSF, organized by Iron.io and hosted by Cisco SF (http://www.meetup.com/golangsf/events/146083272/). Blog post at https://sourcegraph.com/blog/go-interfaces-and-implementations

Transcript

1. ✱ Sourcegraph https://sourcegraph.com Building and Using a Semantic Go Code Search Engine Quinn Slack sqs@sourcegraph.com @sqs GoSF, 2013-11-06 https://sourcegraph.com/blog/go-interfaces-and-implementations Sunday, November 10, 13
2. What is Sourcegraph? • Searches across 20M+ packages/functions from GitHub, Google Code, etc. • Displays docs, usage examples (automatically extracted/ranked), and annotated source code • Shows who uses your code, or specific functions • And now...lists all* implementations of a Go interface Sunday, November 10, 13
3. Inspired by Go What if all languages had godoc? What if all languages gave every symbol a URI? Sunday, November 10, 13
4. Step 1. Analyze a single Go package import  (                "code.google.com/p/go.tools/go/types" ) • Iterate through each scope, assigning each symbol (“anything that can be documented”) a unique URI • Walk the AST, resolving every Ident to its referent symbol URI Sunday, November 10, 13
5. Step 2. Store the resolved representation type  SymbolURI  struct  {                Repo  repo.URI                Path  SymbolPath                Lang  sg.Lang } type  Symbol  struct  {  SymbolURI;  ...  } type  Ref  struct  {  Dst  SymbolURI;  Src  FileLocation;  ...  } • Each symbol is uniquely keyed on (repo, path, lang) • Each ref maps a char range in a file to a symbol Sunday, November 10, 13
6. Step 2.5. Store interface/implements metadata type  GoInterfaceMethod  struct  {                OfSymbol  SymbolURI                Repo  repo.URI                CanonicalSignature  string } • Store the pointer method set signatures of each type, as determined by go/types Sunday, November 10, 13
7. Step 3. Jam & Repeat • Jam it all into PostgreSQL (using lib/pq and coopernurse/gorp) • ... • Handle a million edge cases (versions, deps, syntax errors, cgo, forks, etc.) • Repeat for all 50,000+ Go repositories out there • Then do the same for other languages Sunday, November 10, 13
8. Live demo: • Who uses gorilla/mux.(*Router).Handle? • Who uses my code? • What types implement http.ResponseWriter? • What interfaces does tls.Conn implement? ✱ Sourcegraph https://sourcegraph.com Sunday, November 10, 13
9. Questions? ✱ Sourcegraph https://sourcegraph.com Try it out: Sunday, November 10, 13