Building a go tool to modify struct tags

Building a go tool to modify struct tags

Struct field tags are an important part of encode/decode types, especially when using packages such as encoding/json. However, modifying tags is repetitive, cumbersome and open to human errors. We can make it easy to modify tags with an automated tool that is written for this sole purpose.

B1019ca5714cf8e9951868d6bc517827?s=128

Fatih Arslan

July 13, 2017
Tweet

Transcript

  1. 1.
  2. 6.

    type Example struct { StatusID int64 Foo string Bar bool

    Server struct { Address string TLS bool } DiskSize int64 Volumes []string }
  3. 7.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  4. 23.

    type Example struct { Foo string "json:\"foo\"" } quotes instead

    of backticks (works, but not fun to deal with)
  5. 26.
  6. 29.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  7. 31.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  8. 32.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"tls" xml:"tls"`
  9. 33.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  10. 34.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"server"`
  11. 35.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  12. 36.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"disk_size"`
  13. 37.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  14. 38.

    type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"volumes"`
  15. 40.

    Old implementation • From the cursor search backwards for the

    first 'struct {' literal • Once found, do a forward search for the right hand brace } • Get the line numbers between the two braces • For each line, get the first identifier, convert it to camel_case word and then append to the same line • A combination of Vim's search() function and regex is being used
  16. 47.
  17. 48.

    No Formatting type Example struct { Bar string `json:"bar"` Foo

    bool `json:"foo"` } type Example struct { Bar string `json:"bar"` Foo bool `json:"foo"` } want have
  18. 49.

    In-line comments type Example struct { Bar string `json:"bar"` //

    comment for bar Foo bool `json:"foo"` } type Example struct { Bar string // comment for bar `json:"bar"` Foo bool `json:"foo"` } want have
  19. 50.

    Nested structs type Example struct { Bar bool `json:"bar"` Server

    struct { `json:"server"` Address string `json:"address"` } } type Example struct { Bar bool `json:"bar"` Server struct { Address string `json:"address"` } `json:"server"` } have want
  20. 51.

    Duplicate tags type Example struct { Bar string `json:"bar"` `xml:"bar"`

    Foo bool `xml:"foo"` } have type Example struct { Bar string `json:"bar" xml:"bar"` Foo bool `xml:"foo"` } want
  21. 52.

    more quirks • not able to remove tags • not

    able to add or remove options • field with interface{} types don't work • comment with braces ({ and }) don't work • ...
  22. 72.

    type Example struct { Foo string `json:"foo"` } *ast.TypeSpec *ast.StructType

    *ast.FieldList *ast.Field *ast.BasicLit `json:"foo"` (a.k.a: struct tag)
  23. 73.

    type Example struct { Foo string `json:"foo"` } *ast.TypeSpec *ast.StructType

    *ast.FieldList *ast.Field []*ast.Ident *ast.Ident *ast.BasicLit
  24. 74.
  25. 77.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  26. 78.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  27. 79.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  28. 80.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  29. 81.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  30. 82.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false }) Field: Foo Tag: `json:"foo"`
  31. 86.

    package main import ( "fmt" "reflect" ) func main() {

    tag := reflect.StructTag(`json:"foo"`) value := tag.Get("json") fmt.Printf("value: %q\n", value) }
  32. 87.

    package main import ( "fmt" "reflect" ) func main() {

    tag := reflect.StructTag(`json:"foo"`) value := tag.Get("json") fmt.Printf("value: %q\n", value) } $ go run main.go value: "foo"
  33. 89.

    Issues with reflect.StructTag • can't detect if the tag is

    malformed (only go vet knows that) • doesn't know the semantics of options (i.e: omitempty) • doesn't return all existing tags • modifying existing tags is not possible
  34. 93.
  35. 94.

    Parse and list all tags tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`)

    if err != nil { panic(err) } // iterate over all key-value pairs for _, t := range tags.Tags() { fmt.Printf("tag: %+v\n", t) } $ go run main.go tag: json:"foo,omitempty" tag: xml:"foo"
  36. 95.

    Get a single Tag tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`) if

    err != nil { panic(err) } jsonTag, err := tags.Get("json") if err != nil { panic(err) } fmt.Println(jsonTag) // Output: json:"foo,omitempty" fmt.Println(jsonTag.Key) // Output: json fmt.Println(jsonTag.Name) // Output: foo fmt.Println(jsonTag.Options) // Output: [omitempty]
  37. 96.

    Change existing tag jsonTag, err := tags.Get(`json:"foo,omitempty"`) if err !=

    nil { panic(err) } jsonTag.Name = "bar" jsonTag.Options = nil tags.Set(jsonTag) fmt.Println(tags) json:"bar"
  38. 97.

    Add new tag tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`) hclTag :=

    &structtag.Tag{ Key: "hcl", Name: "gopher", Options: []string{"squash"}, } // add new tag tags.Set(hclTag) fmt.Println(tags) json:"foo,omitempty" xml:"foo" hcl:"gopher,squash"
  39. 98.

    Add/remove options tags, err := structtag.Parse(`json:"foo" xml:"bar,comment"`) if err !=

    nil { panic(err) } tags.AddOptions("json", "omitempty") tags.DeleteOptions("xml", "comment") fmt.Println(tags) // json:"foo,omitempty" xml:"bar"
  40. 103.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  41. 104.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  42. 106.

    cfg := &config{ file: *flagFile, line: *flagLine, structName: *flagStruct, offset:

    *flagOffset, output: *flagOutput, write: *flagWrite, clear: *flagClearTags, clearOption: *flagClearOptions, transform: *flagTransform, sort: *flagSort, override: *flagOverride, } $ gomodifytags --file example.go ... Use flags to set configuration
  43. 107.

    Things we need: 1. What content to process 2. Where

    and what to output 3. Which struct to modify 4. Which tags to modify
  44. 108.

    Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  45. 109.

    Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  46. 110.

    Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  47. 111.

    Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  48. 112.

    package main type Example struct { Foo string } $

    gomodifytags -file example.go -struct Example -add-tags json
  49. 115.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  50. 118.

    gomodifytags parse parse go/parser package main type Example struct {

    Foo string } *ast.TypeSpec *ast.StructType *ast.FieldList *ast.Field []*ast.Ident *ast.Ident tag
  51. 119.

    func (c *config) parse() (ast.Node, error) { c.fset = token.NewFileSet()

    var contents interface{} if c.modified != nil { archive, err := buildutil.ParseOverlayArchive(c.modified) if err != nil { return nil, fmt.Errorf("failed to parse -modified archive: %v", err) } fc, ok := archive[c.file] if !ok { return nil, fmt.Errorf("couldn't find %s in archive", c.file) } contents = fc } return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments) } Parse content
  52. 120.

    func (c *config) parse() (ast.Node, error) { c.fset = token.NewFileSet()

    var contents interface{} if c.modified != nil { archive, err := buildutil.ParseOverlayArchive(c.modified) if err != nil { return nil, fmt.Errorf("failed to parse -modified archive: %v", err) } fc, ok := archive[c.file] if !ok { return nil, fmt.Errorf("couldn't find %s in archive", c.file) } contents = fc } return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments) } Parse content (cont.)
  53. 124.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  54. 129.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  55. 130.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  56. 131.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 struct name: Server
  57. 132.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 struct name: Server start line: 8 end line: 10
  58. 133.

    package main type Example struct { Foo string } type

    Server struct { Name string Port| int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 offset byte: 96 (of 163)
  59. 134.

    package main type Example struct { Foo string } type

    Server struct { Name string Port| int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 offset byte: 96 (of 163) start line: 8 end line: 10
  60. 135.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 start line: 8 end line: 10 .. or specify explicit lines
  61. 136.

    package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 start line: 9 end line: 9 .. or specify explicit lines
  62. 139.

    func (c *config) findSelection(node ast.Node) (int, int, error) { if

    c.line != "" { return c.lineSelection(node) } else if c.offset != 0 { return c.offsetSelection(node) } else if c.structName != "" { return c.structSelection(node) } return 0, 0, errors.New("-line, -offset or -struct is not passed") } Find selection
  63. 140.

    func (c *config) findSelection(node ast.Node) (int, int, error) { if

    c.line != "" { return c.lineSelection(node) } else if c.offset != 0 { return c.offsetSelection(node) } else if c.structName != "" { return c.structSelection(node) } return 0, 0, errors.New("-line, -offset or -struct is not passed") } Find selection
  64. 142.

    // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType, 0) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  65. 143.

    // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs } type structType struct { name string node *ast.StructType }
  66. 144.

    // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  67. 145.

    // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs } type structType struct { name string node *ast.StructType }
  68. 146.

    // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  69. 147.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  70. 148.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  71. 149.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  72. 150.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  73. 151.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  74. 152.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  75. 153.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  76. 154.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  77. 155.

    var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  78. 159.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  79. 164.

    gomodifytags *ast.StructType 6 7 8 9 10 11 12 *ast.Field

    *ast.Field *ast.Field *ast.Field *ast.Field
  80. 166.

    type Example struct { Foo string `json:"foo"` } gomodifytags *ast.StructType

    *ast.Field []*ast.Ident *ast.Ident *ast.BasicLit
  81. 169.

    type Example struct { Foo string `json:"bar" } gomodifytags *ast.StructType

    *ast.Field Foo string `json:"foo" `json:"bar"` type Example struct { Foo string `json:"foo"` }
  82. 171.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = `json:"bar"` } return false }) After finding a field, replace the value
  83. 172.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = `json:"bar"` } return false }) Tag: `json:"foo"` Tag: `json:"bar"` After finding a field, replace the value
  84. 175.

    gomodifytags *ast.StructType *ast.Field DiskSize string json:"disk_size" process() type Example struct

    { DiskSize string } type Example struct { DiskSize string `json:"disk_size"` }
  85. 176.

    src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = process(field) } return false }) tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  86. 177.

    gomodifytags *ast.StructType 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field

    select field between start and end lines ... Modify overview
  87. 178.

    gomodifytags *ast.StructType 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field

    ... and rewrite selected fields tags 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field rewrite Modify overview
  88. 179.
  89. 180.

    func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  90. 181.

    func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  91. 182.

    func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  92. 183.

    for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  93. 184.

    for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  94. 185.

    for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  95. 186.

    for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  96. 187.

    for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  97. 188.

    for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  98. 190.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  99. 191.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  100. 192.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  101. 193.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil } tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  102. 194.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil } tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  103. 195.

    func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  104. 196.

    func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  105. 197.
  106. 199.

    func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  107. 202.

    gomodifytags ast.Node (modified) go/format ... and output the result to

    stdout package main type Example struct { Foo string `json:"foo"` }
  108. 203.

    package main type Server struct { Name string Port int

    EnableLogs bool BaseDomain string Credentials struct { Username string Password string } } package main type Server struct { Name string `json:"name"` Port int `json:"port"` EnableLogs bool `json:"enable_logs"` BaseDomain string `json:"base_domain"` Credentials struct { Username string `json:"username"` Password string `json:"password"` } `json:"credentials"` } Input Output $ gomodifytags -file example.go -struct Server -add-tags json
  109. 204.

    gomodifytags ast.Node (modified) go/format Also has support for custom JSON

    output { "start": 3, "end": 5, "lines": [ "type Example struct {", " Foo string `json:\"foo\"`", "}" ] }
  110. 205.

    package main type Server struct { Name string Port int

    EnableLogs bool BaseDomain string Credentials struct { Username string Password string } } { "start": 3, "end": 12, "lines": [ "type Server struct {", " Name string `xml:\"name\"`", " Port int `xml:\"port\"`", " EnableLogs bool `xml:\"enable_logs\"`", " BaseDomain string `xml:\"base_domain\"`", " Credentials struct {", " Username string `xml:\"username\"`", " Password string `xml:\"password\"`", " } `xml:\"credentials\"`", "}" ] } Input JSON Output $ gomodifytags -file example.go -struct Server -add-tags json -format json
  111. 206.

    Why stdout? • by default, it means "dry-run" • immediate

    feedback • tools can redirect the output • composable (gomodifytags myfile.go > newfile.go)
  112. 207.

    func (c *config) format(file ast.Node) (string, error) { switch c.output

    { case "source": // return formatted source case "json": // return only changes in json default: return "", fmt.Errorf("unknown output mode: %s", c.output) } } Output the result
  113. 208.

    func (c *config) format(file ast.Node) (string, error) { switch c.output

    { case "source": // return formatted source case "json": // return only changes in json default: return "", fmt.Errorf("unknown output mode: %s", c.output) } } Output the result
  114. 209.

    var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  115. 210.

    var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  116. 211.

    var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  117. 212.

    var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } var lines []string scanner := bufio.NewScanner(bytes.NewBufferString(buf.String())) for scanner.Scan() { lines = append(lines, scanner.Text()) } if c.start > len(lines) { return "", errors.New("line selection is invalid") } ... JSON output
  118. 213.

    var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } var lines []string scanner := bufio.NewScanner(bytes.NewBufferString(buf.String())) for scanner.Scan() { lines = append(lines, scanner.Text()) } if c.start > len(lines) { return "", errors.New("line selection is invalid") } ... JSON output
  119. 214.

    out := &output{ Start: c.start, End: c.end, Lines: lines[c.start-1 :

    c.end], } o, err := json.MarshalIndent(out, "", " ") if err != nil { return "", err } return string(o), nil JSON output (cont.)
  120. 215.

    out := &output{ Start: c.start, End: c.end, Lines: lines[c.start-1 :

    c.end], } o, err := json.MarshalIndent(out, "", " ") if err != nil { return "", err } return string(o), nil JSON output (cont.)
  121. 216.

    package main type Example struct { Foo string } parse

    find modify gomodifytags format format
  122. 217.

    package main type Example struct { Foo string } parse

    find modify gomodifytags format Overview
  123. 218.
  124. 221.

    package main type Example struct { Foo string } -file

    example.go -struct Example -add-tags json gomodifytags
  125. 225.
  126. 226.

    package main type Example struct { Foo string } package

    main type Example struct { Foo string `json:"foo"` } parse find modify format gomodifytags
  127. 227.
  128. 228.

    Built from the ground up for editors • selecting structs

    based on offset or range of lines • write result to stdout or file • json output for easy parsing • support for unsaved buffers • individual cli flags for all features
  129. 229.

    Supported editors • vim with vim-go • atom with go-plus

    (thanks Zac Bergquist) • vscode with vscode-go (thanks Ramya Rao) • emacs - WIP (https://github.com/syohex/emacs-go-add-tags/issues/ 6)
  130. 230.
  131. 231.

    Recap • A single tool to rule all editors •

    Easy maintenance • Free of bugs due stdlib tooling (go/parser family) • Happy and productive users