$30 off During Our Annual Pro Sale. View Details »

Building a go tool to modify struct tags

Building a go tool to modify struct tags

Struct field tags are an important part of encode/decode types, especially when using packages such as encoding/json. However, modifying tags is repetitive, cumbersome and open to human errors. We can make it easy to modify tags with an automated tool that is written for this sole purpose.

Fatih Arslan

July 13, 2017
Tweet

More Decks by Fatih Arslan

Other Decks in Programming

Transcript

  1. Building a go tool
    to modify struct tags
    Fatih Arslan
    Software Engineer @DigitalOcean

    View Slide

  2. Why do we need
    tooling?

    View Slide

  3. type Example struct {
    Foo string
    }

    View Slide

  4. type Example struct {
    Foo string `json:"foo"`
    }

    View Slide

  5. What about structs with
    many fields?

    View Slide

  6. type Example struct {
    StatusID int64
    Foo string
    Bar bool
    Server struct {
    Address string
    TLS bool
    }
    DiskSize int64
    Volumes []string
    }

    View Slide

  7. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  8. Working with struct tags
    is not fun

    View Slide

  9. So, what's a struct tag?
    type Example struct {
    Foo string `json:"foo"`
    }

    View Slide

  10. There is no official spec of
    the struct tag

    View Slide

  11. Struct tag definition

    View Slide

  12. But fortunately, we have an
    inofficial one in stdlib!

    View Slide

  13. Spec is defined in the
    reflect package

    View Slide

  14. reflect.StructTag

    View Slide

  15. Let's decompose the spec

    View Slide

  16. json:"foo"

    View Slide

  17. json:"foo"
    key

    View Slide

  18. json:"foo"
    value
    (quoted)

    View Slide

  19. json:"foo"
    colon
    (not specified clearly)

    View Slide

  20. json:"foo" xml:"foo"
    space separation

    View Slide

  21. type Example struct {
    Foo string `json:"foo"`
    }
    backticks

    View Slide

  22. type Example struct {
    Foo string `json:"foo"`
    }

    View Slide

  23. type Example struct {
    Foo string "json:\"foo\""
    }
    quotes instead of backticks
    (works, but not fun to deal with)

    View Slide

  24. json:"foo,omitempty"
    value

    View Slide

  25. json:"foo,omitempty"
    option
    (not part of the spec)

    View Slide

  26. Recap

    View Slide

  27. `json:"foo,omitempty"
    option
    (not part of the spec)
    space separation
    key
    value
    xml:"bar"`
    backtick

    View Slide

  28. Back to our initial slide!

    View Slide

  29. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  30. It has some errors

    View Slide

  31. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  32. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }
    `json:"tls" xml:"tls"`

    View Slide

  33. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  34. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }
    `json:"server"`

    View Slide

  35. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  36. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }
    `json:"disk_size"`

    View Slide

  37. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    }

    View Slide

  38. type Example struct {
    StatusID int64 `json:"status_id"`
    Foo string `json:"foo" xml:"foo"`
    Bar bool `json:"bar" xml:"bar"`
    Server struct {
    Address string `json:"address"`
    TLS bool `json:"tls",xml:"tls"`
    } `json="server"`
    DiskSize int64 `json:disk_size`
    Volumes []string `"json":"volumes"`
    } `json:"volumes"`

    View Slide

  39. First attempt
    (to automate it)

    View Slide

  40. Old implementation
    • From the cursor search backwards for the first 'struct {' literal
    • Once found, do a forward search for the right hand brace }
    • Get the line numbers between the two braces
    • For each line, get the first identifier, convert it to camel_case word
    and then append to the same line
    • A combination of Vim's search() function and regex is being used

    View Slide

  41. type Example struct {
    Foo string
    Bar |bool
    }
    cursor

    View Slide

  42. type Example struct {
    Foo string
    Bar |bool
    }

    View Slide

  43. type Example struct {
    Foo string
    Bar |bool
    }

    View Slide

  44. type Example struct {
    Foo string
    Bar |bool
    }

    View Slide

  45. type Example struct {
    Foo string
    Bar |bool
    }

    View Slide

  46. type Example struct {
    Foo string `json:"foo"`
    Bar |bool `json:"bar"`
    }

    View Slide

  47. Problems

    View Slide

  48. No Formatting
    type Example struct {
    Bar string `json:"bar"`
    Foo bool `json:"foo"`
    }
    type Example struct {
    Bar string `json:"bar"`
    Foo bool `json:"foo"`
    }
    want
    have

    View Slide

  49. In-line comments
    type Example struct {
    Bar string `json:"bar"` // comment for bar
    Foo bool `json:"foo"`
    }
    type Example struct {
    Bar string // comment for bar `json:"bar"`
    Foo bool `json:"foo"`
    }
    want
    have

    View Slide

  50. Nested structs
    type Example struct {
    Bar bool `json:"bar"`
    Server struct { `json:"server"`
    Address string `json:"address"`
    }
    }
    type Example struct {
    Bar bool `json:"bar"`
    Server struct {
    Address string `json:"address"`
    } `json:"server"`
    }
    have
    want

    View Slide

  51. Duplicate tags
    type Example struct {
    Bar string `json:"bar"` `xml:"bar"`
    Foo bool `xml:"foo"`
    }
    have
    type Example struct {
    Bar string `json:"bar" xml:"bar"`
    Foo bool `xml:"foo"`
    }
    want

    View Slide

  52. more quirks
    • not able to remove tags
    • not able to add or remove options
    • field with interface{} types don't work
    • comment with braces ({ and }) don't work
    • ...

    View Slide

  53. How do we fix it?

    View Slide

  54. Second attempt
    (and also the final one )

    View Slide

  55. Two issues that need to be fixed
    Parse struct
    Parse struct tag

    View Slide

  56. 1. Parse Struct

    View Slide

  57. The Go Parser Family

    View Slide

  58. go/token go/scanner
    go/parser
    go/ast
    go/printer

    View Slide

  59. First,
    a quick tutorial about go/ast

    View Slide

  60. 3 + 2
    *ast.BinaryExpr

    View Slide

  61. 3 + 2
    *ast.BinaryExpr
    *ast.BasicLit
    3

    View Slide

  62. 3 + 2
    *ast.BinaryExpr
    *ast.BasicLit
    2

    View Slide

  63. 3 + 2
    *ast.BinaryExpr
    *ast.BasicLit *ast.BasicLit
    3 2
    +

    View Slide

  64. 3 + (7 - 5)
    *ast.BinaryExpr
    *ast.BasicLit
    3
    +
    *ast.BinaryExpr
    *ast.BasicLit *ast.BasicLit
    5
    -
    7

    View Slide

  65. 3 + (7 - 5)
    *ast.BinaryExpr
    *ast.BasicLit
    3
    +
    *ast.BinaryExpr
    *ast.BasicLit *ast.BasicLit
    5
    -
    7

    View Slide

  66. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec

    View Slide

  67. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType

    View Slide

  68. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList

    View Slide

  69. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field

    View Slide

  70. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    Foo

    View Slide

  71. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field *ast.Ident
    string

    View Slide

  72. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    *ast.BasicLit
    `json:"foo"`
    (a.k.a: struct tag)

    View Slide

  73. type Example struct {
    Foo string `json:"foo"`
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    *ast.BasicLit

    View Slide

  74. ast.Node

    View Slide

  75. All nodes define the
    starting and ending positions

    View Slide

  76. How do we parse a Struct?

    View Slide

  77. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })

    View Slide

  78. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })

    View Slide

  79. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })

    View Slide

  80. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })

    View Slide

  81. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })

    View Slide

  82. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    fmt.Printf("Field: %s\n", field.Names[0].Name)
    fmt.Printf("Tag: %s\n", field.Tag.Value)
    }
    return false
    })
    Field: Foo
    Tag: `json:"foo"`

    View Slide

  83. 2. Parse Struct Tag

    View Slide

  84. How to parse a Struct Tag?

    View Slide

  85. Remember this?

    View Slide

  86. package main
    import (
    "fmt"
    "reflect"
    )
    func main() {
    tag := reflect.StructTag(`json:"foo"`)
    value := tag.Get("json")
    fmt.Printf("value: %q\n", value)
    }

    View Slide

  87. package main
    import (
    "fmt"
    "reflect"
    )
    func main() {
    tag := reflect.StructTag(`json:"foo"`)
    value := tag.Get("json")
    fmt.Printf("value: %q\n", value)
    }
    $ go run main.go
    value: "foo"

    View Slide

  88. reflect.StructTag is not
    perfect ...

    View Slide

  89. Issues with reflect.StructTag
    • can't detect if the tag is malformed (only go vet knows that)
    • doesn't know the semantics of options (i.e: omitempty)
    • doesn't return all existing tags
    • modifying existing tags is not possible

    View Slide

  90. reflect
    cmd/vet

    View Slide

  91. Let's improve it
    with a custom package

    View Slide

  92. import "github.com/fatih/structtag"

    View Slide

  93. structtag

    View Slide

  94. Parse and list all tags
    tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`)
    if err != nil {
    panic(err)
    }
    // iterate over all key-value pairs
    for _, t := range tags.Tags() {
    fmt.Printf("tag: %+v\n", t)
    }
    $ go run main.go
    tag: json:"foo,omitempty"
    tag: xml:"foo"

    View Slide

  95. Get a single Tag
    tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`)
    if err != nil {
    panic(err)
    }
    jsonTag, err := tags.Get("json")
    if err != nil {
    panic(err)
    }
    fmt.Println(jsonTag) // Output: json:"foo,omitempty"
    fmt.Println(jsonTag.Key) // Output: json
    fmt.Println(jsonTag.Name) // Output: foo
    fmt.Println(jsonTag.Options) // Output: [omitempty]

    View Slide

  96. Change existing tag
    jsonTag, err := tags.Get(`json:"foo,omitempty"`)
    if err != nil {
    panic(err)
    }
    jsonTag.Name = "bar"
    jsonTag.Options = nil
    tags.Set(jsonTag)
    fmt.Println(tags)
    json:"bar"

    View Slide

  97. Add new tag
    tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`)
    hclTag := &structtag.Tag{
    Key: "hcl",
    Name: "gopher",
    Options: []string{"squash"},
    }
    // add new tag
    tags.Set(hclTag)
    fmt.Println(tags)
    json:"foo,omitempty" xml:"foo" hcl:"gopher,squash"

    View Slide

  98. Add/remove options
    tags, err := structtag.Parse(`json:"foo" xml:"bar,comment"`)
    if err != nil {
    panic(err)
    }
    tags.AddOptions("json", "omitempty")
    tags.DeleteOptions("xml", "comment")
    fmt.Println(tags) // json:"foo,omitempty" xml:"bar"

    View Slide

  99. Both issues are fixed now
    Parse struct (using go/parser)
    Parse struct tag (using fatih/structtag)

    View Slide

  100. Write a CLI tool

    View Slide

  101. gomodifytags

    View Slide

  102. go get github.com/fatih/gomodifytags

    View Slide

  103. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  104. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  105. Fetch configuration settings

    View Slide

  106. cfg := &config{
    file: *flagFile,
    line: *flagLine,
    structName: *flagStruct,
    offset: *flagOffset,
    output: *flagOutput,
    write: *flagWrite,
    clear: *flagClearTags,
    clearOption: *flagClearOptions,
    transform: *flagTransform,
    sort: *flagSort,
    override: *flagOverride,
    }
    $ gomodifytags --file example.go ...
    Use flags to set
    configuration

    View Slide

  107. Things we need:
    1. What content to process
    2. Where and what to output
    3. Which struct to modify
    4. Which tags to modify

    View Slide

  108. Things we need:
    type config struct {
    file string
    modified io.Reader
    output string
    write bool
    offset int
    structName string
    line string
    start, end int
    remove []string
    add []string
    override bool
    transform string
    sort bool
    clear bool
    addOpts []string
    removeOpts []string
    clearOpt bool
    }
    1. What content to process
    2. Where and what to output
    3. Which struct to modify
    4. Which tags to modify

    View Slide

  109. Things we need:
    type config struct {
    file string
    modified io.Reader
    output string
    write bool
    offset int
    structName string
    line string
    start, end int
    remove []string
    add []string
    override bool
    transform string
    sort bool
    clear bool
    addOpts []string
    removeOpts []string
    clearOpt bool
    }
    1. What content to process
    2. Where and what to output
    3. Which struct to modify
    4. Which tags to modify

    View Slide

  110. Things we need:
    type config struct {
    file string
    modified io.Reader
    output string
    write bool
    offset int
    structName string
    line string
    start, end int
    remove []string
    add []string
    override bool
    transform string
    sort bool
    clear bool
    addOpts []string
    removeOpts []string
    clearOpt bool
    }
    1. What content to process
    2. Where and what to output
    3. Which struct to modify
    4. Which tags to modify

    View Slide

  111. Things we need:
    type config struct {
    file string
    modified io.Reader
    output string
    write bool
    offset int
    structName string
    line string
    start, end int
    remove []string
    add []string
    override bool
    transform string
    sort bool
    clear bool
    addOpts []string
    removeOpts []string
    clearOpt bool
    }
    1. What content to process
    2. Where and what to output
    3. Which struct to modify
    4. Which tags to modify

    View Slide

  112. package main
    type Example struct {
    Foo string
    } $ gomodifytags
    -file example.go
    -struct Example
    -add-tags json

    View Slide

  113. package main
    type Example struct {
    Foo string
    }
    parse
    gomodifytags

    View Slide

  114. Parse content

    View Slide

  115. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  116. parse
    gomodifytags
    go/parser
    package main
    type Example struct {
    Foo string
    }

    View Slide

  117. parse
    gomodifytags
    go/parser
    go/ast.Node
    package main
    type Example struct {
    Foo string
    }

    View Slide

  118. gomodifytags
    parse
    parse go/parser
    package main
    type Example struct {
    Foo string
    }
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    tag

    View Slide

  119. func (c *config) parse() (ast.Node, error) {
    c.fset = token.NewFileSet()
    var contents interface{}
    if c.modified != nil {
    archive, err := buildutil.ParseOverlayArchive(c.modified)
    if err != nil {
    return nil, fmt.Errorf("failed to parse -modified archive: %v", err)
    }
    fc, ok := archive[c.file]
    if !ok {
    return nil, fmt.Errorf("couldn't find %s in archive", c.file)
    }
    contents = fc
    }
    return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments)
    }
    Parse content

    View Slide

  120. func (c *config) parse() (ast.Node, error) {
    c.fset = token.NewFileSet()
    var contents interface{}
    if c.modified != nil {
    archive, err := buildutil.ParseOverlayArchive(c.modified)
    if err != nil {
    return nil, fmt.Errorf("failed to parse -modified archive: %v", err)
    }
    fc, ok := archive[c.file]
    if !ok {
    return nil, fmt.Errorf("couldn't find %s in archive", c.file)
    }
    contents = fc
    }
    return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments)
    }
    Parse content (cont.)

    View Slide

  121. package main
    type Example struct {
    Foo string
    }
    parse
    gomodifytags
    parse

    View Slide

  122. package main
    type Example struct {
    Foo string
    }
    parse find
    gomodifytags

    View Slide

  123. Find start and end positions

    View Slide

  124. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  125. gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    tag
    this is our file
    ast.Node

    View Slide

  126. parse find
    gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    which is an AST tree

    View Slide

  127. parse find
    gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    select struct that matches
    our criteria ...

    View Slide

  128. There are many ways to
    find a struct

    View Slide

  129. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15

    View Slide

  130. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15

    View Slide

  131. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    struct name:
    Server

    View Slide

  132. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    struct name:
    Server
    start line: 8
    end line: 10

    View Slide

  133. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port| int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    offset byte:
    96 (of 163)

    View Slide

  134. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port| int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    offset byte:
    96 (of 163)
    start line: 8
    end line: 10

    View Slide

  135. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    start line: 8
    end line: 10
    .. or specify explicit lines

    View Slide

  136. package main
    type Example struct {
    Foo string
    }
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    }
    type Person struct {
    Name string
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    start line: 9
    end line: 9
    .. or specify explicit lines

    View Slide

  137. parse find
    gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    select struct that matches
    our criteria ...

    View Slide

  138. gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    ... and get the
    start and end lines
    start line: 8
    end line: 10

    View Slide

  139. func (c *config) findSelection(node ast.Node) (int, int, error) {
    if c.line != "" {
    return c.lineSelection(node)
    } else if c.offset != 0 {
    return c.offsetSelection(node)
    } else if c.structName != "" {
    return c.structSelection(node)
    }
    return 0, 0, errors.New("-line, -offset or -struct is not passed")
    }
    Find selection

    View Slide

  140. func (c *config) findSelection(node ast.Node) (int, int, error) {
    if c.line != "" {
    return c.lineSelection(node)
    } else if c.offset != 0 {
    return c.offsetSelection(node)
    } else if c.structName != "" {
    return c.structSelection(node)
    }
    return 0, 0, errors.New("-line, -offset or -struct is not passed")
    }
    Find selection

    View Slide

  141. Collecting structs

    View Slide

  142. // collectStructs collects and maps structType nodes to their positions
    func collectStructs(node ast.Node) map[token.Pos]*structType {
    structs := make(map[token.Pos]*structType, 0)
    collectStructs := func(n ast.Node) bool {
    t, ok := n.(*ast.TypeSpec)
    if !ok {
    return true
    }
    structName := t.Name.Name
    x, ok := t.Type.(*ast.StructType)
    if !ok {
    return true
    }
    structs[x.Pos()] = &structType{
    name: structName,
    node: x,
    }
    return true
    }
    ast.Inspect(node, collectStructs)
    return structs
    }

    View Slide

  143. // collectStructs collects and maps structType nodes to their positions
    func collectStructs(node ast.Node) map[token.Pos]*structType {
    structs := make(map[token.Pos]*structType)
    collectStructs := func(n ast.Node) bool {
    t, ok := n.(*ast.TypeSpec)
    if !ok {
    return true
    }
    structName := t.Name.Name
    x, ok := t.Type.(*ast.StructType)
    if !ok {
    return true
    }
    structs[x.Pos()] = &structType{
    name: structName,
    node: x,
    }
    return true
    }
    ast.Inspect(node, collectStructs)
    return structs
    }
    type structType struct {
    name string
    node *ast.StructType
    }

    View Slide

  144. // collectStructs collects and maps structType nodes to their positions
    func collectStructs(node ast.Node) map[token.Pos]*structType {
    structs := make(map[token.Pos]*structType)
    collectStructs := func(n ast.Node) bool {
    t, ok := n.(*ast.TypeSpec)
    if !ok {
    return true
    }
    structName := t.Name.Name
    x, ok := t.Type.(*ast.StructType)
    if !ok {
    return true
    }
    structs[x.Pos()] = &structType{
    name: structName,
    node: x,
    }
    return true
    }
    ast.Inspect(node, collectStructs)
    return structs
    }

    View Slide

  145. // collectStructs collects and maps structType nodes to their positions
    func collectStructs(node ast.Node) map[token.Pos]*structType {
    structs := make(map[token.Pos]*structType)
    collectStructs := func(n ast.Node) bool {
    t, ok := n.(*ast.TypeSpec)
    if !ok {
    return true
    }
    structName := t.Name.Name
    x, ok := t.Type.(*ast.StructType)
    if !ok {
    return true
    }
    structs[x.Pos()] = &structType{
    name: structName,
    node: x,
    }
    return true
    }
    ast.Inspect(node, collectStructs)
    return structs
    }
    type structType struct {
    name string
    node *ast.StructType
    }

    View Slide

  146. // collectStructs collects and maps structType nodes to their positions
    func collectStructs(node ast.Node) map[token.Pos]*structType {
    structs := make(map[token.Pos]*structType)
    collectStructs := func(n ast.Node) bool {
    t, ok := n.(*ast.TypeSpec)
    if !ok {
    return true
    }
    structName := t.Name.Name
    x, ok := t.Type.(*ast.StructType)
    if !ok {
    return true
    }
    structs[x.Pos()] = &structType{
    name: structName,
    node: x,
    }
    return true
    }
    ast.Inspect(node, collectStructs)
    return structs
    }

    View Slide

  147. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    if st.name == c.structName {
    encStruct = st.node
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    }
    Struct selection

    View Slide

  148. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    if st.name == c.structName {
    encStruct = st.node
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    }
    Struct selection

    View Slide

  149. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    if st.name == c.structName {
    encStruct = st.node
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    }
    Struct selection

    View Slide

  150. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    if st.name == c.structName {
    encStruct = st.node
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    }
    Struct selection

    View Slide

  151. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    structBegin := c.fset.Position(st.node.Pos()).Offset
    structEnd := c.fset.Position(st.node.End()).Offset
    if structBegin <= c.offset && c.offset <= structEnd {
    encStruct = st.node
    break
    }
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    Offset selection

    View Slide

  152. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    structBegin := c.fset.Position(st.node.Pos()).Offset
    structEnd := c.fset.Position(st.node.End()).Offset
    if structBegin <= c.offset && c.offset <= structEnd {
    encStruct = st.node
    break
    }
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    Offset selection

    View Slide

  153. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    structBegin := c.fset.Position(st.node.Pos()).Offset
    structEnd := c.fset.Position(st.node.End()).Offset
    if structBegin <= c.offset && c.offset <= structEnd {
    encStruct = st.node
    break
    }
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    Offset selection

    View Slide

  154. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    structBegin := c.fset.Position(st.node.Pos()).Offset
    structEnd := c.fset.Position(st.node.End()).Offset
    if structBegin <= c.offset && c.offset <= structEnd {
    encStruct = st.node
    break
    }
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    Offset selection

    View Slide

  155. var encStruct *ast.StructType
    for _, st := range collectStructs() {
    structBegin := c.fset.Position(st.node.Pos()).Offset
    structEnd := c.fset.Position(st.node.End()).Offset
    if structBegin <= c.offset && c.offset <= structEnd {
    encStruct = st.node
    break
    }
    }
    start = c.fset.Position(encStruct.Pos()).Line
    end = c.fset.Position(encStruct.End()).Line
    Offset selection

    View Slide

  156. package main
    type Example struct {
    Foo string
    }
    parse find
    gomodifytags
    find

    View Slide

  157. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    gomodifytags

    View Slide

  158. Rewrite the struct tag

    View Slide

  159. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  160. gomodifytags
    start and end lines
    we're going to modify
    start line: 8
    end line: 10

    View Slide

  161. gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    our parsed AST
    start line: 8
    end line: 10

    View Slide

  162. gomodifytags
    *ast.TypeSpec
    *ast.StructType
    *ast.FieldList *ast.Field
    []*ast.Ident
    *ast.Ident
    ast.BasicLit
    struct we're
    going to modify
    start line: 8
    end line: 10
    *ast.StructType

    View Slide

  163. gomodifytags
    *ast.StructType
    *ast.FieldList

    View Slide

  164. gomodifytags
    *ast.StructType
    6
    7
    8
    9
    10
    11
    12
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field

    View Slide

  165. gomodifytags
    *ast.StructType
    *ast.Field
    []*ast.Ident
    *ast.Ident
    *ast.BasicLit

    View Slide

  166. type Example struct {
    Foo string `json:"foo"`
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    []*ast.Ident
    *ast.Ident
    *ast.BasicLit

    View Slide

  167. type Example struct {
    Foo string `json:"foo"
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    Foo
    string
    `json:"foo"`

    View Slide

  168. type Example struct {
    Foo string `json:"foo"`
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    Foo
    string
    `json:"foo"`

    View Slide

  169. type Example struct {
    Foo string `json:"bar"
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    Foo
    string
    `json:"foo"
    `json:"bar"`
    type Example struct {
    Foo string `json:"foo"`
    }

    View Slide

  170. How do we rewrite a
    struct tag?

    View Slide

  171. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    // found field!
    field.Tag.Value = `json:"bar"`
    }
    return false
    })
    After finding a field,
    replace the value

    View Slide

  172. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    // found field!
    field.Tag.Value = `json:"bar"`
    }
    return false
    })
    Tag: `json:"foo"`
    Tag: `json:"bar"`
    After finding a field,
    replace the value

    View Slide

  173. type Example struct {
    DiskSize string
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    DiskSize
    string

    View Slide

  174. type Example struct {
    DiskSize string
    }
    gomodifytags
    *ast.StructType
    *ast.Field
    DiskSize
    string

    View Slide

  175. gomodifytags
    *ast.StructType
    *ast.Field
    DiskSize
    string
    json:"disk_size"
    process()
    type Example struct {
    DiskSize string
    }
    type Example struct {
    DiskSize string `json:"disk_size"`
    }

    View Slide

  176. src := `package main
    type Example struct {
    Foo string` + " `json:\"foo\"` }"
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments)
    if err != nil {
    panic(err)
    }
    ast.Inspect(file, func(x ast.Node) bool {
    s, ok := x.(*ast.StructType)
    if !ok {
    return true
    }
    for _, field := range s.Fields.List {
    // found field!
    field.Tag.Value = process(field)
    }
    return false
    })
    tags = c.removeTags(tags)
    tags, err = c.removeTagOptions(tags)
    if err != nil {
    return "", err
    }
    tags = c.clearTags(tags)
    tags = c.clearOptions(tags)
    tags, err = c.addTags(fieldName, tags)
    if err != nil {
    return "", err
    }
    tags, err = c.addTagOptions(tags)
    if err != nil {
    return "", err
    }
    if c.sort {
    sort.Sort(tags)
    }

    View Slide

  177. gomodifytags
    *ast.StructType
    8
    9
    10
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    select field between
    start and end lines ...
    Modify overview

    View Slide

  178. gomodifytags
    *ast.StructType
    8
    9
    10
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    ... and rewrite selected
    fields tags
    8
    9
    10
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    *ast.Field
    rewrite
    Modify overview

    View Slide

  179. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    gomodifytags
    modify

    View Slide

  180. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) {
    var rewriteErr error
    rewriteFunc := func(n ast.Node) bool {
    x, ok := n.(*ast.StructType)
    if !ok {
    return true
    }
    for _, f := range x.Fields.List {
    // process each field
    // ...
    }
    return true
    }
    ast.Inspect(node, rewriteFunc)
    return node, rewriteErr
    }

    View Slide

  181. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) {
    var rewriteErr error
    rewriteFunc := func(n ast.Node) bool {
    x, ok := n.(*ast.StructType)
    if !ok {
    return true
    }
    for _, f := range x.Fields.List {
    // process each field
    // ...
    }
    return true
    }
    ast.Inspect(node, rewriteFunc)
    return node, rewriteErr
    }

    View Slide

  182. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) {
    var rewriteErr error
    rewriteFunc := func(n ast.Node) bool {
    x, ok := n.(*ast.StructType)
    if !ok {
    return true
    }
    for _, f := range x.Fields.List {
    // process each field
    // ...
    }
    return true
    }
    ast.Inspect(node, rewriteFunc)
    return node, rewriteErr
    }

    View Slide

  183. for _, f := range x.Fields.List {
    line := c.fset.Position(f.Pos()).Line
    if !(c.start <= line && line <= c.end) {
    continue
    }
    if f.Tag == nil {
    f.Tag = &ast.BasicLit{}
    }
    // ...
    }

    View Slide

  184. for _, f := range x.Fields.List {
    line := c.fset.Position(f.Pos()).Line
    if !(c.start <= line && line <= c.end) {
    continue
    }
    if f.Tag == nil {
    f.Tag = &ast.BasicLit{}
    }
    // ...
    }

    View Slide

  185. for _, f := range x.Fields.List {
    line := c.fset.Position(f.Pos()).Line
    if !(c.start <= line && line <= c.end) {
    continue
    }
    if f.Tag == nil {
    f.Tag = &ast.BasicLit{}
    }
    // ...
    }

    View Slide

  186. for _, f := range x.Fields.List {
    fieldName := ""
    if len(f.Names) != 0 {
    fieldName = f.Names[0].Name
    }
    if f.Names == nil {
    ident, ok := f.Type.(*ast.Ident)
    if !ok {
    continue // anonymous field
    }
    fieldName = ident.Name
    }
    res, err := c.process(fieldName, f.Tag.Value)
    if err != nil {
    rewriteErr = err
    return true
    }
    f.Tag.Value = res
    }

    View Slide

  187. for _, f := range x.Fields.List {
    fieldName := ""
    if len(f.Names) != 0 {
    fieldName = f.Names[0].Name
    }
    if f.Names == nil {
    ident, ok := f.Type.(*ast.Ident)
    if !ok {
    continue // anonymous field
    }
    fieldName = ident.Name
    }
    res, err := c.process(fieldName, f.Tag.Value)
    if err != nil {
    rewriteErr = err
    return true
    }
    f.Tag.Value = res
    }

    View Slide

  188. for _, f := range x.Fields.List {
    fieldName := ""
    if len(f.Names) != 0 {
    fieldName = f.Names[0].Name
    }
    if f.Names == nil {
    ident, ok := f.Type.(*ast.Ident)
    if !ok {
    continue // anonymous field
    }
    fieldName = ident.Name
    }
    res, err := c.process(fieldName, f.Tag.Value)
    if err != nil {
    rewriteErr = err
    return true
    }
    f.Tag.Value = res
    }

    View Slide

  189. Processing the tags

    View Slide

  190. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }

    View Slide

  191. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }

    View Slide

  192. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }

    View Slide

  193. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }
    tags = c.removeTags(tags)
    tags, err = c.removeTagOptions(tags)
    if err != nil {
    return "", err
    }
    tags = c.clearTags(tags)
    tags = c.clearOptions(tags)
    tags, err = c.addTags(fieldName, tags)
    if err != nil {
    return "", err
    }
    tags, err = c.addTagOptions(tags)
    if err != nil {
    return "", err
    }
    if c.sort {
    sort.Sort(tags)
    }

    View Slide

  194. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }
    tags = c.removeTags(tags)
    tags, err = c.removeTagOptions(tags)
    if err != nil {
    return "", err
    }
    tags = c.clearTags(tags)
    tags = c.clearOptions(tags)
    tags, err = c.addTags(fieldName, tags)
    if err != nil {
    return "", err
    }
    tags, err = c.addTagOptions(tags)
    if err != nil {
    return "", err
    }
    if c.sort {
    sort.Sort(tags)
    }

    View Slide

  195. func (c *config) process(fieldName, tagVal string) (string, error) {
    var tag string
    if tagVal != "" {
    var err error
    tag, err = strconv.Unquote(tagVal)
    if err != nil {
    return "", err
    }
    }
    tags, err := structtag.Parse(tag)
    if err != nil {
    return "", err
    }
    // process tags ...
    res := tags.String()
    if res != "" {
    res = quote(tags.String())
    }
    return res, nil
    }

    View Slide

  196. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) {
    var rewriteErr error
    rewriteFunc := func(n ast.Node) bool {
    x, ok := n.(*ast.StructType)
    if !ok {
    return true
    }
    for _, f := range x.Fields.List {
    // process each field
    // ...
    }
    return true
    }
    ast.Inspect(node, rewriteFunc)
    return node, rewriteErr
    }

    View Slide

  197. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    format
    gomodifytags

    View Slide

  198. Output the result

    View Slide

  199. func main() {
    var cfg config
    node = cfg.parse()
    start, end = cfg.findSelection(node)
    rewritten = cfg.rewrite(node, start, end)
    out = cfg.format(rewritten)
    fmt.Println(out)
    }
    1. Fetch configuration settings
    2. Parse content
    3. Find selection
    4. Modify the struct tag
    5. Output the result

    View Slide

  200. gomodifytags
    ast.Node (modified)

    View Slide

  201. gomodifytags
    ast.Node (modified)
    go/format
    Pass the modified node
    to go/format

    View Slide

  202. gomodifytags
    ast.Node (modified)
    go/format
    ... and output the result
    to stdout
    package main
    type Example struct {
    Foo string `json:"foo"`
    }

    View Slide

  203. package main
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    BaseDomain string
    Credentials struct {
    Username string
    Password string
    }
    }
    package main
    type Server struct {
    Name string `json:"name"`
    Port int `json:"port"`
    EnableLogs bool `json:"enable_logs"`
    BaseDomain string `json:"base_domain"`
    Credentials struct {
    Username string `json:"username"`
    Password string `json:"password"`
    } `json:"credentials"`
    }
    Input Output
    $ gomodifytags
    -file example.go
    -struct Server
    -add-tags json

    View Slide

  204. gomodifytags
    ast.Node (modified)
    go/format
    Also has support for
    custom JSON output
    {
    "start": 3,
    "end": 5,
    "lines": [
    "type Example struct {",
    " Foo string `json:\"foo\"`",
    "}"
    ]
    }

    View Slide

  205. package main
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    BaseDomain string
    Credentials struct {
    Username string
    Password string
    }
    }
    {
    "start": 3,
    "end": 12,
    "lines": [
    "type Server struct {",
    " Name string `xml:\"name\"`",
    " Port int `xml:\"port\"`",
    " EnableLogs bool `xml:\"enable_logs\"`",
    " BaseDomain string `xml:\"base_domain\"`",
    " Credentials struct {",
    " Username string `xml:\"username\"`",
    " Password string `xml:\"password\"`",
    " } `xml:\"credentials\"`",
    "}"
    ]
    }
    Input
    JSON Output
    $ gomodifytags
    -file example.go
    -struct Server
    -add-tags json
    -format json

    View Slide

  206. Why stdout?
    • by default, it means "dry-run"
    • immediate feedback
    • tools can redirect the output
    • composable (gomodifytags myfile.go > newfile.go)

    View Slide

  207. func (c *config) format(file ast.Node) (string, error) {
    switch c.output {
    case "source":
    // return formatted source
    case "json":
    // return only changes in json
    default:
    return "", fmt.Errorf("unknown output mode: %s", c.output)
    }
    }
    Output the result

    View Slide

  208. func (c *config) format(file ast.Node) (string, error) {
    switch c.output {
    case "source":
    // return formatted source
    case "json":
    // return only changes in json
    default:
    return "", fmt.Errorf("unknown output mode: %s", c.output)
    }
    }
    Output the result

    View Slide

  209. var buf bytes.Buffer
    err := format.Node(&buf, c.fset, file)
    if err != nil {
    return "", err
    }
    if c.write {
    err = ioutil.WriteFile(c.file, buf.Bytes(), 0)
    if err != nil {
    return "", err
    }
    }
    return buf.String(), nil
    Source file

    View Slide

  210. var buf bytes.Buffer
    err := format.Node(&buf, c.fset, file)
    if err != nil {
    return "", err
    }
    if c.write {
    err = ioutil.WriteFile(c.file, buf.Bytes(), 0)
    if err != nil {
    return "", err
    }
    }
    return buf.String(), nil
    Source file

    View Slide

  211. var buf bytes.Buffer
    err := format.Node(&buf, c.fset, file)
    if err != nil {
    return "", err
    }
    if c.write {
    err = ioutil.WriteFile(c.file, buf.Bytes(), 0)
    if err != nil {
    return "", err
    }
    }
    return buf.String(), nil
    Source file

    View Slide

  212. var buf bytes.Buffer
    err := format.Node(&buf, c.fset, file)
    if err != nil {
    return "", err
    }
    var lines []string
    scanner := bufio.NewScanner(bytes.NewBufferString(buf.String()))
    for scanner.Scan() {
    lines = append(lines, scanner.Text())
    }
    if c.start > len(lines) {
    return "", errors.New("line selection is invalid")
    }
    ...
    JSON output

    View Slide

  213. var buf bytes.Buffer
    err := format.Node(&buf, c.fset, file)
    if err != nil {
    return "", err
    }
    var lines []string
    scanner := bufio.NewScanner(bytes.NewBufferString(buf.String()))
    for scanner.Scan() {
    lines = append(lines, scanner.Text())
    }
    if c.start > len(lines) {
    return "", errors.New("line selection is invalid")
    }
    ...
    JSON output

    View Slide

  214. out := &output{
    Start: c.start,
    End: c.end,
    Lines: lines[c.start-1 : c.end],
    }
    o, err := json.MarshalIndent(out, "", " ")
    if err != nil {
    return "", err
    }
    return string(o), nil
    JSON output (cont.)

    View Slide

  215. out := &output{
    Start: c.start,
    End: c.end,
    Lines: lines[c.start-1 : c.end],
    }
    o, err := json.MarshalIndent(out, "", " ")
    if err != nil {
    return "", err
    }
    return string(o), nil
    JSON output (cont.)

    View Slide

  216. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    gomodifytags
    format
    format

    View Slide

  217. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    gomodifytags
    format
    Overview

    View Slide

  218. Overview

    View Slide

  219. gomodifytags

    View Slide

  220. gomodifytags
    package main
    type Example struct {
    Foo string
    }

    View Slide

  221. package main
    type Example struct {
    Foo string
    }
    -file example.go
    -struct Example
    -add-tags json
    gomodifytags

    View Slide

  222. package main
    type Example struct {
    Foo string
    }
    parse
    gomodifytags

    View Slide

  223. package main
    type Example struct {
    Foo string
    }
    parse find
    gomodifytags

    View Slide

  224. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    gomodifytags

    View Slide

  225. package main
    type Example struct {
    Foo string
    }
    parse find
    modify
    format
    gomodifytags

    View Slide

  226. package main
    type Example struct {
    Foo string
    }
    package main
    type Example struct {
    Foo string `json:"foo"`
    }
    parse find
    modify
    format
    gomodifytags

    View Slide

  227. Demo time

    View Slide

  228. Built from the ground up for editors
    • selecting structs based on offset or range of lines
    • write result to stdout or file
    • json output for easy parsing
    • support for unsaved buffers
    • individual cli flags for all features

    View Slide

  229. Supported editors
    • vim with vim-go
    • atom with go-plus (thanks Zac Bergquist)
    • vscode with vscode-go (thanks Ramya Rao)
    • emacs - WIP (https://github.com/syohex/emacs-go-add-tags/issues/
    6)

    View Slide

  230. Fixed bugs

    View Slide

  231. Recap
    • A single tool to rule all editors
    • Easy maintenance
    • Free of bugs due stdlib tooling (go/parser family)
    • Happy and productive users

    View Slide

  232. Thanks!
    Fatih Arslan
    @fatih
    @fatih
    [email protected]

    View Slide