Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bazel loves Go

Yuki Ito
December 06, 2019

Bazel loves Go

Yuki Ito

December 06, 2019
Tweet

More Decks by Yuki Ito

Other Decks in Technology

Transcript

  1. 1
    Yuki ITO
    mercari.go #12
    Bazel Go

    View full-size slide

  2. 2
    ʮGoͷ৽ػೳΛ࢖ͬͨDBεΩʔϚ؅ཧπʔϧʯ

    View full-size slide

  3. 3
    https://github.com/mercari/wrench
    Schema Management Tool
    for
    Cloud Spanner

    View full-size slide

  4. 4
    https://speakerdeck.com/micnncim/accelerate-go-development-with-bazel

    View full-size slide

  5. Agenda
    5
    ɾWhat is Bazel
    ɾHow to use for Go
    ɾTips

    View full-size slide

  6. Agenda
    6
    ɾWhat is Bazel
    ɾHow to use for Go
    ɾTips

    View full-size slide

  7. What is Bazel
    7
    Bazel is an open-source build and test tool
    similar to Make, Maven, and Gradle.
    https://docs.bazel.build/versions/master/bazel-overview.html

    View full-size slide

  8. What is Bazel
    8
    One tool, multiple languages/platforms
    ...

    View full-size slide

  9. What is Bazel
    9
    High-level build language
    Starlark (dialect of Python)
    Starlark is a language intended for use as a
    configuration language. It was designed for
    the Bazel build system, but may be useful for
    other projects as well.
    https://github.com/bazelbuild/starlark

    View full-size slide

  10. What is Bazel
    10
    Starlark (dialect of Python)
    def fizz_buzz(n):
    """Print Fizz Buzz numbers from 1 to n."""
    for i in range(1, n + 1):
    s = ""
    if i % 3 == 0:
    s += "Fizz"
    if i % 5 == 0:
    s += "Buzz"
    print(s if s else i)
    fizz_buzz(20)

    View full-size slide

  11. What is Bazel
    11
    Extensible
    load(
    "@bazel_tools//tools/build_defs/repo:http.bzl",
    "http_archive",
    )
    http_archive(
    name = "io_bazel_rules_go",
    urls = ["..."],
    )

    View full-size slide

  12. Agenda
    12
    ɾWhat is Bazel
    ɾHow to use for Go
    ɾTips

    View full-size slide

  13. WORKSPACE & BUILD
    13
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...

    View full-size slide

  14. WORKSPACE & BUILD
    14
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...
    WORKSPACE

    View full-size slide

  15. WORKSPACE & BUILD
    15
    The WORKSPACE file, which identifies the directory
    and its contents as a Bazel workspace and lives at
    the root of the project’s directory structure.
    WORKSPACE

    View full-size slide

  16. 16
    http_archive(
    name = "io_bazel_rules_go",
    urls = ["..."],
    )
    load(
    "@io_bazel_rules_go//go:deps.bzl",
    "go_register_toolchains",
    "go_rules_dependencies",
    )
    go_rules_dependencies()
    go_register_toolchains(
    go_version = "1.13.4",
    nogo = "@//:nogo",
    )

    View full-size slide

  17. 17
    http_archive(
    name = "io_bazel_rules_go",
    urls = ["..."],
    )
    load(
    "@io_bazel_rules_go//go:deps.bzl",
    "go_register_toolchains",
    "go_rules_dependencies",
    )
    go_rules_dependencies()
    go_register_toolchains(
    go_version = "1.13.4",
    nogo = "@//:nogo",
    )

    View full-size slide

  18. 18
    go_repository(
    name = "co_honnef_go_tools",
    importpath = "honnef.co/go/tools",
    sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=",
    version = "v0.0.0-20190523083050-ea95bdfd59fc",
    )
    go_repository(
    name = "com_github_burntsushi_toml",
    importpath = "github.com/BurntSushi/toml",
    sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=",
    version = "v0.3.1",
    )
    go_repository(
    name = "com_github_burntsushi_xgb",
    importpath = "github.com/BurntSushi/xgb",
    sum = "h1:1BDTz0u9nC3//pOCMdNH+CiXJVYJh5UQNCOBG7jbELc=",
    version = "v0.0.0-20160522181843-27f122750802",
    )
    ...

    View full-size slide

  19. 19
    https://github.com/bazelbuild/rules_go
    Extension for Go

    View full-size slide

  20. WORKSPACE & BUILD
    20
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...
    BUILD

    View full-size slide

  21. WORKSPACE & BUILD
    21
    One or more BUILD files, which tell Bazel how to
    build different parts of the project.
    BUILD

    View full-size slide

  22. 22
    go_library(
    name = "go_default_library",
    srcs = ["main.go"],
    importpath = "github.com/mercari/wrench",
    visibility = ["//visibility:private"],
    deps = [
    "//cmd:go_default_library",
    "//pkg/spanner:go_default_library",
    ],
    )
    go_binary(
    name = "wrench",
    embed = [":go_default_library"],
    visibility = ["//visibility:public"],
    )

    View full-size slide

  23. 23
    go_library(
    name = "go_default_library",
    srcs = [
    "apply.go",
    ...
    ],
    importpath = "github.com/mercari/wrench/cmd",
    visibility = ["//visibility:public"],
    deps = [
    "//pkg/spanner:go_default_library",
    "@com_github_spf13_cobra//:go_default_library",
    ],
    )
    go_test(
    name = "go_default_test",
    srcs = [
    "export_test.go",
    "migrate_test.go",
    ],
    data = glob(["testdata/**"]),
    embed = [":go_default_library"],
    )

    View full-size slide

  24. WORKSPACE & BUILD
    24
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...
    Time consuming to write manually...

    View full-size slide

  25. WORKSPACE & BUILD
    25

    go_repository(
    name = "co_honnef_go_tools",
    importpath = "honnef.co/go/tools",
    sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=",
    version = "v0.0.0-20190523083050-ea95bdfd59fc",
    )
    go_repository(
    name = "com_github_burntsushi_toml",
    importpath = "github.com/BurntSushi/toml",
    sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=",
    version = "v0.3.1",
    )
    go_repository(
    name = "com_github_burntsushi_xgb",
    importpath = "github.com/BurntSushi/xgb",
    sum = "h1:1BDTz0u9nC3//pOCMdNH+CiXJVYJh5UQNCOBG7jbELc=",
    version = "v0.0.0-20160522181843-27f122750802",
    )
    ...
    Time consuming to write manually...

    View full-size slide

  26. 26
    https://github.com/bazelbuild/bazel-gazelle
    gazelle
    ɾGenerates BUILD files for all packages
    ɾGenerates all go_repository instructions

    View full-size slide

  27. 27
    https://github.com/bazelbuild/bazel-gazelle
    gazelle
    ɾGenerates BUILD files for all packages
    ɾGenerates all go_repository instructions

    View full-size slide

  28. 28
    gazelle
    > bazel run //:gazelle

    View full-size slide

  29. WORKSPACE & BUILD
    29
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...
    BUILD

    View full-size slide

  30. 30
    https://github.com/bazelbuild/bazel-gazelle
    gazelle
    ɾGenerates BUILD files for all packages
    ɾGenerates all go_repository instructions

    View full-size slide

  31. 31
    gazelle
    > bazel run //:gazelle -- \
    update-repos \
    -from_file go.mod

    View full-size slide

  32. 32
    go_repository(
    name = "co_honnef_go_tools",
    importpath = "honnef.co/go/tools",
    sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=",
    version = "v0.0.0-20190523083050-ea95bdfd59fc",
    )
    go_repository(
    name = "com_github_burntsushi_toml",
    importpath = "github.com/BurntSushi/toml",
    sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=",
    version = "v0.3.1",
    )
    go_repository(
    name = "com_github_burntsushi_xgb",
    importpath = "github.com/BurntSushi/xgb",
    sum = "h1:1BDTz0u9nC3//pOCMdNH+CiXJVYJh5UQNCOBG7jbELc=",
    version = "v0.0.0-20160522181843-27f122750802",
    )
    ...

    View full-size slide

  33. WORKSPACE & BUILD
    33
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    ᵓ── pkg
    │ └── spanner
    │ ᵓ── BUILD.bazel
    │ ᵓ── client.go
    │ └── ...
    └── ...

    View full-size slide

  34. 34
    > bazel build //:wrench

    View full-size slide

  35. 35
    go_library(
    name = "go_default_library",
    srcs = ["main.go"],
    importpath = "github.com/mercari/wrench",
    visibility = ["//visibility:private"],
    deps = [
    "//cmd:go_default_library",
    "//pkg/spanner:go_default_library",
    ],
    )
    go_binary(
    name = "wrench",
    embed = [":go_default_library"],
    visibility = ["//visibility:public"],
    )

    View full-size slide

  36. 36
    > ./bazel-bin/darwin_amd64_stripped/wrench help
    Usage:
    wrench [command]
    ...

    View full-size slide

  37. Bake Docker Image
    37
    https://github.com/bazelbuild/rules_docker

    View full-size slide

  38. Bake Docker Image
    38

    View full-size slide

  39. Bake Docker Image
    39
    load(
    "@io_bazel_rules_docker//go:image.bzl",
    "go_image",
    )
    go_image(
    name = "image",
    embed = [":go_default_library"],
    )

    View full-size slide

  40. Bake Docker Image
    40
    bazel build \
    --platforms=\
    @io_bazel_rules_go//go/toolchain:linux_amd64 \
    //:image

    View full-size slide

  41. Bake Docker Image
    41
    > ./bazel-bin/image help
    Loaded image ID: sha256:xxx
    Tagging xxx as bazel:image
    Usage:
    wrench [command]
    ...

    View full-size slide

  42. Bake Docker Image
    42
    Dockerfile LESS

    View full-size slide

  43. Bake Docker Image
    43
    Push to Registry
    load(
    "@io_bazel_rules_docker//container:container.bzl",
    "container_push",
    )
    container_push(
    name = "registry",
    format = "Docker",
    image = ":image",
    registry = "index.docker.io",
    repository = "mercari/wrench",
    tag = version,
    )

    View full-size slide

  44. Bake Docker Image
    44

    View full-size slide

  45. Agenda
    45
    ɾWhat is Bazel
    ɾHow to use for Go
    ɾTips

    View full-size slide

  46. Tips
    46
    ɾExtract dependencies
    ɾnogo
    ɾRemote Caching

    View full-size slide

  47. Tips
    47
    ɾExtract dependencies
    ɾnogo
    ɾRemote Caching

    View full-size slide

  48. 48
    go_repository(
    name = "co_honnef_go_tools",
    importpath = "honnef.co/go/tools",
    sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=",
    version = "v0.0.0-20190523083050-ea95bdfd59fc",
    )
    go_repository(
    name = "com_github_burntsushi_toml",
    importpath = "github.com/BurntSushi/toml",
    sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=",
    version = "v0.3.1",
    )
    go_repository(
    name = "com_github_burntsushi_xgb",
    importpath = "github.com/BurntSushi/xgb",
    sum = "h1:1BDTz0u9nC3//pOCMdNH+CiXJVYJh5UQNCOBG7jbELc=",
    version = "v0.0.0-20160522181843-27f122750802",
    )
    ...
    Extract Dependencies
    WORKSPACE

    View full-size slide

  49. 49
    Extract Dependencies
    > bazel run //:gazelle -- \
    update-repos \
    -from_file go.mod \
    -to_macro bazel/deps.bzl%wrench_deps

    View full-size slide

  50. 50
    ᵓ── BUILD.bazel
    ᵓ── WORKSPACE
    ᵓ── bazel
    │ ᵓ── deps.bzl
    ᵓ── cmd
    │ ᵓ── BUILD.bazel
    │ ᵓ── apply.go
    │ └── ...
    ᵓ── main.go
    └── ...
    BUILD
    Extract Dependencies

    View full-size slide

  51. 51
    load("@bazel_gazelle//:deps.bzl", "go_repository")
    def wrench_deps():
    go_repository(
    name = "co_honnef_go_tools",
    importpath = "honnef.co/go/tools",
    sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=",
    version = "v0.0.0-20190523083050-ea95bdfd59fc",
    )
    go_repository(
    name = "com_github_burntsushi_toml",
    importpath = "github.com/BurntSushi/toml",
    sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=",
    version = "v0.3.1",
    )
    Extract Dependencies

    View full-size slide

  52. Tips
    52
    ɾExtract dependencies
    ɾnogo
    ɾRemote Caching

    View full-size slide

  53. nogo (experimental)
    53
    nogo is a tool that analyzes the source code of Go
    programs. It runs alongside the Go compiler in the
    Bazel Go rules and rejects programs that contain
    disallowed coding patterns. In addition, nogo may
    report compiler-like errors.
    https://github.com/bazelbuild/rules_go/blob/master/go/nogo.rst

    View full-size slide

  54. nogo (experimental)
    54
    nogo(
    name = "nogo",
    visibility = ["//visibility:public"],
    deps = [
    "@org_golang_x_tools//go/analysis/passes/atomic:go_tool_library",
    "@org_golang_x_tools//go/analysis/passes/bools:go_tool_library",
    "@org_golang_x_tools//go/analysis/passes/buildtag:go_tool_library",
    "@org_golang_x_tools//go/analysis/passes/nilfunc:go_tool_library",
    "@org_golang_x_tools//go/analysis/passes/printf:go_tool_library",
    "@org_golang_x_tools//go/analysis/passes/structtag:go_tool_library",
    ],
    )

    View full-size slide

  55. Tips
    55
    ɾExtract dependencies
    ɾnogo
    ɾRemote Caching

    View full-size slide

  56. Remote Caching
    56
    #

    Developers CI
    Cache

    View full-size slide

  57. Remote Caching
    57
    ɾNginx
    ɾbazel-remote
    ɾGoogle Cloud Storage

    View full-size slide

  58. Remote Caching
    58
    bazel-remote
    bazel-remote is a HTTP/1.1 and gRPC server that is
    intended to be used as a remote build cache
    for Bazel. The cache contents are stored in a
    directory on disk.

    View full-size slide

  59. Remote Caching
    59
    Google Cloud Storage
    #
    Developers CI
    Cache

    View full-size slide

  60. Remote Caching
    60
    Google Cloud Storage
    resource "google_storage_bucket" "bazel-cache" {
    name = "my-bazel-remote-cache"
    location = "asia-northeast1"
    storage_class = "STANDARD"
    }

    View full-size slide

  61. Remote Caching
    61
    Google Cloud Storage
    resource "google_storage_bucket" "bazel-cache" {
    name = "my-bazel-remote-cache"
    location = "asia-northeast1"
    storage_class = "STANDARD"
    }
    ᶃ Create GCS Bucket

    View full-size slide

  62. Remote Caching
    62
    Google Cloud Storage
    ᶄ Specify Remote Bucket
    bazel build \
    --google_credentials= \
    --remote_http_cache=https://storage.googleapis.com/ \
    //:image

    View full-size slide

  63. Agenda
    63
    ɾWhat is Bazel
    ɾHow to use for Go
    ɾTips

    View full-size slide