Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DatastoreからSpannerに 移行したいぞ途中編 / migrate Datastore to Spanner in progress

DatastoreからSpannerに 移行したいぞ途中編 / migrate Datastore to Spanner in progress

GCPUG Tokyo Spanner Day August 2021
https://gcpug-tokyo.connpass.com/event/220499/

DatastoreからSpannerに 移行したいぞ途中編 という話

Masahiro Wakame

August 06, 2021
Tweet

More Decks by Masahiro Wakame

Other Decks in Programming

Transcript

  1. Datastore͔ΒSpannerʹ


    Ҡߦ్͍ͨͧ͠தฤ
    GCPUG Tokyo Spanner Day August 2021
    Θ͔Ί ·͞ͻΖ

    View Slide

  2. Θ͔Ί ·͞ͻΖ @v vakame
    Masahiro Wakame
    GAE/Go
    GraphQL
    ٕज़ॻయ

    View Slide

  3. ຊ೔ͷςʔϚ
    •ٕज़ॻయWebΛҠߦ͍ͨ͠ʂ
    •Datastore→Spanner ࡞ۀதͰ͢
    •ݱঢ়ͷ஌ݟ
    ͍ͭ׬ྃ͢Δ͔͸Θ͔ΒΜ…

    View Slide

  4. Ϟνϕʔγϣϯ
    •UNION ALLͱ͔͔ͨͬͨ͠
    •ձ͕ࣾओʹSpannerͳͷͰ৮Γ͔ͨͬͨ
    •ίετ͕ 1/10 ʹͳͬͨͷͰݱ࣮తʹ
    ΫΤϦͷॊೈ͞ʹظ଴…ʂ

    View Slide

  5. ΞʔΩςΫνϟ1
    •AppEngine/Go 1.1
    5

    •Datastore → Spanne
    r

    •GraphQL (gqlgen)
    GraphQLͷԸܙ͕݁ߏσΧ͍

    View Slide

  6. ΞʔΩςΫνϟ2
    1. API (GraphQL
    )

    2. Service (domain
    )

    3. Repository (DB)
    LayeredͬΆ͍͚ͲશવͦΜͳ͜ͱͳ͍

    View Slide

  7. جຊํ਑
    •Datastore༻RepositoryΛࠩ͠ସ͑
    •؀ڥม਺Ͱ੾Γସ͑
    •Spanner friendlyͳมߋ͸ೖΕͳ͍
    ·ͣ͸΍Γ͖Γ͍ͨ

    View Slide

  8. ਐḿ
    •શRepositoryͷSpanner൛ͷ࡞੒׬
    •Datastore,SpannerͰಉ͡ςετ͕௨ͬͨ
    •؀ڥ੔උ΍σʔλҠߦݕ౼͸͜Ε͔Β
    ંΓฦͨ͠ײ͸͋Δ

    View Slide

  9. Tips
    ຊ೔ͷຊ୊Ͱ͢

    View Slide

  10. ओு
    •OpenCensusͷTrace͸େࣄʂʂʂ

    View Slide

  11. Spanner Emulator
    •ศར CIͰ࢖ͬͯ·͢
    •ReadWriteTransaction͕ෳ਺షΕͳ͍
    •͜ͷ੍໿͕݁ߏΩπ͍ʂ
    •spanner.Client#Apply ΋ྫ֎͡Όͳ͍
    ApplyAtLeastOnce΋ͩΊ

    View Slide

  12. Ͳ͏ͳΔʁ
    •ΤϥʔʹͳΔ
    •Timeout·ͰϦτϥΠ͢Δ
    •ΞϓϦ͔ΒΤϥʔݟ͑ͳ͍
    ී௨ʹ਺༹͚࣌ؒ·͢

    View Slide

  13. ରࡦ
    •Google͸OpenCensus࢖ͬͯΔ
    •https://bit.ly/37qymo
    W

    •Open CensusͷTraceϩάΛݟΔͱ݁ߏ
    ৭ʑग़ͯΔ…ʂʂ
    Spanner Clientͷίʔυࣗମ΋ࢀߟʹͳΓ·͢

    View Slide

  14. ΍ͬͯΈͨ
    Traceࠐͷ࣮ߦ݁Ռ
    2021-08-05T10:42:25.1409165Z === RUN TestSpanne
    r

    2021-08-05T10:42:25.2019180Z 2021/08/05 10:42:25 Duration: 68.2807ms Name: google.spanner.admin.database.v1.DatabaseAdmin.CreateDatabase Code: 0 Message: Attributes: map[Client:true
    FailFast:true] Annotations: [
    ]

    2021-08-05T10:42:25.2030092Z ParentSpanID: 66d0f65276a786dc SpanID: 41a185b1034c5e57 ChildSpanCount:
    0

    2021-08-05T10:42:25.2034069Z INFO: create spanner client with projects/tbf-test/instances/tbf-test-instance/databases/for-ut-a76
    3

    2021-08-05T10:42:25.2035431Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_utils.go:7
    0

    2021-08-05T10:42:25.2037531Z 2021/08/05 10:42:25 Duration: 179.701µs Name: cloud.google.com/go/spanner.NewClient Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2038616Z ParentSpanID: 66d0f65276a786dc SpanID: 1c72141091f035d2 ChildSpanCount:
    0

    2021-08-05T10:42:25.2050051Z 2021/08/05 10:42:25 Duration: 2.842616ms Name: google.spanner.v1.Spanner.BatchCreateSessions Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]

    2021-08-05T10:42:25.2051837Z ParentSpanID: d21332cdab39e5c7 SpanID: ade4c02b39debc42 ChildSpanCount:
    0

    2021-08-05T10:42:25.2054728Z 2021/08/05 10:42:25 Duration: 2.925417ms Name: cloud.google.com/go/spanner.BatchCreateSessions Code: 0 Message: Attributes: map[] Annotations:
    [***Time:2021-08-05 10:42:25.201796419 +0000 UTC m=+157.694579279 Message:Creating a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204703536 +0000 UTC m=+157.697486496
    Message:Received a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204718536 +0000 UTC m=+157.697501396 Message:Finished creating 2 sessions Attributes:map[]***
    ]

    2021-08-05T10:42:25.2056889Z ParentSpanID: 0000000000000000 SpanID: d21332cdab39e5c7 ChildSpanCount:
    1

    2021-08-05T10:42:25.2058866Z 2021/08/05 10:42:25 Duration: 410.303µs Name: google.spanner.v1.Spanner.BeginTransaction Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]

    2021-08-05T10:42:25.2060645Z ParentSpanID: f742a36e1e950d4d SpanID: 88b54f8ac68294bd ChildSpanCount:
    0

    2021-08-05T10:42:25.2061510Z INFO: before ReadRo
    w

    2021-08-05T10:42:25.2062577Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    0

    2021-08-05T10:42:25.2064464Z 2021/08/05 10:42:25 Duration: 9.8µs Name: cloud.google.com/go/spanner.Read Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2065690Z ParentSpanID: f742a36e1e950d4d SpanID: 6386dee853276c38 ChildSpanCount:
    1

    2021-08-05T10:42:25.2067399Z 2021/08/05 10:42:25 Duration: 555.803µs Name: google.spanner.v1.Spanner.StreamingRead Code: 0 Message: Attributes: map[Client:true FailFast:true] Annotations:
    [
    ]

    2021-08-05T10:42:25.2069446Z ParentSpanID: 3e576d47e1cb43b3 SpanID: 1928fca56e701b2e ChildSpanCount:
    0

    2021-08-05T10:42:25.2071376Z 2021/08/05 10:42:25 Duration: 748.605µs Name: cloud.google.com/go/spanner.RowIterator Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2072630Z ParentSpanID: 6386dee853276c38 SpanID: 3e576d47e1cb43b3 ChildSpanCount:
    1

    2021-08-05T10:42:25.2074888Z WARNING: err: spanner: code = "NotFound", desc = "row not found(Table: Organization, PrimaryKey: (\"111\"))
    "

    2021-08-05T10:42:25.2076611Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    9

    2021-08-05T10:42:25.2078426Z INFO: before Apply with ApplyAtLeastOnc
    e

    2021-08-05T10:42:25.2081087Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:4
    9

    2021-08-05T10:42:25.2084536Z 2021/08/05 10:42:25 Duration: 533.803µs Name: google.spanner.v1.Spanner.Commit Code: 10 Message: Transaction 2 aborted due to active transaction 1. The emulator
    only supports one transaction at a time. Some best practices to avoid ABORT errors in Cloud Spanner service are
    :

    2021-08-05T10:42:25.2087265Z 1. Avoid use of nested transactions
    .

    2021-08-05T10:42:25.2088255Z 2. Explicitly Rollback failed transactions
    .

    2021-08-05T10:42:25.2090414Z 3. All transactions should be running inside of retry loops
    .

    2021-08-05T10:42:25.2091644Z Attributes: map[Client:true FailFast:true] Annotations: [
    ]

    2021-08-05T10:42:25.2092587Z ParentSpanID: f4f88a04fc14f3a8 SpanID: cfc9196389b9ca23 ChildSpanCount:
    0

    View Slide

  15. ΍ͬͯΈͨ
    ͦͯ͠Timeout΁…
    2021-08-05T10:42:25.1409165Z === RUN TestSpanne
    r

    2021-08-05T10:42:25.2019180Z 2021/08/05 10:42:25 Duration: 68.2807ms Name: google.spanner.admin.database.v1.DatabaseAdmin.CreateDatabase Code: 0 Message: Attributes: map[Client:true
    FailFast:true] Annotations: [
    ]

    2021-08-05T10:42:25.2030092Z ParentSpanID: 66d0f65276a786dc SpanID: 41a185b1034c5e57 ChildSpanCount:
    0

    2021-08-05T10:42:25.2034069Z INFO: create spanner client with projects/tbf-test/instances/tbf-test-instance/databases/for-ut-a76
    3

    2021-08-05T10:42:25.2035431Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_utils.go:7
    0

    2021-08-05T10:42:25.2037531Z 2021/08/05 10:42:25 Duration: 179.701µs Name: cloud.google.com/go/spanner.NewClient Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2038616Z ParentSpanID: 66d0f65276a786dc SpanID: 1c72141091f035d2 ChildSpanCount:
    0

    2021-08-05T10:42:25.2050051Z 2021/08/05 10:42:25 Duration: 2.842616ms Name: google.spanner.v1.Spanner.BatchCreateSessions Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]

    2021-08-05T10:42:25.2051837Z ParentSpanID: d21332cdab39e5c7 SpanID: ade4c02b39debc42 ChildSpanCount:
    0

    2021-08-05T10:42:25.2054728Z 2021/08/05 10:42:25 Duration: 2.925417ms Name: cloud.google.com/go/spanner.BatchCreateSessions Code: 0 Message: Attributes: map[] Annotations:
    [***Time:2021-08-05 10:42:25.201796419 +0000 UTC m=+157.694579279 Message:Creating a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204703536 +0000 UTC m=+157.697486496
    Message:Received a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204718536 +0000 UTC m=+157.697501396 Message:Finished creating 2 sessions Attributes:map[]***
    ]

    2021-08-05T10:42:25.2056889Z ParentSpanID: 0000000000000000 SpanID: d21332cdab39e5c7 ChildSpanCount:
    1

    2021-08-05T10:42:25.2058866Z 2021/08/05 10:42:25 Duration: 410.303µs Name: google.spanner.v1.Spanner.BeginTransaction Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]

    2021-08-05T10:42:25.2060645Z ParentSpanID: f742a36e1e950d4d SpanID: 88b54f8ac68294bd ChildSpanCount:
    0

    2021-08-05T10:42:25.2061510Z INFO: before ReadRo
    w

    2021-08-05T10:42:25.2062577Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    0

    2021-08-05T10:42:25.2064464Z 2021/08/05 10:42:25 Duration: 9.8µs Name: cloud.google.com/go/spanner.Read Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2065690Z ParentSpanID: f742a36e1e950d4d SpanID: 6386dee853276c38 ChildSpanCount:
    1

    2021-08-05T10:42:25.2067399Z 2021/08/05 10:42:25 Duration: 555.803µs Name: google.spanner.v1.Spanner.StreamingRead Code: 0 Message: Attributes: map[Client:true FailFast:true] Annotations:
    [
    ]

    2021-08-05T10:42:25.2069446Z ParentSpanID: 3e576d47e1cb43b3 SpanID: 1928fca56e701b2e ChildSpanCount:
    0

    2021-08-05T10:42:25.2071376Z 2021/08/05 10:42:25 Duration: 748.605µs Name: cloud.google.com/go/spanner.RowIterator Code: 0 Message: Attributes: map[] Annotations: [
    ]

    2021-08-05T10:42:25.2072630Z ParentSpanID: 6386dee853276c38 SpanID: 3e576d47e1cb43b3 ChildSpanCount:
    1

    2021-08-05T10:42:25.2074888Z WARNING: err: spanner: code = "NotFound", desc = "row not found(Table: Organization, PrimaryKey: (\"111\"))
    "

    2021-08-05T10:42:25.2076611Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    9

    2021-08-05T10:42:25.2078426Z INFO: before Apply with ApplyAtLeastOnc
    e

    2021-08-05T10:42:25.2081087Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:4
    9

    2021-08-05T10:42:25.2084536Z 2021/08/05 10:42:25 Duration: 533.803µs Name: google.spanner.v1.Spanner.Commit Code: 10 Message: Transaction 2 aborted due to active transaction 1. The emulator
    only supports one transaction at a time. Some best practices to avoid ABORT errors in Cloud Spanner service are
    :

    2021-08-05T10:42:25.2087265Z 1. Avoid use of nested transactions
    .

    2021-08-05T10:42:25.2088255Z 2. Explicitly Rollback failed transactions
    .

    2021-08-05T10:42:25.2090414Z 3. All transactions should be running inside of retry loops
    .

    2021-08-05T10:42:25.2091644Z Attributes: map[Client:true FailFast:true] Annotations: [
    ]

    2021-08-05T10:42:25.2092587Z ParentSpanID: f4f88a04fc14f3a8 SpanID: cfc9196389b9ca23 ChildSpanCount:
    0

    View Slide

  16. Ͳ͏΍Δʁ
    •ςετͷͱ͖͚ͩTraceΛstdoutʹग़ྗ͢
    ΔExporterΛ࢖͏
    ͱͬͯ΋؆୯

    View Slide

  17. TextTraceExporter
    ͜Μ͚ͩ
    type textTraceExporter struct{
    }

    var _ trace.Exporter = (*textTraceExporter)(nil
    )

    func NewTextTraceExporter() trace.Exporter
    {

    return &textTraceExporter{
    }

    }

    func (exporter *textTraceExporter) ExportSpan(sd *trace.SpanData)
    {

    if os.Getenv("DUMP_TEST_TRACE") != "TRUE"
    {

    retur
    n

    }

    if sd == nil
    {

    retur
    n

    }

    rawlog.Printf
    (

    "Duration: %s Name: %s Code: %d Message: %s Attributes: %+v Annotations: %+v\n\tParentSpanID:
    %s SpanID: %s ChildSpanCount: %d"
    ,

    sd.EndTime.Sub(sd.StartTime), sd.Name, sd.Code, sd.Message, sd.Attributes, sd.Annotations
    ,

    sd.ParentSpanID, sd.SpanID, sd.ChildSpanCount
    ,

    )

    }

    View Slide

  18. TextTraceExporter
    TestMainͱ͔Ͱৗʹग़͠·͢
    exporter := NewTextTraceExporter(
    )

    trace.RegisterExporter(exporter
    )

    trace.ApplyConfig(trace.Config{DefaultSampler:
    trace.AlwaysSample()})

    View Slide

  19. ͓·͚
    Spannerͷઃఆྫ
    spanner.ClientConfig
    {

    // ઀ଓͷઃఆΛߦ͏
    // DUMP_TEST_TRACE Λ TRUE ʹͨ͠ͱ͖ɺBatchCreateSessions,
    BeginTransaction, DeleteSession ͱ͔ͷtrace͕ΊͪΌΊͪΌग़ΔͷͰ
    ਺Λগͳ͍ͨ͘͠
    SessionPoolConfig: spanner.SessionPoolConfig
    {

    MinOpened: 2, // RWTx + ApplyAtLeastOnc
    e

    MaxOpened: 5
    ,

    MaxIdle: 0
    ,

    MaxBurst: 5
    ,

    WriteSessions: 0.5
    ,

    HealthCheckWorkers: 2
    ,

    HealthCheckInterval: 0
    ,

    TrackSessionHandles: true
    ,

    }
    ,

    },

    View Slide

  20. ༨ஊ
    ຊ೔ͷࡶஊͰ͢

    View Slide

  21. ݱঢ়ͷࠔΓ
    •spanner.CommitTimestam
    p

    •Txޙʹ஋ࠩ͠ସ͍͑ͨ(frontʹฦ͢ͷͰ

    View Slide

  22. ݱঢ়ͷࠔΓ
    •Schema managementͲ͏͠Α
    •wrench?

    View Slide

  23. ݱঢ়ͷࠔΓ
    •TextTraceExporterΑΓGUI͕΄͍͠


    •ݟΔͷ͕஍ຯʹΊΜͲ͏

    View Slide

  24. ࣗ࡞πʔϧ
    • sq
    b

    • Datastore likeͳॻ͖ຯͷDMLϏϧμ
    • si
    g

    • Goͷstruct͔ΒSQL૊Έཱͯ༻ͷίʔυΛੜ੒͢Δ
    • sid
    x

    • DDL͔ΒIndex৘ใΛ࣋ͭίʔυΛੜ੒͢Δ
    • cursor(Ծ
    )

    • Datastore likeͳcursorੜ੒ϥΠϒϥϦ
    ݱঢ়ະެ։

    View Slide

  25. View Slide