DatastoreからSpannerに 移行したいぞ途中編 / migrate Datastore to Spanner in progress

GCPUG Tokyo Spanner Day August 2021
https://gcpug-tokyo.connpass.com/event/220499/

DatastoreからSpannerに 移行したいぞ途中編 という話

Transcript

1. Datastore͔ΒSpannerʹ
   
   
   Ҡߦ్͍ͨͧ͠தฤ
   GCPUG Tokyo Spanner Day August 2021
   Θ͔Ί ·͞ͻΖ
   

   View Slide

2. Θ͔Ί ·͞ͻΖ @v vakame
   Masahiro Wakame
   GAE/Go
   GraphQL
   ٕज़ॻయ
   

   View Slide

3. ຊ೔ͷςʔϚ
   •ٕज़ॻయWebΛҠߦ͍ͨ͠ʂ
   •Datastore→Spanner ࡞ۀதͰ͢
   •ݱঢ়ͷ஌ݟ
   ͍ͭ׬ྃ͢Δ͔͸Θ͔ΒΜ…
   

   View Slide

4. Ϟνϕʔγϣϯ
   •UNION ALLͱ͔͔ͨͬͨ͠
   •ձ͕ࣾओʹSpannerͳͷͰ৮Γ͔ͨͬͨ
   •ίετ͕ 1/10 ʹͳͬͨͷͰݱ࣮తʹ
   ΫΤϦͷॊೈ͞ʹظ଴…ʂ
   

   View Slide

5. ΞʔΩςΫνϟ1
   •AppEngine/Go 1.1
   5
   
   •Datastore → Spanne
   r
   
   •GraphQL (gqlgen)
   GraphQLͷԸܙ͕݁ߏσΧ͍
   

   View Slide

6. ΞʔΩςΫνϟ2
   1. API (GraphQL
   )
   
   2. Service (domain
   )
   
   3. Repository (DB)
   LayeredͬΆ͍͚ͲશવͦΜͳ͜ͱͳ͍
   

   View Slide

7. جຊํ਑
   •Datastore༻RepositoryΛࠩ͠ସ͑
   •؀ڥม਺Ͱ੾Γସ͑
   •Spanner friendlyͳมߋ͸ೖΕͳ͍
   ·ͣ͸΍Γ͖Γ͍ͨ
   

   View Slide

8. ਐḿ
   •શRepositoryͷSpanner൛ͷ࡞੒׬
   •Datastore,SpannerͰಉ͡ςετ͕௨ͬͨ
   •؀ڥ੔උ΍σʔλҠߦݕ౼͸͜Ε͔Β
   ંΓฦͨ͠ײ͸͋Δ
   

   View Slide

9. Tips
   ຊ೔ͷຊ୊Ͱ͢
   

   View Slide

10. ओு
    •OpenCensusͷTrace͸େࣄʂʂʂ
    

    View Slide

11. Spanner Emulator
    •ศར CIͰ࢖ͬͯ·͢
    •ReadWriteTransaction͕ෳ਺షΕͳ͍
    •͜ͷ੍໿͕݁ߏΩπ͍ʂ
    •spanner.Client#Apply ΋ྫ֎͡Όͳ͍
    ApplyAtLeastOnce΋ͩΊ
    

    View Slide

12. Ͳ͏ͳΔʁ
    •ΤϥʔʹͳΔ
    •Timeout·ͰϦτϥΠ͢Δ
    •ΞϓϦ͔ΒΤϥʔݟ͑ͳ͍
    ී௨ʹ਺༹͚࣌ؒ·͢
    

    View Slide

13. ରࡦ
    •Google͸OpenCensus࢖ͬͯΔ
    •https://bit.ly/37qymo
    W
    
    •Open CensusͷTraceϩάΛݟΔͱ݁ߏ
    ৭ʑग़ͯΔ…ʂʂ
    Spanner Clientͷίʔυࣗମ΋ࢀߟʹͳΓ·͢
    

    View Slide

14. ΍ͬͯΈͨ
    Traceࠐͷ࣮ߦ݁Ռ
    2021-08-05T10:42:25.1409165Z === RUN TestSpanne
    r
    
    2021-08-05T10:42:25.2019180Z 2021/08/05 10:42:25 Duration: 68.2807ms Name: google.spanner.admin.database.v1.DatabaseAdmin.CreateDatabase Code: 0 Message: Attributes: map[Client:true
    FailFast:true] Annotations: [
    ]
    
    2021-08-05T10:42:25.2030092Z ParentSpanID: 66d0f65276a786dc SpanID: 41a185b1034c5e57 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2034069Z INFO: create spanner client with projects/tbf-test/instances/tbf-test-instance/databases/for-ut-a76
    3
    
    2021-08-05T10:42:25.2035431Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_utils.go:7
    0
    
    2021-08-05T10:42:25.2037531Z 2021/08/05 10:42:25 Duration: 179.701µs Name: cloud.google.com/go/spanner.NewClient Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2038616Z ParentSpanID: 66d0f65276a786dc SpanID: 1c72141091f035d2 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2050051Z 2021/08/05 10:42:25 Duration: 2.842616ms Name: google.spanner.v1.Spanner.BatchCreateSessions Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]
    
    2021-08-05T10:42:25.2051837Z ParentSpanID: d21332cdab39e5c7 SpanID: ade4c02b39debc42 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2054728Z 2021/08/05 10:42:25 Duration: 2.925417ms Name: cloud.google.com/go/spanner.BatchCreateSessions Code: 0 Message: Attributes: map[] Annotations:
    [***Time:2021-08-05 10:42:25.201796419 +0000 UTC m=+157.694579279 Message:Creating a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204703536 +0000 UTC m=+157.697486496
    Message:Received a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204718536 +0000 UTC m=+157.697501396 Message:Finished creating 2 sessions Attributes:map[]***
    ]
    
    2021-08-05T10:42:25.2056889Z ParentSpanID: 0000000000000000 SpanID: d21332cdab39e5c7 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2058866Z 2021/08/05 10:42:25 Duration: 410.303µs Name: google.spanner.v1.Spanner.BeginTransaction Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]
    
    2021-08-05T10:42:25.2060645Z ParentSpanID: f742a36e1e950d4d SpanID: 88b54f8ac68294bd ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2061510Z INFO: before ReadRo
    w
    
    2021-08-05T10:42:25.2062577Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    0
    
    2021-08-05T10:42:25.2064464Z 2021/08/05 10:42:25 Duration: 9.8µs Name: cloud.google.com/go/spanner.Read Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2065690Z ParentSpanID: f742a36e1e950d4d SpanID: 6386dee853276c38 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2067399Z 2021/08/05 10:42:25 Duration: 555.803µs Name: google.spanner.v1.Spanner.StreamingRead Code: 0 Message: Attributes: map[Client:true FailFast:true] Annotations:
    [
    ]
    
    2021-08-05T10:42:25.2069446Z ParentSpanID: 3e576d47e1cb43b3 SpanID: 1928fca56e701b2e ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2071376Z 2021/08/05 10:42:25 Duration: 748.605µs Name: cloud.google.com/go/spanner.RowIterator Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2072630Z ParentSpanID: 6386dee853276c38 SpanID: 3e576d47e1cb43b3 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2074888Z WARNING: err: spanner: code = "NotFound", desc = "row not found(Table: Organization, PrimaryKey: (\"111\"))
    "
    
    2021-08-05T10:42:25.2076611Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    9
    
    2021-08-05T10:42:25.2078426Z INFO: before Apply with ApplyAtLeastOnc
    e
    
    2021-08-05T10:42:25.2081087Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:4
    9
    
    2021-08-05T10:42:25.2084536Z 2021/08/05 10:42:25 Duration: 533.803µs Name: google.spanner.v1.Spanner.Commit Code: 10 Message: Transaction 2 aborted due to active transaction 1. The emulator
    only supports one transaction at a time. Some best practices to avoid ABORT errors in Cloud Spanner service are
    :
    
    2021-08-05T10:42:25.2087265Z 1. Avoid use of nested transactions
    .
    
    2021-08-05T10:42:25.2088255Z 2. Explicitly Rollback failed transactions
    .
    
    2021-08-05T10:42:25.2090414Z 3. All transactions should be running inside of retry loops
    .
    
    2021-08-05T10:42:25.2091644Z Attributes: map[Client:true FailFast:true] Annotations: [
    ]
    
    2021-08-05T10:42:25.2092587Z ParentSpanID: f4f88a04fc14f3a8 SpanID: cfc9196389b9ca23 ChildSpanCount:
    0
    
    

    View Slide

15. ΍ͬͯΈͨ
    ͦͯ͠Timeout΁…
    2021-08-05T10:42:25.1409165Z === RUN TestSpanne
    r
    
    2021-08-05T10:42:25.2019180Z 2021/08/05 10:42:25 Duration: 68.2807ms Name: google.spanner.admin.database.v1.DatabaseAdmin.CreateDatabase Code: 0 Message: Attributes: map[Client:true
    FailFast:true] Annotations: [
    ]
    
    2021-08-05T10:42:25.2030092Z ParentSpanID: 66d0f65276a786dc SpanID: 41a185b1034c5e57 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2034069Z INFO: create spanner client with projects/tbf-test/instances/tbf-test-instance/databases/for-ut-a76
    3
    
    2021-08-05T10:42:25.2035431Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_utils.go:7
    0
    
    2021-08-05T10:42:25.2037531Z 2021/08/05 10:42:25 Duration: 179.701µs Name: cloud.google.com/go/spanner.NewClient Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2038616Z ParentSpanID: 66d0f65276a786dc SpanID: 1c72141091f035d2 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2050051Z 2021/08/05 10:42:25 Duration: 2.842616ms Name: google.spanner.v1.Spanner.BatchCreateSessions Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]
    
    2021-08-05T10:42:25.2051837Z ParentSpanID: d21332cdab39e5c7 SpanID: ade4c02b39debc42 ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2054728Z 2021/08/05 10:42:25 Duration: 2.925417ms Name: cloud.google.com/go/spanner.BatchCreateSessions Code: 0 Message: Attributes: map[] Annotations:
    [***Time:2021-08-05 10:42:25.201796419 +0000 UTC m=+157.694579279 Message:Creating a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204703536 +0000 UTC m=+157.697486496
    Message:Received a batch of 2 sessions Attributes:map[]*** ***Time:2021-08-05 10:42:25.204718536 +0000 UTC m=+157.697501396 Message:Finished creating 2 sessions Attributes:map[]***
    ]
    
    2021-08-05T10:42:25.2056889Z ParentSpanID: 0000000000000000 SpanID: d21332cdab39e5c7 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2058866Z 2021/08/05 10:42:25 Duration: 410.303µs Name: google.spanner.v1.Spanner.BeginTransaction Code: 0 Message: Attributes: map[Client:true FailFast:true]
    Annotations: [
    ]
    
    2021-08-05T10:42:25.2060645Z ParentSpanID: f742a36e1e950d4d SpanID: 88b54f8ac68294bd ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2061510Z INFO: before ReadRo
    w
    
    2021-08-05T10:42:25.2062577Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    0
    
    2021-08-05T10:42:25.2064464Z 2021/08/05 10:42:25 Duration: 9.8µs Name: cloud.google.com/go/spanner.Read Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2065690Z ParentSpanID: f742a36e1e950d4d SpanID: 6386dee853276c38 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2067399Z 2021/08/05 10:42:25 Duration: 555.803µs Name: google.spanner.v1.Spanner.StreamingRead Code: 0 Message: Attributes: map[Client:true FailFast:true] Annotations:
    [
    ]
    
    2021-08-05T10:42:25.2069446Z ParentSpanID: 3e576d47e1cb43b3 SpanID: 1928fca56e701b2e ChildSpanCount:
    0
    
    2021-08-05T10:42:25.2071376Z 2021/08/05 10:42:25 Duration: 748.605µs Name: cloud.google.com/go/spanner.RowIterator Code: 0 Message: Attributes: map[] Annotations: [
    ]
    
    2021-08-05T10:42:25.2072630Z ParentSpanID: 6386dee853276c38 SpanID: 3e576d47e1cb43b3 ChildSpanCount:
    1
    
    2021-08-05T10:42:25.2074888Z WARNING: err: spanner: code = "NotFound", desc = "row not found(Table: Organization, PrimaryKey: (\"111\"))
    "
    
    2021-08-05T10:42:25.2076611Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:2
    9
    
    2021-08-05T10:42:25.2078426Z INFO: before Apply with ApplyAtLeastOnc
    e
    
    2021-08-05T10:42:25.2081087Z /home/runner/work/TechBookFestWeb/TechBookFestWeb/server/domains/spanner_test.go:4
    9
    
    2021-08-05T10:42:25.2084536Z 2021/08/05 10:42:25 Duration: 533.803µs Name: google.spanner.v1.Spanner.Commit Code: 10 Message: Transaction 2 aborted due to active transaction 1. The emulator
    only supports one transaction at a time. Some best practices to avoid ABORT errors in Cloud Spanner service are
    :
    
    2021-08-05T10:42:25.2087265Z 1. Avoid use of nested transactions
    .
    
    2021-08-05T10:42:25.2088255Z 2. Explicitly Rollback failed transactions
    .
    
    2021-08-05T10:42:25.2090414Z 3. All transactions should be running inside of retry loops
    .
    
    2021-08-05T10:42:25.2091644Z Attributes: map[Client:true FailFast:true] Annotations: [
    ]
    
    2021-08-05T10:42:25.2092587Z ParentSpanID: f4f88a04fc14f3a8 SpanID: cfc9196389b9ca23 ChildSpanCount:
    0
    
    

    View Slide

16. Ͳ͏΍Δʁ
    •ςετͷͱ͖͚ͩTraceΛstdoutʹग़ྗ͢
    ΔExporterΛ࢖͏
    ͱͬͯ΋؆୯
    

    View Slide

17. TextTraceExporter
    ͜Μ͚ͩ
    type textTraceExporter struct{
    }
    
    var _ trace.Exporter = (*textTraceExporter)(nil
    )
    
    func NewTextTraceExporter() trace.Exporter
    {
    
    return &textTraceExporter{
    }
    
    }
    
    func (exporter *textTraceExporter) ExportSpan(sd *trace.SpanData)
    {
    
    if os.Getenv("DUMP_TEST_TRACE") != "TRUE"
    {
    
    retur
    n
    
    }
    
    if sd == nil
    {
    
    retur
    n
    
    }
    
    rawlog.Printf
    (
    
    "Duration: %s Name: %s Code: %d Message: %s Attributes: %+v Annotations: %+v\n\tParentSpanID:
    %s SpanID: %s ChildSpanCount: %d"
    ,
    
    sd.EndTime.Sub(sd.StartTime), sd.Name, sd.Code, sd.Message, sd.Attributes, sd.Annotations
    ,
    
    sd.ParentSpanID, sd.SpanID, sd.ChildSpanCount
    ,
    
    )
    
    }
    
    

    View Slide

18. TextTraceExporter
    TestMainͱ͔Ͱৗʹग़͠·͢
    exporter := NewTextTraceExporter(
    )
    
    trace.RegisterExporter(exporter
    )
    
    trace.ApplyConfig(trace.Config{DefaultSampler:
    trace.AlwaysSample()})
    

    View Slide

19. ͓·͚
    Spannerͷઃఆྫ
    spanner.ClientConfig
    {
    
    // ઀ଓͷઃఆΛߦ͏
    // DUMP_TEST_TRACE Λ TRUE ʹͨ͠ͱ͖ɺBatchCreateSessions,
    BeginTransaction, DeleteSession ͱ͔ͷtrace͕ΊͪΌΊͪΌग़ΔͷͰ
    ਺Λগͳ͍ͨ͘͠
    SessionPoolConfig: spanner.SessionPoolConfig
    {
    
    MinOpened: 2, // RWTx + ApplyAtLeastOnc
    e
    
    MaxOpened: 5
    ,
    
    MaxIdle: 0
    ,
    
    MaxBurst: 5
    ,
    
    WriteSessions: 0.5
    ,
    
    HealthCheckWorkers: 2
    ,
    
    HealthCheckInterval: 0
    ,
    
    TrackSessionHandles: true
    ,
    
    }
    ,
    
    },
    

    View Slide

20. ༨ஊ
    ຊ೔ͷࡶஊͰ͢
    

    View Slide

21. ݱঢ়ͷࠔΓ
    •spanner.CommitTimestam
    p
    
    •Txޙʹ஋ࠩ͠ସ͍͑ͨ(frontʹฦ͢ͷͰ
    

    View Slide

22. ݱঢ়ͷࠔΓ
    •Schema managementͲ͏͠Α
    •wrench?
    

    View Slide

23. ݱঢ়ͷࠔΓ
    •TextTraceExporterΑΓGUI͕΄͍͠
    …
    
    •ݟΔͷ͕஍ຯʹΊΜͲ͏
    

    View Slide

24. ࣗ࡞πʔϧ
    • sq
    b
    
    • Datastore likeͳॻ͖ຯͷDMLϏϧμ
    • si
    g
    
    • Goͷstruct͔ΒSQL૊Έཱͯ༻ͷίʔυΛੜ੒͢Δ
    • sid
    x
    
    • DDL͔ΒIndex৘ใΛ࣋ͭίʔυΛੜ੒͢Δ
    • cursor(Ծ
    )
    
    • Datastore likeͳcursorੜ੒ϥΠϒϥϦ
    ݱঢ়ະެ։
    

    View Slide

25. View Slide