Perlでも分散トレーシングしたい！AWS::XRay による解析とその実装 / YAPC::Tokyo 2019

Transcript

1. 1.

   PerlͰ΋෼ࢄτϨʔγϯά͍ͨ͠ʂ AWS::XRay ʹΑΔղੳͱͦͷ࣮૷ ɹ 2019.01.26 YAPC::Tokyo @fujiwara

2. 2.

   @fujiwara ٕज़෦ SRE github.com/fujiwara sfujiwara.hatenablog.com ISUCON8 ຊબग़୊νʔϜ

3. 3.

   Game & Community

4. 4.

   Agenda ෼ࢄτϨʔγϯάͱ͸ AWS X-Ray ͷ঺հ X-Ray ʹΑΔ ISUCON8 ຊબ໰୊ͷղੳྫ AWS::XRay

   ͷ಺෦࣮૷ ίετͷ࿩
5. 5.

   ෼ࢄτϨʔγϯάͱ͸

6. 6.

   ෼ࢄτϨʔγϯάͱ͸ GoogleͷDapper࿦จ1͕΋ͱ (2010೥) ෳ਺ͷݴޠͰߏங͞ΕΔେن໛෼ࢄγεςϜͷύϑΥʔϚϯε Λղੳ͢ΔͨΊʹ࡞ΒΕͨ 1 https://ai.google/research/pubs/pub36356

7. 7.

   ෼ࢄτϨʔγϯάͷඞཁੑ ෼ࢄγεςϜ(ϚΠΫϩαʔϏε)Ͱ͸ͻ ͱͭͷॲཧ͕ෳ਺ͷίϯϙʔωϯτʹ· ͕ͨΔ ύϑΥʔϚϯεͷ໰୊͕෼͔ΓͮΒ͍ ো֐Λݟ͚ͭΔͷ΋೉͍͠ ਤ͸ Dapper࿦จ 1 ΑΓ

   1 https://ai.google/research/pubs/pub36356
8. 8.

   ෼ࢄτϨʔγϯάͰՄࢹԽ ෳ਺ͷίϯϙʔωϯτʹ·͕ͨͬͯॲཧ ͞Εͨ݁ՌΛՄࢹԽͯ͠೺ѲΛ༰қʹ ֤ίϯϙʔωϯτ͕ݺͼग़͞Εͨॱংɺ λΠϛϯάɺཁ͕ͨ࣌ؒ͠ݟ͑Δ

9. 9.

   ֓೦ Trace͸ෳ਺ͷSpanΛؚΉ༗޲ඇ८ճάϥϑ5 5 https://github.com/opentracing/specification/blob/master/specification.md

10. 10.

    ֓೦ Span ͸ҎԼͷཁૉΛ΋ͭ ໊લ ։࢝࣌ࠁ ऴྃ࣌ࠁ Context (ID, Trace-ID, Parent)

    Tag {Key: Value},... Log {Key: Value},...
11. 11.

    ෼ࢄτϨʔγϯάͷ࣮૷ͱ࢓༷ Zipkin(OpenZipkin)2: Twitter → OSS 2012 Jeager3: Uber → CNCF

    OSS 2016 OpenTracing4: ෼ࢄτϨʔγϯάͷ࢓༷ CNCF 2016 AWS X-Ray: AWS͕ఏڙ ϚωʔδυαʔϏε 2016 StackDriver Trace: Google͕ఏڙ ZipkinΫϥΠΞϯτޓ׵ 2017 4 https://opentracing.io/ 3 https://www.jaegertracing.io/ 2 https://zipkin.io/
12. 12.

    Ͳ͏΍ͬͯτϨʔε͢Δ͔ ΞϓϦέʔγϣϯͰ ΞϓϦέʔγϣϯ(SDK)ͰτϨʔεσʔ λΛ࡞੒ͯ͠సૹ༻ΤʔδΣϯτʹૹ৴ ͔ͦ͜Βαʔό(ίϨΫλ)ʹૹΒΕΔ ! ΞϓϦέʔγϣϯ಺෦ͷڍಈ΋τ ϨʔεͰ͖Δ " ݴޠʹґଘ(SDK)

13. 13.

    Ͳ͏΍ͬͯτϨʔε͢Δ͔ αʔϏεϝογϡͰ αʔϏεϝογϡͷσʔλϓϨʔϯ(ྫ Envoy) Ͱߦ͏ ! ݴޠʹґଘ͠ͳ͍ " ΞϓϦέʔγϣϯ಺෦ͷڍಈ͸τ ϨʔεͰ͖ͳ͍

14. 14.

    AWS X-Ray

15. 15.

    AWS X-Ray AWS ʹΑΔ෼ࢄτϨʔγϯάͷϚωʔδυαʔϏε

16. 16.

    AWS X-Ray ͷߏ੒ཁૉ6 X-Ray API AWS͕ఏڙ͍ͯ͠ΔAPI X-Ray daemon τϨʔεసૹ༻σʔϞϯ X-Ray

    SDK X-Ray daemonʹτϨʔεΛૹ৴͢Δͨ ΊͷϥΠϒϥϦ X-Ray console τϨʔεΛՄࢹԽ͢ΔViewer 6 https://docs.aws.amazon.com/ja_jp/xray/latest/devguide/aws-xray.html
17. 17.

    ϚωʔδυαʔϏεͳͷͰτϨʔεΛૹΔ͚ͩͰ͙͢࢖͑Δ σʔλͷอଘ΍αʔόͷՄ༻ੑΛؾʹ͢Δඞཁ͕શ͘ͳ͍

18. 18.

    ͲͷΑ͏ʹτϨʔεΛૹ৴͢Δ͔ X-Ray daemon7 ͕ UDP port 2000 Λ listen ΞϓϦέʔγϣϯ(SDK)͔ΒτϨʔεσʔλΛUDPͰ౤͛Δ

    {"format": "json", "version": 1} { "trace_id": "1-5759e988-bd862e3fe1be46a994272793", "id": "defdfd9912dc5a56", "start_time": 1461096053.37518, "end_time": 1461096053.4042, "name": "MyApp" } 7 https://github.com/aws/aws-xray-daemon
19. 19.

    ͲͷΑ͏ʹτϨʔεΛૹ৴͢Δ͔ X-Ray daemon ͕όοϑΝϦϯάͯ͠όονͰ X-Ray API ʹૹ৴ X-Ray API ʹಧ͍ͨ͋ͱ͸׬શʹϚωʔδυ

    σʔλอ࣋ 30೔(ݹ͍ͷ͸ফ͑Δ) τϨʔε਺ͱදࣔ࣌ͷεΩϟϯͨ͠ྔʹΑΔ׬શैྔ՝ۚ
20. 20.

    X-Ray ͷΞʔΩςΫνϟͷ͍͍ͱ͜Ζ τϨʔεσʔλ͸େྔʹൃੜ͢Δ 1ϦΫΤετॲཧதʹଟ͍ͱ਺ेҎ্ ຖճΞϓϦέʔγϣϯ͔Βಉظ௨৴(TCP) → ϨΠςϯγ૿Ճʹ௚݁͢Δ ಛʹো֐࣌ X-Ray daemon

    ʹ͸UDPͰ౤͛ͬͺͳ͠ → ΞϓϦέʔγϣϯ΁ͷӨڹ͕ܰඍ X-Ray daemon ͕མ͍ͪͯͯ΋ΞϓϦʹӨڹ͠ͳ͍
21. 21.

    X-Ray ༻ޠͰ͸ εύϯ → ηάϝϯτ ࠷ॳʹઆ໌ͨ͠Ұൠͷ༻ޠ(εύϯ)ͱ͸ҧ͏͚Ͳಉ͡΋ͷ Ҏ߱͸ X-Ray ͷ࿩ͳͷͰηάϝϯτͱݺͼ·͢

22. 22.

    AWS X-Ray Λ Perl Ͱ࢖͓͏

23. 23.

    X-Ray SDK τϨʔεΛ X-Ray daemon ʹૹΔϥΠϒϥϦ AWS͕ఏڙ͍ͯ͠Δͷ͸ Java, Go, Node,

    Python, Ruby, .NET ͋Εʁࠓ೔͸ YAPC (Yet Another Perl Conference)…
24. 24.

    ! Perl SDK ͸ެࣜʹ͸ͳ͍ CPAN ʹ΋ݟ౰ͨΒͳ͍ X-Ray API Λୟ͘ϥΠϒϥϦ͸͋Δ (Paws::XRay)

25. 25.

    ͍͍ͩͨશ෦ Perl ࠷ۙͷ৽͍͠ϓϩδΣΫτ͸ Go ͚ͩͲ௕೥ͷப͸ Perl

26. 26.

    AWS::XRay Λॻ͖·ͨ͠ ཁ͢Δʹ JSON Λ UDP Ͱ౤͛Ε͹͍͍͚ͩ ͳͷͰ؆୯ 300ߦ͙Β͍͔͠ͳ͍ capture

    $name, sub {} ͰηάϝϯτΛ࡞ͬͯૹ৴ use AWS::XRay qw/ capture /; capture "myApp", sub { capture "foo", sub { # do something ... capture "bar", sub { # ... }; }; };
27. 27.

    Plack::Middleware::XRay ૊ΈࠐΉͱτϨʔεΛ։࢝ͯ͘͠ΕΔ Plack Middleware 100ߦ͙Β͍ .psgi Ͱ enable ͢Δ͚ͩ (Φϓγϣϯ͸ޙड़)

    use Plack::Builder; builder { enable "XRay", name => "myApp"; $app; };
28. 28.

    AWS::XRay ։ൃܦҢ Perl Ͱ΋ X-Ray ࢖͍͍ͨ… → JSONΛUDPͰ౤͛Δ͚ͩͩ͠؆୯ͩΑͶʁ → Ͱ΋τϨʔε࢓ࠐΉͷΊΜͲ͍͘͞ͳ…

    → ! Devel::KYTProf ͕औͬͨ݁ՌΛૹΕ͹ʂ
29. 29.

    Devel::KYTProf::Logger::XRay Devel::KYTProf ͱ͍͏ϓϩϑΝΠϥͷϩΨʔͱͯ͠ಈ࡞ͯ͠ τϨʔεσʔλΛૹ৴ Devel::KYTProf ͸ use ͢Δ͚ͩͰDBIͳͲͷݺͼग़͠Λܭଌ ϩάΛग़ྗͯ͘͠ΕΔศརϓϩϑΝΠϥ use

    Devel::KYTProf; # your code 315.837 ms [DBI::st] select * from table where name = ? (1 rows) | main:23 1464.204 ms [LWP::UserAgent] GET http://www.hatena.ne.jp/ | main:25
30. 30.

    Devel::KYTProf::Logger::XRay use Devel::KYTProf::Logger::XRay; Devel::KYTProf->logger("Devel::KYTProf::Logger::XRay"); ͜ΕͰDevel::KYTProfͰܭଌͰ͖Δ΋ͷ͸શͯτϨʔεର৅ʹ DBI, LWP::UserAgent, Furl::HTTP, Cache::Memcached::Fast, MogileFS::Client

    ࣗ෼Ͱର৅ϞδϡʔϧΛ௥Ճ͢Δ͜ͱ΋Մೳ (Redis::Fastͱ͔)
31. 31.

    AWS::XRay ։ൃܦҢ Perl Ͱ΋ X-Ray ࢖͍͍ͨ… → JSONΛUDPͰ౤͛Δ͚ͩͩ͠؆୯ͩΑͶʁ → Ͱ΋τϨʔε࢓ࠐΉͷΊΜͲ͍͘͞ͳ…

    → ! Devel::KYTProf ͕औͬͨ݁ՌΛૹΕ͹ʂ → Logger ࠩ͠ସ͑Ͱ͖Δ KYTProf++ ਖ਼௚ KYTProf ͕ͳ͔ͬͨΒ࡞ͬͯͳ͔͔ͬͨ΋͠Εͳ͍
32. 32.

    ISUCON 8 ຊબ໰୊ͷղੳ

33. 33.

    ISUCON is ͳʹ ͍͍ײ͡ʹεϐʔυΞοϓίϯςετͷུ ఏڙ͞Εͨαʔό্Ͱಈ͘WebΞϓϦέʔγϣϯΛߴ଎Խ !!!!!!!!!! ༏উ৆ۚ100ສԁ 2011೥͔Βຖ೥։࠵͞Ε͍ͯΔ (ओ࠵ LINEࣾ)

    2018೥ͷ ISUCON 8 Λ DeNA ͞ΜͱΧϠοΫͰग़୊ ࣍ͷ࿮͸ karupanerura ͞ΜʹΑΔ ʮISUCON8༧બ໰୊࡞੒ͷཪଆʯͷτʔΫͰ͢
34. 34.

    ISUCON 8 ຊબ໰୊ Ծ૝௨՟औҾαʔϏε ISUCOIN ۜߦAPI, ϩάه࿥API͸ӡӦ͕༻ҙͨ͠ ϒϥοΫϘοΫεͰఏڙ͞Εɺhttps Ͱ ϦΫΤετΛૹΔඞཁ͕͋Δ

    ֎෦APIݺͼग़͠Λ͍͔ʹ্ख͘ॲཧ͢ Δ͔͕ϙΠϯτ
35. 35.

    ISUCON 8 ຊબ໰୊

36. 36.

    ISUCON 8 ຊબ໰୊Λ X-Ray Ͱղੳͯ͠ΈΔ

37. 37.

    Perl ࣮૷ʹ AWS::XRay Λ૊ΈࠐΉ ఏڙ͞Εͨݴޠ࣮૷͸ Go, Perl, PHP, Python, Ruby

    YAPC ͳͷͰ౰વ Perl ࣮૷Ͱ https://github.com/isucon/isucon8-final
38. 38.

    X-Ray daemon Λ༻ҙ ॳظ࣮૷͸ docker-compose ͳͷͰద౰ʹ Dockerfile Λ༻ҙ8 FROM amazonlinux

    RUN yum install -y unzip RUN curl -o daemon.zip https://s3.dualstack.us-east-2.amazonaws.com/ aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-linux-3.x.zip RUN unzip daemon.zip && cp xray /usr/bin/xray ENTRYPOINT ["/usr/bin/xray", "-b", "0.0.0.0:2000", "--local-mode"] EXPOSE 2000/udp 8 https://docs.aws.amazon.com/ja_jp/xray/latest/devguide/xray-daemon-local.html
39. 39.

    docker-compose.yml services: isucoin: environment: # ུ AWS_XRAY_DAEMON_ADDRESS: 'xray:2000' links: -

    mysql - xray xray: build: xray environment: AWS_REGION: "ap-northeast-1" AWS_ACCESS_KEY_ID: "000000000000" AWS_SECRET_ACCESS_KEY: "****"
40. 40.

    AWS::XRay ૊ΈࠐΈ app.psgi ʹ5ߦ௥Ճ + cpanfile # app.psgi use Devel::KYTProf::Logger::XRay;

    use AWS::XRay; Devel::KYTProf->logger("Devel::KYTProf::Logger::XRay"); builder { enable 'XRay', name => 'isucoin'; # ... # cpanfile requires 'AWS::XRay'; requires 'Plack::Middleware::XRay'; requires 'Devel::KYTProf::Logger::XRay';
41. 41.

    ϕϯνΛ࣮ߦ͢Δͱ…τϨʔε͕ʂ

42. 42.

    None
43. 43.

    None
44. 44.

    None
45. 45.

    None
46. 46.

    None
47. 47.

    None
48. 48.

    None
49. 49.

    ϘτϧωοΫൃݟ SELECT * FROM trade ORDER BY id DESC 23ສߦΛಡΈऔͬͯ1ߦ࢖ࣺ͍ͬͯͯͯΔ

    LIMIT 1 ͢Ε͹ OK ! ʮ͜Ε͸΂ͭʹ෼ࢄτϨʔγϯά͍Βͳ͍ΑͶ?ʯ ! ʮslowlog ݟΕ͹෼͔Δ΍Ζʯ
50. 50.

    ֎෦APIݺͼग़͠Λղੳ

51. 51.

    None
52. 52.

    None
53. 53.

    None
54. 54.

    ֎෦ͷHTTP APIݺͼग़͠ͷதͰ஗͍΋ͷΛಛఆ (ग़୊ҙਤͰ) ֎෦API͝ͱʹϨΠςϯγʹࠩΛ෇͚͍ͯͨ ! ʮ͜Ε΋ผʹʰ෼ࢄʱͰ͸ͳ͍ΑͶʯ ! ʮDevel::KYTProf ͰϩάΛग़ͤ͹෼͔Δ΍Ζʯ

55. 55.

    ֎෦API࣮૷ʹ X-Ray Λ૊ΈࠐΜͰΈΔ (ڝٕͰ͸࣮ࡍ͸ϒϥοΫϘοΫε͚ͩͲ) xray-sdk-go Λ૊ΈࠐΈ // sql.Open -> xray.SQL

    db, err := xray.SQL("mysql", dsn) // http.Handler Λ xray.Handler Ͱ wrap return xray.Handler( xray.NewFixedSegmentNamer("Isubank"), myHandler, )
56. 56.

    Perl ଆͷ HTTPϦΫΤετૹ৴΋ capture ݺͼग़͠ݩΛ఻͑ΔͨΊʹ X-Amzn-Trace-ID ϔομΛ෇༩ capture "externalHTTP", sub

    { my $segment = shift; $res = $self->client->post( $self->endpoint . $p, [ "Content-Type" => "application/json", "Authorization" => "Bearer " . $self->app_id, "X-Amzn-Trace-ID" => $segment->trace_header, ], $body, ); };
57. 57.

    Isucoin(ΞϓϦ)→Isubank(API)

58. 58.

    None
59. 59.

    None
60. 60.

    ֎෦APIݺͼग़͠ͷதͰߦΘΕͨSQLΫΤϦ΋ಉ͡ը໘Ͱ

61. 61.

    ʰ෼ࢄʱτϨʔγϯάʂ

62. 62.

    ϓϩηεΛ·͕ͨͬͨ௥੻ͷ࢓૊Έ Isucoin(Perl) ϦΫΤετ։࢝࣌ʹ X-Amzn-Trace-ID Λൃߦ 1-5c4902d7-e8fe620cab66e01e57df8a12 1-[timestamp]-[unique_id] sprintf( "1-%x-%s", CORE::time(),

    unpack("H*", Crypt::URandom::urandom(12)), );
63. 63.

    ֎෦ϦΫΤετݺͼग़͠ʹϔομΛ෇༩ use AWS::XRay qw/capture/; capture "externalHTTP", sub { # Ͳ͜Ͱ΋ݺ΂Δ(਌͕͍ͳ͚Ε͹RootʹͳΔ)

    my $segment = shift; $furl->POST( "https://example.com", [ "X-Amzn-Trace-ID" => $segment->trace_header ], $body, ); }; POST / HTTP/1.1 Host: example.com X-Amzn-Trace-ID: Root=1-5c4902d7-e8fe620cab66e01e57df8a12;Parent=851cf9ec07b528c2 ...
64. 64.

    ϔομΛड৴ͨ͠API(Isubank)͕Ҿ͖ܧ͙ Root=1-5c4902d7-e8fe620cab66e01e57df8a12;Parent=851cf9ec07b528c2 # ࣮ࡍ͸Go͚ͩͲઆ໌ͷͨΊʹPerl AWS::XRay::Segment->new( id => new_id(), trace_id =>

    "1-5c4902d7-e8fe620cab66e01e57df8a12", parent_id => "851cf9ec07b528c2", ); ͦͷτϨʔεΛ X-Ray daemon ΁ૹ৴ → X-Ray consoleͰτϨʔε͕࿈݁ͯ͠දࣔ͞ΕΔ
65. 65.

    AWS::XRay ͷ಺෦࣮૷

66. 66.

    τϨʔε͍ͨ͠ͱ͜ΖͰʮݱࡏͷʯRoot΍Parent͕ඞཁ package App; sub main { capture "App::main", sub {

    # Root my $segment = shift; Model::call($param); # ֎෦ݺͼग़͠Λ൐͏ॲཧ }; }; package Model; sub call() { my $param = shift; # ࣮ࡍʹ֎ΛݺͿͷ͸͕ͩ͜͜ݺͼग़͠ݩͷsegmentΛ஌Βͳ͍ͱRoot͕Θ͔Βͳ͍ $furl->post($url, [ "X-Amzn-Trace-ID" => "????", ], $param); } call($param)Λcall($segment,$param)ʹॻ͖׵͑ͨ͘ͳ͍
67. 67.

    AWS::XRay Ͱ͸Ҿ਺Λ౉͞ͳ͍͍ͯ͘Α͏ʹͳͬͯΔ package App; sub main { capture "App::main", sub

    { # Root my $segment = shift; Model::call($param); # ֎෦ݺͼग़͠Λ൐͏ॲཧ }; }; package Model; sub call() { my $param = shift; # ਌ͷ͜ͱ͸஌Βͳ͍͚Ͳͱʹ͔͘ capture ͢Ε͹Α͍ capture "Model::call", sub { my $segment = shift; # ਌͔Βݺ͹Ε͍ͯΕ͹ࣗಈతʹࢠʹͳΔ $furl->post($url, [ "X-Amzn-Trace-ID" => $segment->trace_header, ], $param); }; }
68. 68.

    Ͳ͏΍࣮ͬͯ૷͞Ε͍ͯΔ͔ Perl ͷຐ๏ͷͻͱͭ local perldoc -f local ͋ͳ͕ͨຊ౰ʹ๬ΜͰ͍Δͷ͸ my ͷํͰ͠ΐ͏;

    local ͸΄ͱΜ Ͳͷਓʑ͕ʮϩʔΧϧʯͱߟ͑Δ΋ͷͱ ҧ͏͔ΒͰ͢ɻ "local" ͸ϦετΞοϓ͞Εͨม਺Λɺғ͍ͬͯΔϒϩοΫɺ ϑΝΠϧɺeval ͷதͰɺϩʔΧϧͳ΋ͷʹ͠·͢ɻ !!!!!!!
69. 69.

    local → μΠφϛοΫείʔϓ (ݺͼग़͠ݩͰείʔϓ͕ܾ·Δ) my → ϨΩγΧϧείʔϓ (ιʔείʔυͷࣈ໘Ͱείʔϓ͕ܾ·Δ) package main;

    our $Foo = "A"; # ύοέʔδม਺ sub foo { print $Foo; } foo(); #= A { local $Foo = "B"; # ͜ͷϒϩοΫ͔Βݺ͹Ε͍ͯΔؒ͸B foo(); #= B } foo(); #= A ϒϩοΫΛൈ͚Δͱࣗಈతʹ໭Δ
70. 70.

    Կճωετͯ͠ݺͼग़ͯ͠΋ॱ൪ʹ্ॻ͖ͯ͠ɺ΋ͲΕ͹෮ݩ foo(); #= A { local $Foo = "B"; foo();

    #= B { local $Foo = "C"; foo(); #= C { local $Foo = "D"; foo() #= D } foo(); #= C } foo(); #= B } foo(); #= A
71. 71.

    AWS::XRay Ͱ͸۩ମతʹ͸͜Μͳײ͡ ݱࡏͷ Trace-ID, Segment-ID Λ local Ͱอ࣋ package AWS::XRay;

    our $TRACE_ID; # Root our $SEGMENT_ID; # ݱࡏͷSegmentID sub capture { my ($name, $code) = @_; # $TRACE_ID͕ະఆٛͳΒRootͳͷͰ৽نIDൃߦ local ʹ local $TRACE_ID = $TRACE_ID // new_trace_id(); # ৽͍͠ηάϝϯτΛ࡞Δ # $SEGMENT_ID ͕͋Ε͹ parent_id => $SEGMENT_ID ʹͳ͍ͬͯΔ my $segment = AWS::XRay::Segment->new({ name => $name, type => $SEGMENT_ID ? "subsegment" : undef, parent_id => $SEGMENT_ID, });
72. 72.

    local $SEGMENT_ID = $segment->{id}; # ৽͍͠ηάϝϯτIDΛ্ॻ͖ͯ͠localʹଋറ # ίʔυΛ࣮ߦ # ͜ͷதͰ͸

    $SEGMENT_ID ͕ઃఆ͞Ε͍ͯΔͷͰ # ͞Βʹ capture ͨ͠Β $segment ͷࢠڙʹͳΔ $ret = $code->($segment); $segment->close(); # ηάϝϯτσʔλૹ৴ return $ret; # $segment ͸ফ͑Δ(my͔ͩΒ) # $SEGMENT_ID ΋ݺͼग़࣌͠ͷ஋ʹ໭Δ # RootͳΒ $TRACE_ID ΋ফ͑Δ(ݩ͕ະఆ͔ٛͩΒ) }
73. 73.

    Perl ʹ͓͚Δ local ͦͷϒϩοΫ(౳)͔Βݺͼग़͞Ε͍ͯΔ͚ؒͩม਺ͷ஋Λมߋ ྫ֎Ͱ΋ԿͰ΋ൈ͚ͨΒݩʹ໭Δ Կஈ֊Ͱ΋ωετͰ͖Δ ݴޠͰαϙʔτ͞Ε͍ͯΔͷͰʮ࣮֬ʹʯ໭Δ Perl ࠷ߴศརʂʂ

74. 74.

    ༨ஊ Go Ͱ X-Ray SDK Λ࢖͏৔߹ ϦΫΤετͰൃੜͨ͠ context.Context ΛͲ͜·Ͱ΋ Ҿ਺ʹ౉͍͔ͯ͠ͳ͍ͱτϨʔεͰ͖ͳ͍

    ్தʹ ctx ΛҾ͖ܧ͛ͳ͍ϥΠϒϥϦ(ORMͱ͔)͕ڬ·Δͱ (Isubank ΋࣮ࡍ͸40ՕॴҎ্ ctx Λ౉͢Α͏ʹॻ͖׵͑ͨ)
75. 75.

    ! ίετͷ࿩

76. 76.

    ! ίετͷ࿩ ʮX-Ray ͸Αͦ͞͏͚ͩͲɺ͓ߴ͍ΜͰ͠ΐ͏…?ʯ ௚઀ίετ = X-Ray ར༻ྉۚ ؒ઀ίετ =

    ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ (݁Ռతʹ࢖༻Ϧιʔε͕૿͑ΔɺϢʔβମݧ͕ଛͳΘΕΔͳͲ)
77. 77.

    ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ τϨʔεΛ૊ΈࠐΉ͜ͱʹΑͬͯͲΕ͙Β͍஗͘ͳΔ͔ • 1ॲཧͰ100ݸͷ capture (ίʔυ͸ۭ sub{}) Λੜ੒ • αϯϓϦϯά͠ͳ͍ͰશͯͷτϨʔεΛૹ৴

    Λ 500/sec ఔ౓࣮ߦͰ͖Δ (macbook pro 13 2017) ͭ·Γ1ॲཧ͋ͨΓ2msఔ౓ͷΦʔόʔϔου τϨʔεΛૹ৴͠ͳ͍(captureΛωετ͢Δ͚ͩ)ͳΒ 10,000/sec, 0.1msఔ౓ͷΦʔόʔϔου
78. 78.

    ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ ͍͍ͨͯͷWebΞϓϦέʔγϣϯʹରͯ͠͸໰୊ͳ͍଎౓ ࣮ࡍʹ͸αϯϓϦϯάΛߦ͏ͷͰߋʹӨڹ͸ܰඍ αϯϓϦϯά1%Ͱຊ൪౤ೖͯ͠ύϑΥʔϚϯε΁ͷӨڹͳ͠ ࣮ࡍͷܭଌίʔυ͸ xt/01_overhead.t10 ࢀর 10 https://github.com/fujiwara/AWS-XRay/blob/master/xt/01_overhead.t

79. 79.

    X-Ray ར༻ྉۚ ຖ݄ͷແྉ࿮ τϨʔεͷه࿥͸ 10 ສճ·Ͱແྉ τϨʔεͷऔಘͱεΩϟϯ͸߹Θͤͯ 100 ສճ·Ͱແྉ ௥ՃͰ

    τϨʔεͷه࿥ 100 ສ݅͋ͨΓ 5 USD τϨʔεͷऔಘͱεΩϟϯΛ߹Θͤͯ 100 ສ݅͋ͨΓ 0.50 USD
80. 80.

    ࣮ࡍͲΕ͙Βֻ͍͔Δͷʁ ฏۉ 100 req/sec ͷ৔߹ 100 * 86,400 * 30

    / 1,000,000 * 5 USD = 1,296 USD ฏۉ 1,000 req/sec ͳΒ 12,960 USD / month શͯͷϦΫΤετʹ͍ͭͯه࿥͢Δͷ͸ίετ͕ߴ͍
81. 81.

    αϯϓϦϯά͢Δ ద౰ʹαϯϓϦϯά͢Ε͹܏޲͸௫ΊΔ ͋·Γൃੜ͠ͳ͍Τϥʔɺslowdown ࣌ͷௐࠪʹ͸࢖͍ͮΒ͍

82. 82.

    αϯϓϦϯάͷ޻෉ ྫɿ1ඵ͝ͱʹ࠷ॳͷϦΫΤετͱ ௥ՃϦΫΤετͷ5%Λه࿥ (ެࣜSDKͷσϑΥϧτઃఆ) 100 req/sec ͷ৔߹ 1 + 100

    * 5% = 6 req/sec ෼͕ه࿥͞ΕΔ ϦΫΤετ͕গͳͯ͘΋ 1 req/sec ͸࣮֬ʹه࿥ ެࣜ SDK ͸ X-Ray console ͔ΒαϯϓϦϯάϧʔϧΛઃఆՄ (AWS::XRay Ͱ͸ະରԠ…)
83. 83.

    X-Ray console ͰͷαϯϓϦϯάઃఆ

84. 84.

    Plack::Middleware::XRay ͰͷαϯϓϦϯάઃఆ ݻఆϨʔτ enable "XRay" name => "myApp", sampling_rate =>

    0.05; # 5%
85. 85.

    αϯϓϦϯάϧʔϧΛίʔυͰهड़ PSGI $env Λड͚औͬͯਅِΛฦ͢ίʔυΛࢦఆͰ͖Δ enable "XRay" name => "myApp", sampler

    => sub { my $env = shift; state %paths; if ( $paths{$env->{PATH_INFO}++ == 0 ) { # ಛఆͷPATHΛॳΊͯड৴ͨ͠৔߹͸ඞͣه࿥ return 1; } rand() < 0.05; # ͦΕҎ֎͸ 5% };
86. 86.

    Τϥʔ΍஗͍ϦΫΤετ͸ඞͣه࿥͍ͨ͠ 5% αϯϓϦϯάˠ ΤϥʔΛه࿥Ͱ͖Δ֬཰΋5% શϦΫΤετͰϝϞϦ্ʹτϨʔεΛอଘ͓ͯ͘͠ ϨεϙϯεΛฦ࣌͢఺ͰτϨʔεΛૹ৴͢Δ͔Ͳ͏͔ΛܾΊΔ enable "XRay" name =>

    "myApp", response_filter => sub { my ($env, $res, $elapsed) = @_; # 5xx OR 1ඵҎ্ֻ͔ͬͨ৔߹ʹૹ৴ return $res->[0] >= 500 || $elapsed >= 1; };
87. 87.

    ·ͱΊ

88. 88.

    ·ͱΊ ෼ࢄτϨʔγϯά͸ύϑΥʔϚϯεɺো֐ղੳʹେม༗༻ ର ISUCON ༻ܾઓฌثͱͯ͠΋༗༻ AWS X-Ray ͸ϚωʔδυͰಋೖ͕༰қ αϯϓϦϯάઃఆʹ͸ཁ஫ҙ !

    Perl SDK ͸ͳ͔ͬͨͷͰ AWS::XRay Λ࡞ͬͨ पลϞδϡʔϧؚΊͯ΋਺ඦߦͰ͍͍͔Μ͡ʹ