$30 off During Our Annual Pro Sale. View Details »

はてなリモートインターンシップ2022 Web API 講義資料

Hatena
December 19, 2022

はてなリモートインターンシップ2022 Web API 講義資料

Hatena

December 19, 2022
Tweet

More Decks by Hatena

Other Decks in Programming

Transcript

  1. HTTP & Web API
    #hatenaintern

    View Slide

  2. ׆ס铺紶ךע
    HTTP Web API ,
    Web API

    View Slide

  3. ׆ס铺紶ך⹦׽䣽ֹ׵סꯂ䈋ע㵼׊鷿ֹ
    HTTP
    HTTP
    HTTP
    TLS
    TCP, UDP, QUIC
    API
    Web API
    REST
    GraphQL
    gRPC

    View Slide

  4. HTTP
    Hypertext Transfer Protocol

    View Slide

  5. URL
    Uniform Resource Locator
    WHATWG URL Living Standard
    RFC : Uniform Resource Identi er (URI): Generic Syntax

    View Slide

  6. HTTP
    HTTP/ .
    HTTP/ .
    HTTP/ .
    Google SPDY
    Google QUIC
    SPDY HTTP/
    HTTP-over-QUIC HTTP/
    QUIC
    / / HTTP/

    View Slide

  7. HTTP RFCs
    - RFC : HTTP Semantics
    - RFC : HTTP Caching
    - RFC : HTTP/ .
    - RFC : HTTP/
    - RFC : HTTP/
    - RFC : Expect-CT Extension for HTTP
    - RFC : QPACK: Field Compression for HTTP/
    - RFC : Building Protocols with HTTP
    - RFC : The Proxy-Status HTTP Response Header Field
    - RFC : The Cache-Status HTTP Response Header Field
    - RFC : Targeted HTTP Cache Control
    - RFC : Extensible Prioritization Scheme for HTTP
    - RFC : Bootstrapping WebSockets with HTTP/
    - RFC : Oblivious DNS over HTTPS

    View Slide

  8. HTTP/ . RFCs
    RFC Hypertext Transfer Protocol -- HTTP/ .
    Obsoleted by: , , , , ,
    RFC Hypertext Transfer Protocol -- HTTP/ .
    Obsoleted by: , , , , ,
    RFC Hypertext Transfer Protocol (HTTP/ . ): Message Syntax and Routing
    Obsoleted by: ,
    RFC Hypertext Transfer Protocol (HTTP/ . ): Semantics and Content
    Obsoleted by:
    RFC Hypertext Transfer Protocol (HTTP/ . ): Conditional Requests
    Obsoleted by:
    RFC Hypertext Transfer Protocol (HTTP/ . ): Range Requests
    Obsoleted by:
    RFC Hypertext Transfer Protocol (HTTP/ . ): Caching
    Obsoleted by:
    RFC Hypertext Transfer Protocol (HTTP/ . ): Authentication
    Obsoleted by:
    RFC HTTP Semantics
    RFC HTTP Caching
    RFC HTTP/ .

    View Slide

  9. HTTP
    RFC HTTP Semantics
    HTTP

    View Slide

  10. HTTP
    ْخشغכذ٭أشع
    تط٭ذت
    ىشر٭
    RFC Field
    (RFC Section )
    ٍظؔ
    RFC Content
    HTTP/ . Content Message Body (RFC Section )

    View Slide

  11. Netcat HTTP/ .
    : :
    https://www.wdic.org/w/WDIC/

    View Slide

  12. TLS .
    RFC : The Transport Layer Security (TLS) Protocol Version .
    HTTP
    HTTP
    https TLS
    RFC . .
    ALPN: RFC Transport Layer Security (TLS) Application-Layer Protocol Negotiation Extension
    TLS HTTP
    Clinet Hello(TLS ) - :
    Encrypted Extensions - :

    View Slide

  13. TLS HTTP
    TLS HTTP :
    $ echo -e "HEAD / HTTP/1.1\r\nHost: hatenablog.com \r\n" | openssl s_client -ign_eof -connect hatenablog.com:443
    ALPN http/ . http/
    $ echo | openssl s_client -alpn h2,http/1.1 -connect hatenablog.com:443
    http/ . h
    - IANAIANA ALPN Protocol IDs HTTP
    IANA Internet Assigned Numbers Authority: Web ID

    View Slide

  14. Netcat HTTP/ . HTTP/ .

    View Slide

  15. HTTP
    GET
    POST
    PUT
    HEAD
    DELETE
    OPTIONS
    TRACE
    CONNECT
    PATCH

    View Slide

  16. GitHub
    ˝ GET /repos/:owner/:repo/issues/
    comments/:comment_id
    ˝ POST /repos/:owner/:repo/issues/:issue_number/
    comments
    ˝ PATCH /repos/:owner/:repo/issues/
    comments/:comment_id
    ˝ DELETE /repos/:owner/:repo/issues/
    comments/:comment_id

    View Slide

  17. تط٭ذت车
    HTTP/1.1 000 Reason
    1xx Informational
    2xx Successful
    3xx Redirection
    4xx Client Error
    5xx Server Error

    View Slide

  18. ىشر٭ס❆
    Host: HTTP/ .
    User-Agent
    sec-ch-ua:
    Content Negotiation Fields: RFC .
    https://wicg.github.io/ua-client-hints/
    https://www.rfc-editor.org/rfc/
    rfc #section- . .
    A client MUST send a Host header field (Section 7.2 of [HTTP]) in all HTTP/1.1
    request messages. https://www.rfc-editor.org/rfc/rfc .html#section- . -

    View Slide

  19. ( ) http://www.example.com
    ٛؠؙتع
    GET / HTTP/1.1
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Upgrade-Insecure-Requests: 1
    Host: www.example.com
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15
    Accept-Language: ja
    Accept-Encoding: gzip, deflate
    Connection: keep-alive
    ٝتَ٤ت
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=UTF-8
    Vary: Accept-Encoding
    Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
    Age: 573292
    Content-Encoding: gzip
    Expires: Fri, 26 Aug 2022 02:22:20 GMT
    Cache-Control: max-age=604800
    Date: Fri, 19 Aug 2022 02:22:20 GMT
    Content-Length: 648
    ETag: "3147526947"
    Accept-Ranges: bytes
    Server: ECS (oxr/8315)
    X-Cache: HIT

    View Slide

  20. Content Negotiation Fields:
    Accept
    Content-Type
    - Content-Type: text/html HTML
    - Content-Type: application/JSON JSON
    - Content-Type: application/x-www-form-urlencoded
    application/x-www-form-urlencodedס䎬䌋
    key=value&another_key=another_value

    View Slide

  21. Content Negotiation Fields:
    Accept-Encoding: gzip, deflate
    Content-Encoding: gzip
    ˝ gzip
    ˝ compress
    ˝ deflate
    ˝ identity
    ˝ br

    View Slide

  22. Connection: keep-
    alive, TCP, HoL
    Connection: keep-alive
    TCP
    TLS TLS
    TCP: RFC Transmission Control Protocol (TCP)
    1981 : RFC : TRANSMISSION CONTROL
    PROTOCOL
    1.5-RTT(1 ) -way handshaking)
    TLS 1~2-RTT TCP 0.5-RTT TLS
    HTTP HoL(Head of Line) Blocking:
    keep-alive TCP HTTP

    View Slide

  23. RFC : HTTP Caching
    ☪㍑עם׎םַ

    View Slide

  24. HTTP/
    RFC : HTTP/

    View Slide

  25. HTTP/
    㚺؂׼םַ׆כ
    HTTP/ . RFC
    http https URL

    View Slide

  26. HTTP/
    鵟⟓ס냕鵭⴫סג״ס䄄㜂
    Keep-Alive
    TCP
    HTTP HoL Blocking
    אס☽
    ˝ ئ٭ف٭وشبٖ

    View Slide

  27. 橎⛍ىشر٭
    RFC . . . Request Pseudo-Header Fields
    ٛؠؙتع
    :method
    :authority:
    RFC . . Host and :authority
    :scheme
    :path
    ٝتَ٤ت
    ˝ :status

    View Slide

  28. 橎⛍ىشر٭

    View Slide

  29. فؕػٛך׷׽כ׽׌׾
    : نٝ٭ّ
    HTTP Frame {
    Length (24),
    Type (8),
    Flags (8),
    Reserved (1),
    Stream Identifier (31),
    Frame Payload (!"),
    }
    RFC . HTTP Frames

    View Slide

  30. HEADERS type=0x01
    HEADERS Frame {
    Length (24),
    Type (8) = 0x01,
    Unused Flags (2),
    PRIORITY Flag (1),
    Unused Flag (1),
    PADDED Flag (1),
    END_HEADERS Flag (1),
    Unused Flag (1),
    END_STREAM Flag (1),
    Reserved (1),
    Stream Identifier (31),
    [Pad Length (8)],
    [Exclusive (1)],
    [Stream Dependency (31)],
    [Weight (8)],
    Field Block Fragment (!"),
    Padding (!"2040),
    }

    View Slide

  31. HPACK
    RFC HPACK: Header Compression for HTTP/
    RFC : Appendix B
    62
    RFC : Appendix A
    1 HTTP
    62

    View Slide

  32. ꪐ氳ط٭هٜס╈骰
    Index Header Name Header Value
    :authority
    :method GET
    :method POST
    :path /
    :path /index.html
    :scheme http
    :scheme https
    :status 200

    View Slide

  33. DATA type=0x00
    DATA Frame {
    Length (24),
    Type (8) = 0x00,
    Unused Flags (4),
    PADDED Flag (1),
    Unused Flags (2),
    END_STREAM Flag (1),
    Reserved (1),
    Stream Identifier (31),
    [Pad Length (8)],
    Data (!"),
    Padding (!"2040),
    }

    View Slide

  34. HTTP/
    HTTP/ .
    TCP

    View Slide

  35. TCP HoL Blocking
    HTTP/ TCP
    TCP TCP
    TCP

    View Slide

  36. HTTP/
    RFC : HTTP/

    View Slide

  37. HTTP/
    RFC QUIC: A UDP-Based Multiplexed and Secure
    Transport
    UDP TCP TLS
    RFC HTTP/
    HTTP QUIC
    RFC QPACK: Field Compression for HTTP/

    View Slide

  38. QUIC
    RFC Version-Independent Properties of QUIC
    RFC QUIC: A UDP-Based Multiplexed and Secure
    Transport
    RFC Using TLS to Secure QUIC
    RFC QUIC Loss Detection and Congestion Control

    View Slide

  39. QUIC
    -RTT
    (Early Data)
    PSK( )
    clientearlytra c_secret
    :
    0-RTT RFC :
    The Transport Layer
    Security (TLS) Protocol
    Version .

    View Slide

  40. RFC QPACK: Field Compression for
    HTTP/
    QPACK
    HPACK
    HPACK HoL

    View Slide

  41. ؤؾؠب٘٤ُؕءٝ٭ب٘٤
    ID

    View Slide

  42. ؓغٝتفٛظ٭ب٘٤
    ( )

    View Slide

  43. HTTP/
    UDP TCP+TLS
    HTTPS
    -RTT
    NAT

    View Slide

  44. API
    Application Programming Interface

    View Slide

  45. API
    glibc :
    libdrm :
    Web
    document.getElementById :
    Web : Web API
    REST
    GET https:!"api.twitter.com/1.1/statuses/show.json?
    id=210462857140252672
    This image is licensed under the Creative Commons Attribution-
    Share Alike . International license. Attribution: Shmuel Csaba
    Otto Traian. source https://commons.wikimedia.org/wiki/
    File:Linux_API.svg

    View Slide

  46. API
    LSUDs (Large Set of Unknown Developers)
    API
    SSKDs (Small Set of Known Developers)
    API

    View Slide

  47. Web API

    View Slide

  48. REST
    Representational State Transfer

    View Slide

  49. REST
    HTTP
    https:!"api.github.com/repos/hatena/example/
    issues/1

    View Slide

  50. CRUD
    HTTP
    Create POST
    Read GET
    Update PUT / PATCH
    Delete DELETE

    View Slide

  51. GitHub
    ˝ GET /repos/:owner/:repo/issues/
    comments/:comment_id
    ˝ POST /repos/:owner/:repo/issues/:issue_number/
    comments
    ˝ PATCH /repos/:owner/:repo/issues/
    comments/:comment_id
    ˝ DELETE /repos/:owner/:repo/issues/
    comments/:comment_id

    View Slide

  52. OpenAPI
    Swagger
    REST API

    View Slide

  53. GraphQL

    View Slide

  54. GraphQL
    Facebook

    View Slide

  55. SDL (Schema De nition Language)
    interface Actor {
    login: String!
    }
    type Issue {
    author: Actor
    body: String!
    title: String!
    }
    type Repository {
    issue(number: Int!): Issue
    }
    type Query {
    repository(name: String!, owner: String!): Repository
    }

    View Slide

  56. ؠؙٛ
    query GetFirstIssue {
    repository(name: "Hatena-Intern-2020", owner: "hatena") {
    issue(number: 1) {
    author {
    login
    }
    body
    title
    }
    }
    }

    View Slide

  57. gRPC

    View Slide

  58. gRPC

    View Slide

  59. gRPC over HTTP/
    HTTP/
    HEADERS DATA
    gRPC Web
    Web HTTP/

    View Slide

  60. Protocol Bu ers
    IDL (Interface Description Language)
    .proto
    RPC

    View Slide

  61. syntax = "proto3";
    package account;
    service Account {
    rpc Signup(SignupRequest) returns (SignupReply);
    }
    message SignupRequest {
    string name = 1;
    string password = 2;
    }
    message SignupReply {
    string token = 1;
    }

    View Slide

  62. ْشج٭ة㑔
    message SignupRequest {
    string name = 1;
    string password = 2;
    }
    ϑΟʔϧυܕ ϑΟʔϧυ໊ = ϑΟʔϧυ൪߸;
    1-15 1byte
    19000-19999

    View Slide

  63. ت؜ٚ٭㑔
    double
    float
    int32
    int64
    uint32
    uint64
    sint32
    sint64
    fixed32
    fixed64
    sfixed32
    sfixed64
    bool string
    bytes

    View Slide

  64. repeated Result results = 1;
    map projects = 3;
    oneof test_oneof {
    string name = 4;
    SubMessage sub_message = 9;
    }

    View Slide

  65. message SearchRequest {
    string query = 1;
    int32 page_number = 2;
    int32 result_per_page = 3;
    enum Corpus {
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
    LOCAL = 3;
    NEWS = 4;
    PRODUCTS = 5;
    VIDEO = 6;
    }
    Corpus corpus = 4;
    }

    View Slide

  66. import "google/protobuf/any.proto";
    message Message {
    google.protobuf.Any field = 1;
    }
    ˝ google.protobuf.Empty
    ˝ google.protobuf.Duration
    ˝ google.protobuf.Timestamp

    View Slide

  67. ◍䬵䓪
    ˝ نؔ٭ٜغ樑⺘
    int32 old_field = 6 [deprecated = true];
    reserved 2, 15, 9 to 11;
    reserved "foo", "bar";

    View Slide

  68. ؛وب٘٤
    option go_package = "github.com/hatena/example/pb";
    enum EnumAllowingAlias {
    option allow_alias = true;
    UNKNOWN = 0;
    STARTED = 1;
    RUNNING = 1;
    }
    message Example {
    int32 old_field = 6 [deprecated = true];
    }

    View Slide

  69. ئ٭لتכتعٛ٭ّ
    service RouteGuide {
    rpc GetFeature(Point) returns (Feature) {}
    rpc ListFeatures(Rectangle) returns (stream Feature) {}
    rpc RecordRoute(stream Point) returns (RouteSummary) {}
    rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
    }

    View Slide

  70. gRPC API
    CRUD + List
    ˝ CreateEntity
    ˝ GetEntity
    ˝ UpdateEntity
    ˝ DeleteEntity
    ˝ ListEntities

    View Slide

  71. Cloud Functions
    ˝ CallFunction
    ˝ CreateFunction
    ˝ GetFunction
    ˝ UpdateFunction
    ˝ DeleteFunction
    ˝ ListFunctions

    View Slide

  72. gRPC
    : Hatena-Intern- -Lecture/api/grpc_adventure/

    View Slide

  73. syntax = "proto3";
    option go_package = "./pb";
    package welcome;
    service Welcome {
    rpc Greet(GreetRequest) returns (GreetReply);
    }
    message GreetRequest {
    string name = 1;
    }
    message GreetReply {
    string message = 1;
    }

    View Slide

  74. Protocol Compiler
    Codespaces protoc
    $ # localͰmacͳΒbrew install protobufͳͲͰ
    $ sudo apt update
    $ sudo apt install protobuf-compiler
    $ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
    $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

    View Slide

  75. Protocol Compiler Go
    $ mkdir pb
    $ protoc !"go_out ./pb !"go-grpc_out ./pb !"go_opt=paths=source_relative !"go-grpc_opt paths=source_relative welcome.proto

    View Slide

  76. Protocol Compiler Go
    package main
    import (
    "context"
    "flag"
    "fmt"
    "github.com/hatena/Hatena-Intern-2021-Lecture/api/grpc_adventure/pb"
    "google.golang.org/grpc"
    "google.golang.org/grpc/reflection"
    "log"
    "net"
    "os"
    "os/signal"
    )
    var (
    port = flag.Int("port", 10000, "The server port")
    )

    View Slide

  77. type welcomeServer struct {
    pb.UnimplementedWelcomeServer
    }
    func (s *welcomeServer) Greet(
    ctx context.Context,
    req *pb.GreetRequest
    ) (*pb.GreetReply, error) {
    return &pb.GreetReply{
    Message: fmt.Sprintf("Welcome %s", req.Name),
    }, nil
    }
    func newServer() *welcomeServer {
    return &welcomeServer{}
    }

    View Slide

  78. func main() {
    lis, err !" net.Listen("tcp", fmt.Sprintf(":%d", *port))
    if err !# nil {
    log.Fatalf("failed to listen: %v", err)
    }
    grpcServer !" grpc.NewServer()
    pb.RegisterWelcomeServer(grpcServer, newServer())
    reflection.Register(grpcServer)
    go func() {
    log.Printf("start gRPC server port: %v", *port)
    grpcServer.Serve(lis)
    }()
    quit !" make(chan os.Signal)
    signal.Notify(quit, os.Interrupt)
    !$quit
    log.Printf("stopping gRPC server!!%")
    grpcServer.GracefulStop()
    }

    View Slide

  79. Let s try!: grpcurl

    View Slide

  80. grpcurl
    $ go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest
    $ grpcurl -plaintext localhost:10000 list
    $ grpcurl -plaintext localhost:10000 describe welcome.GreetRequest
    $ grpcurl -plaintext -d '{"name": "John Appleseed"}' localhost:23455 welcome.Welcome/Greet
    usage:
    grpcurl -d '{"id": 1234, "tags": ["foo","bar"]}'
    \
    grpc.server.com:443 my.custom.server.Service/
    Method

    View Slide

  81. ֽ׊ױַ

    View Slide