Tweet
Tweet

Slide 1

Slide 1 text

HTTP & Web API #hatenaintern

Slide 2

Slide 2 text

׆ס铺紶ךע HTTP Web API , Web API

Slide 3

Slide 3 text

׆ס铺紶ך⹦׽䣽ֹ׵סꯂ䈋ע㵼׊鷿ֹ HTTP HTTP HTTP TLS TCP, UDP, QUIC API Web API REST GraphQL gRPC

Slide 4

Slide 4 text

HTTP Hypertext Transfer Protocol

Slide 5

Slide 5 text

URL Uniform Resource Locator WHATWG URL Living Standard RFC : Uniform Resource Identi er (URI): Generic Syntax

Slide 6

Slide 6 text

HTTP HTTP/ . HTTP/ . HTTP/ . Google SPDY Google QUIC SPDY HTTP/ HTTP-over-QUIC HTTP/ QUIC / / HTTP/

Slide 7

Slide 7 text

HTTP RFCs - RFC : HTTP Semantics - RFC : HTTP Caching - RFC : HTTP/ . - RFC : HTTP/ - RFC : HTTP/ - RFC : Expect-CT Extension for HTTP - RFC : QPACK: Field Compression for HTTP/ - RFC : Building Protocols with HTTP - RFC : The Proxy-Status HTTP Response Header Field - RFC : The Cache-Status HTTP Response Header Field - RFC : Targeted HTTP Cache Control - RFC : Extensible Prioritization Scheme for HTTP - RFC : Bootstrapping WebSockets with HTTP/ - RFC : Oblivious DNS over HTTPS

Slide 8

Slide 8 text

HTTP/ . RFCs RFC Hypertext Transfer Protocol -- HTTP/ . Obsoleted by: , , , , , RFC Hypertext Transfer Protocol -- HTTP/ . Obsoleted by: , , , , , RFC Hypertext Transfer Protocol (HTTP/ . ): Message Syntax and Routing Obsoleted by: , RFC Hypertext Transfer Protocol (HTTP/ . ): Semantics and Content Obsoleted by: RFC Hypertext Transfer Protocol (HTTP/ . ): Conditional Requests Obsoleted by: RFC Hypertext Transfer Protocol (HTTP/ . ): Range Requests Obsoleted by: RFC Hypertext Transfer Protocol (HTTP/ . ): Caching Obsoleted by: RFC Hypertext Transfer Protocol (HTTP/ . ): Authentication Obsoleted by: RFC HTTP Semantics RFC HTTP Caching RFC HTTP/ .

Slide 9

Slide 9 text

HTTP RFC HTTP Semantics HTTP

Slide 10

Slide 10 text

HTTP ْخشغכذ٭أشع تط٭ذت ىشر٭ RFC Field (RFC Section ) ٍظؔ RFC Content HTTP/ . Content Message Body (RFC Section )

Slide 11

Slide 11 text

Netcat HTTP/ . : : https://www.wdic.org/w/WDIC/

Slide 12

Slide 12 text

TLS . RFC : The Transport Layer Security (TLS) Protocol Version . HTTP HTTP https TLS RFC . . ALPN: RFC Transport Layer Security (TLS) Application-Layer Protocol Negotiation Extension TLS HTTP Clinet Hello(TLS ) - : Encrypted Extensions - :

Slide 13

Slide 13 text

TLS HTTP TLS HTTP : $ echo -e "HEAD / HTTP/1.1\r\nHost: hatenablog.com \r\n" | openssl s_client -ign_eof -connect hatenablog.com:443 ALPN http/ . http/ $ echo | openssl s_client -alpn h2,http/1.1 -connect hatenablog.com:443 http/ . h - IANAIANA ALPN Protocol IDs HTTP IANA Internet Assigned Numbers Authority: Web ID

Slide 14

Slide 14 text

Netcat HTTP/ . HTTP/ .

Slide 15

Slide 15 text

HTTP GET POST PUT HEAD DELETE OPTIONS TRACE CONNECT PATCH

Slide 16

Slide 16 text

GitHub ˝ GET /repos/:owner/:repo/issues/ comments/:comment_id ˝ POST /repos/:owner/:repo/issues/:issue_number/ comments ˝ PATCH /repos/:owner/:repo/issues/ comments/:comment_id ˝ DELETE /repos/:owner/:repo/issues/ comments/:comment_id

Slide 17

Slide 17 text

تط٭ذت车 HTTP/1.1 000 Reason 1xx Informational 2xx Successful 3xx Redirection 4xx Client Error 5xx Server Error

Slide 18

Slide 18 text

ىشر٭ס❆ Host: HTTP/ . User-Agent sec-ch-ua: Content Negotiation Fields: RFC . https://wicg.github.io/ua-client-hints/ https://www.rfc-editor.org/rfc/ rfc #section- . . A client MUST send a Host header field (Section 7.2 of [HTTP]) in all HTTP/1.1 request messages. https://www.rfc-editor.org/rfc/rfc .html#section- . -

Slide 19

Slide 19 text

( ) http://www.example.com ٛؠؙتع GET / HTTP/1.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Upgrade-Insecure-Requests: 1 Host: www.example.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15 Accept-Language: ja Accept-Encoding: gzip, deflate Connection: keep-alive ٝتَ٤ت HTTP/1.1 200 OK Content-Type: text/html; charset=UTF-8 Vary: Accept-Encoding Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT Age: 573292 Content-Encoding: gzip Expires: Fri, 26 Aug 2022 02:22:20 GMT Cache-Control: max-age=604800 Date: Fri, 19 Aug 2022 02:22:20 GMT Content-Length: 648 ETag: "3147526947" Accept-Ranges: bytes Server: ECS (oxr/8315) X-Cache: HIT

Slide 20

Slide 20 text

Content Negotiation Fields: Accept Content-Type - Content-Type: text/html HTML - Content-Type: application/JSON JSON - Content-Type: application/x-www-form-urlencoded application/x-www-form-urlencodedס䎬䌋 key=value&another_key=another_value

Slide 21

Slide 21 text

Content Negotiation Fields: Accept-Encoding: gzip, deflate Content-Encoding: gzip ˝ gzip ˝ compress ˝ deflate ˝ identity ˝ br

Slide 22

Slide 22 text

Connection: keep- alive, TCP, HoL Connection: keep-alive TCP TLS TLS TCP: RFC Transmission Control Protocol (TCP) 1981 : RFC : TRANSMISSION CONTROL PROTOCOL 1.5-RTT(1 ) -way handshaking) TLS 1~2-RTT TCP 0.5-RTT TLS HTTP HoL(Head of Line) Blocking: keep-alive TCP HTTP

Slide 23

Slide 23 text

RFC : HTTP Caching ☪㍑עם׎םַ

Slide 24

Slide 24 text

HTTP/ RFC : HTTP/

Slide 25

Slide 25 text

HTTP/ 㚺؂׼םַ׆כ HTTP/ . RFC http https URL

Slide 26

Slide 26 text

HTTP/ 鵟⟓ס냕鵭⴫סג״ס䄄㜂 Keep-Alive TCP HTTP HoL Blocking אס☽ ˝ ئ٭ف٭وشبٖ

Slide 27

Slide 27 text

橎⛍ىشر٭ RFC . . . Request Pseudo-Header Fields ٛؠؙتع :method :authority: RFC . . Host and :authority :scheme :path ٝتَ٤ت ˝ :status

Slide 28

Slide 28 text

橎⛍ىشر٭

Slide 29

Slide 29 text

فؕػٛך׷׽כ׽׌׾ : نٝ٭ّ HTTP Frame { Length (24), Type (8), Flags (8), Reserved (1), Stream Identifier (31), Frame Payload (!"), } RFC . HTTP Frames

Slide 30

Slide 30 text

HEADERS type=0x01 HEADERS Frame { Length (24), Type (8) = 0x01, Unused Flags (2), PRIORITY Flag (1), Unused Flag (1), PADDED Flag (1), END_HEADERS Flag (1), Unused Flag (1), END_STREAM Flag (1), Reserved (1), Stream Identifier (31), [Pad Length (8)], [Exclusive (1)], [Stream Dependency (31)], [Weight (8)], Field Block Fragment (!"), Padding (!"2040), }

Slide 31

Slide 31 text

HPACK RFC HPACK: Header Compression for HTTP/ RFC : Appendix B 62 RFC : Appendix A 1 HTTP 62

Slide 32

Slide 32 text

ꪐ氳ط٭هٜס╈骰 Index Header Name Header Value :authority :method GET :method POST :path / :path /index.html :scheme http :scheme https :status 200

Slide 33

Slide 33 text

DATA type=0x00 DATA Frame { Length (24), Type (8) = 0x00, Unused Flags (4), PADDED Flag (1), Unused Flags (2), END_STREAM Flag (1), Reserved (1), Stream Identifier (31), [Pad Length (8)], Data (!"), Padding (!"2040), }

Slide 34

Slide 34 text

HTTP/ HTTP/ . TCP

Slide 35

Slide 35 text

TCP HoL Blocking HTTP/ TCP TCP TCP TCP

Slide 36

Slide 36 text

HTTP/ RFC : HTTP/

Slide 37

Slide 37 text

HTTP/ RFC QUIC: A UDP-Based Multiplexed and Secure Transport UDP TCP TLS RFC HTTP/ HTTP QUIC RFC QPACK: Field Compression for HTTP/

Slide 38

Slide 38 text

QUIC RFC Version-Independent Properties of QUIC RFC QUIC: A UDP-Based Multiplexed and Secure Transport RFC Using TLS to Secure QUIC RFC QUIC Loss Detection and Congestion Control

Slide 39

Slide 39 text

QUIC -RTT (Early Data) PSK( ) clientearlytra c_secret : 0-RTT RFC : The Transport Layer Security (TLS) Protocol Version .

Slide 40

Slide 40 text

RFC QPACK: Field Compression for HTTP/ QPACK HPACK HPACK HoL

Slide 41

Slide 41 text

ؤؾؠب٘٤ُؕءٝ٭ب٘٤ ID

Slide 42

Slide 42 text

ؓغٝتفٛظ٭ب٘٤ ( )

Slide 43

Slide 43 text

HTTP/ UDP TCP+TLS HTTPS -RTT NAT

Slide 44

Slide 44 text

API Application Programming Interface

Slide 45

Slide 45 text

API glibc : libdrm : Web document.getElementById : Web : Web API REST GET https:!"api.twitter.com/1.1/statuses/show.json? id=210462857140252672 This image is licensed under the Creative Commons Attribution- Share Alike . International license. Attribution: Shmuel Csaba Otto Traian. source https://commons.wikimedia.org/wiki/ File:Linux_API.svg

Slide 46

Slide 46 text

API LSUDs (Large Set of Unknown Developers) API SSKDs (Small Set of Known Developers) API

Slide 47

Slide 47 text

Web API

Slide 48

Slide 48 text

REST Representational State Transfer

Slide 49

Slide 49 text

REST HTTP https:!"api.github.com/repos/hatena/example/ issues/1

Slide 50

Slide 50 text

CRUD HTTP Create POST Read GET Update PUT / PATCH Delete DELETE

Slide 51

Slide 51 text

GitHub ˝ GET /repos/:owner/:repo/issues/ comments/:comment_id ˝ POST /repos/:owner/:repo/issues/:issue_number/ comments ˝ PATCH /repos/:owner/:repo/issues/ comments/:comment_id ˝ DELETE /repos/:owner/:repo/issues/ comments/:comment_id

Slide 52

Slide 52 text

OpenAPI Swagger REST API

Slide 53

Slide 53 text

GraphQL

Slide 54

Slide 54 text

GraphQL Facebook

Slide 55

Slide 55 text

SDL (Schema De nition Language) interface Actor { login: String! } type Issue { author: Actor body: String! title: String! } type Repository { issue(number: Int!): Issue } type Query { repository(name: String!, owner: String!): Repository }

Slide 56

Slide 56 text

ؠؙٛ query GetFirstIssue { repository(name: "Hatena-Intern-2020", owner: "hatena") { issue(number: 1) { author { login } body title } } }

Slide 57

Slide 57 text

gRPC

Slide 58

Slide 58 text

gRPC

Slide 59

Slide 59 text

gRPC over HTTP/ HTTP/ HEADERS DATA gRPC Web Web HTTP/

Slide 60

Slide 60 text

Protocol Bu ers IDL (Interface Description Language) .proto RPC

Slide 61

Slide 61 text

syntax = "proto3"; package account; service Account { rpc Signup(SignupRequest) returns (SignupReply); } message SignupRequest { string name = 1; string password = 2; } message SignupReply { string token = 1; }

Slide 62

Slide 62 text

ْشج٭ة㑔 message SignupRequest { string name = 1; string password = 2; } ϑΟʔϧυܕ ϑΟʔϧυ໊ = ϑΟʔϧυ൪߸; 1-15 1byte 19000-19999

Slide 63

Slide 63 text

ت؜ٚ٭㑔 double float int32 int64 uint32 uint64 sint32 sint64 fixed32 fixed64 sfixed32 sfixed64 bool string bytes

Slide 64

Slide 64 text

repeated Result results = 1; map projects = 3; oneof test_oneof { string name = 4; SubMessage sub_message = 9; }

Slide 65

Slide 65 text

message SearchRequest { string query = 1; int32 page_number = 2; int32 result_per_page = 3; enum Corpus { UNIVERSAL = 0; WEB = 1; IMAGES = 2; LOCAL = 3; NEWS = 4; PRODUCTS = 5; VIDEO = 6; } Corpus corpus = 4; }

Slide 66

Slide 66 text

import "google/protobuf/any.proto"; message Message { google.protobuf.Any field = 1; } ˝ google.protobuf.Empty ˝ google.protobuf.Duration ˝ google.protobuf.Timestamp

Slide 67

Slide 67 text

◍䬵䓪 ˝ نؔ٭ٜغ樑⺘ int32 old_field = 6 [deprecated = true]; reserved 2, 15, 9 to 11; reserved "foo", "bar";

Slide 68

Slide 68 text

؛وب٘٤ option go_package = "github.com/hatena/example/pb"; enum EnumAllowingAlias { option allow_alias = true; UNKNOWN = 0; STARTED = 1; RUNNING = 1; } message Example { int32 old_field = 6 [deprecated = true]; }

Slide 69

Slide 69 text

ئ٭لتכتعٛ٭ّ service RouteGuide { rpc GetFeature(Point) returns (Feature) {} rpc ListFeatures(Rectangle) returns (stream Feature) {} rpc RecordRoute(stream Point) returns (RouteSummary) {} rpc RouteChat(stream RouteNote) returns (stream RouteNote) {} }

Slide 70

Slide 70 text

gRPC API CRUD + List ˝ CreateEntity ˝ GetEntity ˝ UpdateEntity ˝ DeleteEntity ˝ ListEntities

Slide 71

Slide 71 text

Cloud Functions ˝ CallFunction ˝ CreateFunction ˝ GetFunction ˝ UpdateFunction ˝ DeleteFunction ˝ ListFunctions

Slide 72

Slide 72 text

gRPC : Hatena-Intern- -Lecture/api/grpc_adventure/

Slide 73

Slide 73 text

syntax = "proto3"; option go_package = "./pb"; package welcome; service Welcome { rpc Greet(GreetRequest) returns (GreetReply); } message GreetRequest { string name = 1; } message GreetReply { string message = 1; }

Slide 74

Slide 74 text

Protocol Compiler Codespaces protoc $ # localͰmacͳΒbrew install protobufͳͲͰ $ sudo apt update $ sudo apt install protobuf-compiler $ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Slide 75

Slide 75 text

Protocol Compiler Go $ mkdir pb $ protoc !"go_out ./pb !"go-grpc_out ./pb !"go_opt=paths=source_relative !"go-grpc_opt paths=source_relative welcome.proto

Slide 76

Slide 76 text

Protocol Compiler Go package main import ( "context" "flag" "fmt" "github.com/hatena/Hatena-Intern-2021-Lecture/api/grpc_adventure/pb" "google.golang.org/grpc" "google.golang.org/grpc/reflection" "log" "net" "os" "os/signal" ) var ( port = flag.Int("port", 10000, "The server port") )

Slide 77

Slide 77 text

type welcomeServer struct { pb.UnimplementedWelcomeServer } func (s *welcomeServer) Greet( ctx context.Context, req *pb.GreetRequest ) (*pb.GreetReply, error) { return &pb.GreetReply{ Message: fmt.Sprintf("Welcome %s", req.Name), }, nil } func newServer() *welcomeServer { return &welcomeServer{} }

Slide 78

Slide 78 text

func main() { lis, err !" net.Listen("tcp", fmt.Sprintf(":%d", *port)) if err !# nil { log.Fatalf("failed to listen: %v", err) } grpcServer !" grpc.NewServer() pb.RegisterWelcomeServer(grpcServer, newServer()) reflection.Register(grpcServer) go func() { log.Printf("start gRPC server port: %v", *port) grpcServer.Serve(lis) }() quit !" make(chan os.Signal) signal.Notify(quit, os.Interrupt) !$quit log.Printf("stopping gRPC server!!%") grpcServer.GracefulStop() }

Slide 79

Slide 79 text

Let s try!: grpcurl

Slide 80

Slide 80 text

grpcurl $ go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest $ grpcurl -plaintext localhost:10000 list $ grpcurl -plaintext localhost:10000 describe welcome.GreetRequest $ grpcurl -plaintext -d '{"name": "John Appleseed"}' localhost:23455 welcome.Welcome/Greet usage: grpcurl -d '{"id": 1234, "tags": ["foo","bar"]}' \ grpc.server.com:443 my.custom.server.Service/ Method

Slide 81

Slide 81 text

ֽ׊ױַ