Oracle Cloud Infrastructure (OCI) の技術説明資料、ロード・バランサーの詳細編 (Level 200) です。
セッション永続性、仮想ホストによるルーティング、URIパス・ベース・ルーティング、SSLの利用とSSLターミネーションの設定、ロードバランサー で取得できるメトリックなどについて解説しています。
ロード・バランサ 詳細Load Balancer Level 200Oracle Cloud Infrastructure2023 1
View Slide
https://oracle-japan.github.io/ocidocs/services/networking/load-balancer-100/• OCI 1 VCN• OCI (FLB)(NLB) 2• – TCP, HTTP/1.0, HTTP/1.1, HTTP/2, WebSocket• SSL SSL SSL• FLB 7•• ( IP )•• TCP ( 4) HTTP ( 7)Copyright © 2022, Oracle and/or its affiliates2
•• IP• SSL SSL••• WAF(Web Application Firewall) Web•••Copyright © 2022, Oracle and/or its affiliates3
Copyright © 2022, Oracle and/or its affiliates4タイムアウトとキープ・アライブ設定
FLBCopyright © 2022, Oracle and/or its affiliates5clients Load BalancerBackendServersHTTP message/TCP segmentHTTP 2003way handshake3way handshakeFin/ACKGETHTTP message/TCP segmentHTTP message/TCP segmentGETHTTP 200HTTP message/TCP segmentFin/ACK= 65 10,000 ( )(C)(C')( )= TCP : 300 , 7200= HTTP : 60 , 7200(A)(A')(B) (B')= 300 ( )(D)(D')( )= 310(E)(E')(A)(A')(D')(D)(C)(C') (E')(E)(A-A') receive timer(B-B') send timer(B)(B')https://docs.oracle.com/ja-jp/iaas/Content/Balance/Reference/connectionreuse.htm
65 FLB FIN TCP1 0.000000000 10.0.1.25 -> 10.0.2.228 TCP 74 43744 > http [SYN] Seq=0 Win=65340 Len=0MSS=1485 SACK_PERM=1 TSval=3551202664 TSecr=0 WS=1282 0.002493772 10.0.2.228 -> 10.0.1.25 TCP 74 http > 43744 [SYN, ACK] Seq=0 Ack=1 Win=26844Len=0 MSS=8960 SACK_PERM=1 TSval=4158075881 TSecr=3551202664 WS=10243 0.000013286 10.0.1.25 -> 10.0.2.228 TCP 66 43744 > http [ACK] Seq=1 Ack=1 Win=65408 Len=0TSval=3551202667 TSecr=4158075881( )13 0.000005690 10.0.1.25 -> 10.0.2.228 TCP 66 43744 > http [ACK] Seq=241 Ack=1663 Win=64128Len=0 TSval=3551202681 TSecr=415807589614 65.001322203 10.0.2.228 -> 10.0.1.25 TCP 66 http > 43744 [FIN, ACK] Seq=1663 Ack=241Win=27648 Len=0 TSval=4158140891 TSecr=355120268115 0.040574411 10.0.1.25 -> 10.0.2.228 TCP 66 43744 > http [ACK] Seq=241 Ack=1664 Win=64128Len=0 TSval=3551267723 TSecr=415814089116 40.054097462 10.0.1.25 -> 10.0.2.228 TCP 66 43744 > http [FIN, ACK] Seq=241 Ack=1664Win=64128 Len=0 TSval=3551307777 TSecr=415814089117 0.001380630 10.0.2.228 -> 10.0.1.25 TCP 66 http > 43744 [ACK] Seq=1664 Ack=242 Win=27648Len=0 TSval=4158180983 TSecr=3551307777– FLBCopyright © 2022, Oracle and/or its affiliates63-way handshake65FIN LB->
Copyright © 2022, Oracle and/or its affiliates7バックエンド・サーバーでのクライアントのIPアドレスの識別
IPIPIP …• HTTP HTTP X-Forwarded-• TCPIPIPCopyright © 2022, Oracle and/or its affiliates8: 1.1.1.1: 192.168.1.2Src 1.1.1.1Dst 2.2.2.2VIP : 2.2.2.2192.168.1.254Src 192.168.1.254Dst 192.168.1.2Src 192.168.1.2Dst 192.168.1.254Src 2.2.2.2Dst 1.1.1.1X-Forwarded-ForIPIPX-Forwarded-HostX-Forwarded-PortX-Forwarded-ProtoX-Real-IP X-Forwarded-For IP
Copyright © 2022, Oracle and/or its affiliates9SSL通信への対応SSL Handling
3 SSLCopyright © 2022, Oracle and/or its affiliates10SSL • SSL•• SSL• Web• FLB HTTPSSSL • SSL SSL•• SSL• FLB HTTPSSSL• SSLSSL• SSL• FLB TCP NLBHTTPS( )HTTP( )HTTPS( )HTTPS( )HTTPS( )
OCI (Certificates)Copyright © 2022, Oracle and/or its affiliates11OCI SSL (FLB)→ OCIOCI (OCI Certificates Service)• (Certificate) (CA) CA• CA/ /CA•••
SSLCopyright © 2022, Oracle and/or its affiliates12
RSA(CSR)SSL - 1. (CSR)Copyright © 2022, Oracle and/or its affiliates13$ openssl req -new -key MyKey.key -out MyCSR.csrYou are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:WALocality Name (eg, city) []:RedmondOrganization Name (eg, company) [Internet Widgits Pty Ltd]:OracleOrganizational Unit Name (eg, section) []:OCICommon Name (e.g. server FQDN or YOUR name) []:*.example.orgEmail Address []:[email protected]Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:$ openssl genrsa -out MyKey.key 2048Generating RSA private key, 2048 bit long modulus.........................+++.....................+++e is 65537 (0x10001)
SSL - 2.Copyright © 2022, Oracle and/or its affiliates14$ openssl x509 -req -days 365 -in MyCSR.csr -signkey MyKey.key -out ExampleCert.crtSignature oksubject=/C=US/ST=WA/L=Redmond/O=Oracle/OU=OCI/CN=*.example.org/[email protected]Getting Private key$ openssl x509 -in exampleCert.crt -noout -textCertificate:Data:Version: 1 (0x0)Serial Number:fa:98:bb:ae:1e:19:4d:a3Signature Algorithm: sha256WithRSAEncryptionIssuer: C=US, ST=WA, L=Redmond, O=Oracle, OU=OCI, CN=*.example.org/[email protected]ValidityNot Before: Jun 6 18:34:41 2018 GMTNot After : Jun 6 18:34:41 2019 GMTSubject: C=US, ST=WA, L=Redmond, O=Oracle, OU=OCI, CN=*.example.org/[email protected]Subject Public Key Info:Public Key Algorithm: rsaEncryptionPublic-Key: (2048 bit)Modulus:00:c0:63:f1:aa:d8:98:b1:01:0f:9f:fa:71:6a:9a:f1:05:9d:d6:84:01:88:8d:51:6e:b5:d4:fa:5e:fb:95:f7:ac:ed:07:11:bf:89:85:4b:39:70:71:9e:7e:cd:ba:24:96:65:d9:41:69:d1:05:f7:1a:a2:43:29:7a:6b:de:11:e7:2b:6f:95:ee:04:de:2b:23:b1:0b:a6:a2:76:8f:40:42:50:1e:d8:2a:16:2c:d5:97:2b:
PEMSSL - 3.Copyright © 2022, Oracle and/or its affiliates15$ openssl x509 -in ExampleCert.crt -out ExampleCert.pem -outform PEM
SSL - 4. 443Copyright © 2022, Oracle and/or its affiliates16
SSLCopyright © 2022, Oracle and/or its affiliates17※
Copyright © 2022, Oracle and/or its affiliates18セッション永続性機能を利⽤したバックエンド・サーバーの固定Using Session Persistence
Web:Cookie Cookie persistence※CookieCopyright © 2022, Oracle and/or its affiliates19CookieClient AClient BcookieClient AClient BFLB HTTP/HTTPS
Cookie 21. Cookie• Cookie• CookieOK2. Cookie• Cookie•• CookieCookie(+α)Cookie CookieFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates20
CookieCookieClient Load Balancer Backend Server12200 OK3200 OKSet-cookie:X-Oracle-BMC-LBS-Route=yyy45Set-cookie:X-Oracle-BMC-LBS-Route=yyy1.2.3.X-Oracle-BMC-LBS-RouteCookie4. Cookie5. CookieFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates21
CookieCookieClient Load Balancer Backend Server12200 OKSet-cookie: sessionid=xxx3200 OKSet-cookie:sessionid=xxxSet-cookie:X-Oracle-BMC-LBS-Route=yyy45Set-cookie:sessionid=xxxSet-cookie:X-Oracle-BMC-LBS-Route=yyy1.2. Set-cookieCookie3.CookieCookie+X-Oracle-BMC-LBS-RouteCookie4. Cookie5. Cookie: FLB X-Oracle-VMC-LBS-Route Cookie2 CookieFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates22
Q. ?• (Fallback)• :• : HTTP 502Q. ?• cookie(match-all cookie)• cookie Cookie• Cookie ( ) ()FAQCopyright © 2022, Oracle and/or its affiliates23FLB HTTP/HTTPS
Q. ?• Cookie• Cookie• CookieQ. Cookie ?• FLB Cookie• CookieCookie FLB Cookie• Cookie : Expires, Max-Age, SameSite, Secure, HTTP-Only• : FLB CookieFLBFAQCopyright © 2022, Oracle and/or its affiliates24FLB HTTP/HTTPS
Cookie- (FLB) HTTP/HTTPS× - (FLB) TCP× - (NLB)Cookie HTTP Cookie FLB HTTP(S)Cookie IP (FLB) 2 (NLB)IP( )CookieCopyright © 2022, Oracle and/or its affiliates25Proxy1.1.1.1 2.2.2.2 3.3.3.31.1.1.1 2.2.2.2 3.3.3.35.5.5.5NLB 2 FLB TCPIP
Copyright © 2022, Oracle and/or its affiliates26リクエスト・ルーティングを⽤いた⾼度なルーティングRequest Routing
• IP• FLB HTTP HTTPS• : 7• FLB TCP NLB• 31. -2. URI Cookie• 2021 3 ( )3. - HTTP(S)Copyright © 2022, Oracle and/or its affiliates27FLB HTTP/HTTPS
(IP : xxx.xxx.xxx.xxx)12(IP + + )• 1 IP• 1• 1• 1– 1.1 (TCP/80)2 (TCP/80)www.example1.comwww.example2.comFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates28
• FQDN (e.g. app.example.com)• (*.example.com)• (app.example.*)(Catch-all)• ※•• 16• 16– 1.Copyright © 2022, Oracle and/or its affiliates29FLB HTTP/HTTPS
– 2.1 (IP + + )• 縮• URL URI( )• HTTP• Cookie• 1• 1• 1(IP : xxx.xxx.xxx.xxx)(TCP/80)121user-agent = mobile2= /admin/FLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates30
(IP : xxx.xxx.xxx.xxx)(Port : 80)122022/3/24 [CN-64788]1 (IP + + )URIURI• 1• 1• 1– 2'. (~2022/3/24)1/pages/2/video/FLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates31
• (EXACT_MATCH)• URI : ^$• (FORCE_LONGEST_PREFIX_MATCH)• URI : .*• (PREFIX_MATCH)• URI : ^.*• (SUFFIX_MATCH)• URI : .*$– 2'. (~2022/3/24)Copyright © 2022, Oracle and/or its affiliates32FLB HTTP/HTTPS
1. (EXACT_MATCH)2.(FORCE_LONGEST_PREFIX_MATCH)3. (PREFIX_MATCH)(SUFFIX_MATCH)– 2'. (~2022/3/24)Copyright © 2022, Oracle and/or its affiliates33FLB HTTP/HTTPS
• 20• 1– 2'. (~2022/3/24)Copyright © 2022, Oracle and/or its affiliates34FLB HTTP/HTTPS
URI Cookie: 1 A B C 3 A•• foo.com → B• bar.com → C• URI• /biz → B ( )• /baz → C ( )Copyright © 2022, Oracle and/or its affiliates35URLhttp://foo.com Bhttp://foo.com/biz Bhttp://foo.com/baz Chttp://bar.com Chttp://bar.com/biz Bhttp://bar.com/baz Chttp://example.com Ahttp://example.com/biz Bhttp://example.com/baz CFLB HTTP/HTTPS
LB-APPS-1ipAddress = ip-1name = foo-and-bar-LBBackendSet[[]Listeners[]PathRouteSet[]Listener-defaultport = 554serverName = nulldefaultBackendSet = foo.BES-1certificate = foo-CertificatePathRouteSet = PRS-1Listener-fooport = 443serverName = *.foo.comdefaultBackendSet = foo.BES-1certificate = foo-CertificatePathRouteSet = nullListener-barport = 443serverName = *.bar.comdefaultBackendSet = bar.BES-1certificate = foo-CertificatePathRouteSet = PRS-1foo-CertificateCN = www.foo.comSAN = *.foo.combar-CertificateCN = www.bar.comSAN = *.bar.comfoo-BES-1[] – foo-Backends-1bar-BES-1[] – bar-Backends-1bar-BES-2[] – bar-Backends-2PRS-1[] – bar-path-route-rulesExact Matches/biz à bar-BES-1/baz à bar-BES-2URLhttp://example.com foo-BES-1http://example.com/biz bar-BES-1http://example.com/baz bar-BES-2http://foo.com foo-BES-1http://bar.com bar-BES-1http://bar.com/baz bar-BES-2FLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates36
HTTP(S)• IP• GET HEAD POST HTTPURL• URI HTTPURI• HTTP(80)HTTPS(443) SSL•HTTP ( )•• HTTP( )• [x-xss-protection] [x-content-type]HTTP• HTTP 8KB64KB• . _– 3.FLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates37
HTTP50•• [Server]• Web• [x-xss-protection] [x-content-type]FLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates38
•• IP• GET HEAD POST HTTPFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates39
URLHTTP URL• HTTP HTTPSFLB HTTP/HTTPSCopyright © 2022, Oracle and/or its affiliates40
?• Cookie• Cookie• CookieFAQCopyright © 2022, Oracle and/or its affiliates41FLB HTTP/HTTPS
Copyright © 2022, Oracle and/or its affiliates42WAF(Web Application Firewall)、Webアプリケーション・アクセラレーションとの連携
FLB WAF VCN• OCI Web Application Firewall WAF L7 DDOSSQL Web• VCN / WAF• VCNWAF• Blog Announcing Oracle Cloud Infrastructure WAF Protection on Flexible Load Balancers• WAFWeb Application FirewallCopyright © 2022, Oracle and/or its affiliates432021/10/27OCIVCNDRGWAF WAFProduct Price MetricInstance $5 Instance Per MonthRequests $0.61,000,000 IncomingRequests Per Month
HTTP• 100MB• Accept-Encoding•[Network] Web Application AccelerationCopyright © 2022, Oracle and/or its affiliates442022/6/15“cacheStatus”HITMISShttps://blogs.oracle.com/oracle4engineer/post/ja-intro-waa
Copyright © 2022, Oracle and/or its affiliates45ロード・バランサのロギングLogging for Load Balancers
(NLB) OCI• 2••※ (NLB) /• /• OCI( : 10GB/ )/ OCICopyright © 2022, Oracle and/or its affiliates46LoggingVCNFlow LogsOCI
https://docs.oracle.com/ja-jp/iaas/Content/Network/Concepts/vcn_flow_logs.htmVCNOCI• VCN (or VCN )→• NIC IP• OCIIPVTAPVCN ( )ACCEPT TCP 172.21.2.185 Port 43360 → 129.146.13.236 Port 443 Bytes 10515 Packets 19ACCEPT TCP 129.146.13.236 Port 443 → 172.21.2.185 Port 43360 Bytes 5548 Packets 14VCNCopyright © 2022, Oracle and/or its affiliates47VCNFlow Logs
https://docs.oracle.com/ja-jp/iaas/Content/Network/Tasks/vtap.htm(FLB) VTAP(Virtual Test Access Point)(NLB)•( )• IDS•FLB VTAPVTAP (Virtual Test Access Point)Copyright © 2022, Oracle and/or its affiliates48( )
Copyright © 2022, Oracle and/or its affiliates49ロード・バランサのメトリックMetrics for Load Balancers
OCI (FLB, NLB)OCI••• OCI• OCI Events•→ OCI OCI2Copyright © 2022, Oracle and/or its affiliates50
https://docs.oracle.com/ja-jp/iaas/Content/Balance/Reference/loadbalancermetrics.htmOCI ( : oci_lbaas) 3 (lbComonent)• lbComponent = loadbalancer• SSL (11 )• lbComponent = listener• HTTP (HTTP ) (8 )• lbComponent = backendset•HTTP (HTTP ) (19 ): Load BalancerCopyright © 2022, Oracle and/or its affiliates51
2lbHostId(PeakBandwidth)lbHostId=ae41d2132 lbHostId=f6d9d18 2lbHostId( grouping().sum())MQLPeakBandwidth[1m]{resourceId="ocid1…"}.grouping().sum()Tips : lbHostIdCopyright © 2022, Oracle and/or its affiliates52
PeakBandwidth[1m]{lbName=""}.grouping().sum()LBHOST(UnHealthyBackendServers[1m].mean() > 0).groupBy(backendSetName).mean() > 0.55 HTTP 200 95%HttpResponses200[5m]{lbname=""}.groupBy(listenerName).sum() /HttpResponses[5m]{lbname=""}.groupBy(listenerName).sum() < 0.95Copyright © 2022, Oracle and/or its affiliates53
Copyright © 2022, Oracle and/or its affiliates54トラブル時の対応ガイドラインTroubleshooting Guidelines
• 3• [ ] [ ]••••••• VCN•Copyright © 2022, Oracle and/or its affiliates55
•• IP• SSL SSL••• WAF(Web Application Firewall) Web•••Copyright © 2022, Oracle and/or its affiliates56
–• https://docs.oracle.com/ja-jp/iaas/Content/Balance/Concepts/balanceoverview.htm–• https://docs.oracle.com/ja-jp/iaas/Content/NetworkLoadBalancer/overview.htm– Web• https://oracle-japan.github.io/ocitutorials/intermediates/using-load-balancer/Copyright © 2022, Oracle and/or its affiliates57
Oracle Cloud Infrastructure• https://docs.oracle.com/ja-jp/iaas/Content/home.htm - ( )• https://docs.cloud.oracle.com/iaas/api/ - API• https://docs.oracle.com/ja-jp/iaas/Content/General/Reference/aqswhitepapers.htm -• https://docs.cloud.oracle.com/iaas/releasenotes/ -• https://docs.oracle.com/ja-jp/iaas/Content/knownissues.htm - (Known Issues)• https://docs.oracle.com/ja-jp/iaas/Content/General/Reference/graphicsfordiagrams.htm - OCI(PPT SVG Visio )Oracle Cloud InfrastructureCopyright © 2022, Oracle and/or its affiliates58
Oracle Cloud Infrastructure• https://oracle-japan.github.io/ocidocs/- Oracle Cloud Infrastructure• https://oracle-japan.github.io/ocitutorials/Oracle• https://www.oracle.com/search/events/( Filter Locations -> Asia Pacific -> Japan )Oracle Cloud Infrastructure – General Forum ( )• https://cloudcustomerconnect.oracle.com/resources/9c8fa8f96f/summaryOracle Cloud InfrastructureCopyright © 2022, Oracle and/or its affiliates59
Thank YouCopyright © 2022, Oracle and/or its affiliates60
ocise
sattosan
seike460
bonjiri_niwa
hirosatogamo
foursue
soachr
okunokentaro
ahana
carta_engineering
line_developers_tw
miura55
yeseniaperezcruz
swwweet
chrislema
sferik
aleyda
deanohume
rmw
jennybc
colly
addyosmani
caitiem20
skipperchong