AWSサポートエンジニア怒濤のLTチャレンジ

Transcript

1. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWSサポートエンジニア 怒涛のLTチャレンジ Oct 8, 2022 - JAWS DAYS 2022 AWS Support, Amazon Web Services G.K.
2. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWSサポートから来ました！ Sr. Cloud Support Engineer AWS Support , Amazon Web Services Japan G.K. Oct 8, 2022 - JAWS DAYS 2022 小武 三博(Kotake, Mitsuhiro)
3. © 2022, Amazon Web Services, Inc. or its affiliates. 小武 三博(Kotake, Mitsuhiro) AWS 2015 AWS 関心領域: OS(Linux, Windows) 、仮想化技術 SME(Subject Matter Expert): EC2 Core, EC2 Linux 3 @_mitkot
4. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWS 4
5. © 2022, Amazon Web Services, Inc. or its affiliates. 5 AWS Support Non- technical Technical Cloud Support Engineer (CSE) Cloud Support Associate (CSA)
6. © 2022, Amazon Web Services, Inc. or its affiliates. 6 AWS Support：3 Basic Developer ü web ü Business ü Web ü ü 24/365 Enterprise ü Web ü ü 24/365 Technical Account Manager
7. © 2022, Amazon Web Services, Inc. or its affiliates. 24/365 TOKYO DUBLIN SEATTLE
8. © 2022, Amazon Web Services, Inc. or its affiliates. AWS https://www.youtube.com/watch?v=hE8PRjbfyic Q: ? AWS AWS • AWS • • API AWS SDK • AWS • ( : OS E ) AWS • • • https://aws.amazon.com/jp/premiumsupport/faqs/
9. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWS LT Sr. Cloud Support Engineer AWS
10. © 2022, Amazon Web Services, Inc. or its affiliates. - 2016 2 Sr. Cloud Support Engineer Windows Profile SME ( Subject Matter Expert ) EC2 Windows, VM Import/Export, EC2 Core Support Operations - Windows Support Operations - VM Import/Export EC2Rescue Developer
11. © 2022, Amazon Web Services, Inc. or its affiliates. 7:00 ( 21:00 ) Web/Mail ( Slack )
12. © 2022, Amazon Web Services, Inc. or its affiliates. - xxx - xxx
13. © 2022, Amazon Web Services, Inc. or its affiliates.
14. © 2022, Amazon Web Services, Inc. or its affiliates. 1,000
15. © 2022, Amazon Web Services, Inc. or its affiliates. - - ( Support Operations ) - / FAQ
16. © 2022, Amazon Web Services, Inc. or its affiliates. AWS
17. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. Munechika Sumikawa Principal Support Engineer
18. © 2022, Amazon Web Services, Inc. or its affiliates. • 2013 • § SME: Amazon CloudFront Amazon VPC • AWS IAM 18
19. © 2022, Amazon Web Services, Inc. or its affiliates. 19
20. © 2022, Amazon Web Services, Inc. or its affiliates. 20
21. © 2022, Amazon Web Services, Inc. or its affiliates. AWS • • • 21
22. © 2022, Amazon Web Services, Inc. or its affiliates. • § § § AWS 22
23. © 2022, Amazon Web Services, Inc. or its affiliates. (1) ×: ELB り: https://myweb***.com/index.html (ELB ) • § : 23
24. © 2022, Amazon Web Services, Inc. or its affiliates. (2) ×: り: URL *** https://docs.aws.amazon.com/***/*** • • 24
25. © 2022, Amazon Web Services, Inc. or its affiliates. ! • https://aws.amazon.com/jp/premiumsupport/tech-support- guidelines/ • § ご自身で課題を解決した場合でも、解決された状況をサポートケース上にご 記載いただき、クローズすることをお願いします § サポートにてナレッジを集めることで、過去の実績として、課題解決の有効 な手がかりとなる可能性があります 25
26. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWS Well-Architected Hiroki Tateno Sr. Cloud Support Engineer AWS Support RDBMS
27. © 2022, Amazon Web Services, Inc. or its affiliates. • • Database Profile • SME: RDS Core, RDS MySQL • 2017/07 AWS • Java 27
28. © 2022, Amazon Web Services, Inc. or its affiliates. 28
29. © 2022, Amazon Web Services, Inc. or its affiliates. 29 RDS
30. © 2022, Amazon Web Services, Inc. or its affiliates. 30
31. © 2022, Amazon Web Services, Inc. or its affiliates. • RDBMS • AWS Well-Architected • • 31 AWS Well-Architected
32. © 2022, Amazon Web Services, Inc. or its affiliates. • 0 / 100% • • 築 A: AWS RDS 99.95% MultiAZ 99.99% • § • 32
33. © 2022, Amazon Web Services, Inc. or its affiliates. ( ) • RDS • • 33 「Design for Failure」
34. © 2022, Amazon Web Services, Inc. or its affiliates. Failure …… • 1: SYN § • 2: § • 3: § § 34
35. © 2022, Amazon Web Services, Inc. or its affiliates. • § DB / EC2 / OS / Network / . § • 35
36. © 2022, Amazon Web Services, Inc. or its affiliates. 36
37. © 2022, Amazon Web Services, Inc. or its affiliates. • Elastic Load Balancing EC2 Auto Scaling § § DB • § § • 37
38. © 2022, Amazon Web Services, Inc. or its affiliates. • : • : 5 • : • : or • 38
39. © 2022, Amazon Web Services, Inc. or its affiliates. AWS Well-Architected Labs • https://www.wellarchitectedlabs.com/reliability/300_labs/300_te sting_for_resiliency_of_ec2_rds_and_s3/5_failure_injection_rds/ • RDS Failure Injection 39
40. © 2022, Amazon Web Services, Inc. or its affiliates. Amazon RDS • • • • • 40
41. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. AWS ⾒ Amazon S3, Amazon CloudFront SME(Subject Matter Expert) Web AWS Japan AWS AWS • Amazon S3 • Amazon Athena
42. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. 42
43. © 2022, Amazon Web Services, Inc. or its affiliates. • Amazon CloudFront Amazon S3 Web • Web Amazon CloudWatch Synthetics • 1 • HTTP 200 494 • 43
44. © 2022, Amazon Web Services, Inc. or its affiliates. Amazon CloudWatch Synthetics • Canary API • Canary AWS Lambda • Python Node.js • Canary Blueprint 44
45. © 2022, Amazon Web Services, Inc. or its affiliates. 45 AWS Cloud Synthetics Amazon CloudFront Amazon Simple Storage Service (Amazon S3) Amazon CloudWatch AWS Certificate Manager (ACM) AWS WAF Lambda function
46. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. 46
47. © 2022, Amazon Web Services, Inc. or its affiliates. ID • AWS CloudFront X-Amz-Cf-Id ID • • 494 Request Header Fields Too Large 47
48. © 2022, Amazon Web Services, Inc. or its affiliates. • CloudFront User-Agent 20,480 • CloudFront 20,480 • Athena 48 1 25095 2 24917 3 24828 4 24739 SELECT distinct length(user_agent) FROM "default"."cloudfront_logs order by length(user_agent) desc limit 10
49. © 2022, Amazon Web Services, Inc. or its affiliates. • 4xx • User-Agent • • 49
50. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. 50
51. © 2022, Amazon Web Services, Inc. or its affiliates. • Canary API Canary Blueprints Python http.client • • User-Agent 51
52. © 2022, Amazon Web Services, Inc. or its affiliates. 52 def verify_request(method, url, post_data=None, headers={}): parsed_url = urllib.parse.urlparse(url) user_agent = str(syn_webdriver.get_canary_user_agent_string()) if "User-Agent" in headers: headers["User-Agent"] = " ".join([user_agent, headers["User-Agent"]]) else: headers["User-Agent"] = "{}".format(user_agent) logger.info("Making request with Method: '%s' URL: %s: Data: %s Headers: %s" % ( method, url, json.dumps(post_data), json.dumps(headers))) if parsed_url.scheme == "https": conn = http.client.HTTPSConnection(parsed_url.hostname, parsed_url.port) else: conn = http.client.HTTPConnection(parsed_url.hostname, parsed_url.port) conn.set_debuglevel(1) conn.request(method, url, headers=headers)
53. © 2022, Amazon Web Services, Inc. or its affiliates. Python • Python • • • https://docs.python-guide.org/writing/gotchas/#mutable- default-arguments 53
54. © 2022, Amazon Web Services, Inc. or its affiliates. 54 def verify_request(headers={}): user_agent = "test" if "User-Agent" in headers: headers["User-Agent"] = " ".join([user_agent, headers["User-Agent"]]) else: headers["User-Agent"] = "{}".format(user_agent) print("headers: {}".format(headers)) with futures.ThreadPoolExecutor(max_workers=8) as executor: for i in range(0, 32): executor.submit(verify_request) $ python test.py {'User-Agent': 'test'} {'User-Agent': 'test test'} {'User-Agent': 'test test test'} {'User-Agent': 'test test test test'} {'User-Agent': 'test test test test test test'} {'User-Agent': 'test test test test test test test test'} {'User-Agent': 'test test test test test'} {'User-Agent': 'test test test test test test test test test'} {'User-Agent': 'test test test test test test test test test test test test'}
55. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. 32 55
56. © 2022, Amazon Web Services, Inc. or its affiliates. AWS Lambda • AWS Lambda • • 56
57. © 2022, Amazon Web Services, Inc. or its affiliates. AWS Lambda 57 counter = [] def lambda_handler(event, context): counter.append(len(counter)) return {'body': 'Count: %s' % counter} { "body": "Count: [0]" } { "body": "Count: [0, 1]" } { "body": "Count: [0, 1, 2]" } { "body": "Count: [0, 1, 2, 3]" } { "body": "Count: [0, 1, 2, 3, 4]" }
58. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. 1 2 58
59. © 2022, Amazon Web Services, Inc. or its affiliates. • AWS Lambda Python User Agent Amazon CloudFront • • • AWS 59
60. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Confidential and Trademark. AWS Glue ! Tomohiro Tanaka, Sr. Cloud Support Engineer JAWS DAYS 2022, Oct. 8, 2022
61. © 2022, Amazon Web Services, Inc. or its affiliates. 61 • Tomohiro Tanaka ( ), Sr. Cloud Support Engineer • 2017 4 AWS • 2019 1 Dublin, Ireland • 2021 4 • AWS Glue AWS IoT SME (Subject Matter Expert) • Glue, Lake Formation, IoT, Greengrass • BigData Open source Apache Spark, Apache Iceberg !
62. © 2022, Amazon Web Services, Inc. or its affiliates. 62 AWS Glue
63. © 2022, Amazon Web Services, Inc. or its affiliates. 63 AWS Glue ? AWS Glue S3 Amazon S3, Amazon Redshift, Amazon RDS, Amazon DynamoDB Amazon Kinesis Data Streams AWS Apache Spark ETL Job Python Python Shell Job … Data Catalog Crawler
64. © 2022, Amazon Web Services, Inc. or its affiliates. 64 共著でAWS Glueの本を出版しました!! • 通称 ”Glue本” (自称かもしれません) • 出版社: Packt Publications • 共著の方々: • Noritaka Sekiyama • Vishal Pathak • Subramanya Vajaraya • Albert Quiroga • Ishan Gaur • Reviewers: • Akira Ajisaka, Keerthi Chadalavada
65. © 2022, Amazon Web Services, Inc. or its affiliates. 65 Glue ? 400 ? Glue ? • BigData (Data lake, lakehouse ) • Glue • Glue ( / ) • (Lake Formation, ML ) Glue !!
66. © 2022, Amazon Web Services, Inc. or its affiliates. 66 – 😊😊😊 2021 5 ( ) !! !! !! !! 2021 7 Chapter Chapter 5 Chapter 10 Chapter 5 1st draft 2021.11.11 Chapter 10 2022.2.9 !!!!! 2021 9-10 ( ) (Ch.5 20 1 0.3 ) 1 !!
67. © 2022, Amazon Web Services, Inc. or its affiliates. 67 – … 😊😊😊 2021 10 1 1 …( ) 2021 11 Ch.5, 1st draft !! Ch.10 ( …) Thanks for your great work! 2022 1-2 Ch.5, 2nd draft Ch.10, 1st draft Ch.5 & Ch.10 50 Ch.5 2nd draft 2 ? ( ) 2022 4 Ch.5 draft Ch.10 draft 1 1 !! 2022 7 Ch.5/Ch.10 ? 2022 9 !!!
68. © 2022, Amazon Web Services, Inc. or its affiliates. 68 • !! ( …) • !! ( ) • ( ) ( ) • ! • Draft Packt Reviewer !
69. © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. … Thank you! 5 Glue Ebook ! : Twitter @moomindani !! (Glue !) :