Scala × AWS αʔόʔαΠυϨγϐ

ࣗݾ঺հ ࿨ా༞հ 1987೥7݄6೔ര஀ ෱Ԭݝ๺۝भࢢখ૔๺۠ খ૔ߴߍ -> ۝भେֶ -> 
 ͍͍ੜ׆ -> Ϋϥεϝιου αʔόαΠυΞϓϦέʔγϣ ϯ։ൃΛओ୲౰

෱Ԭͱ౦ژͷੜ׆ײ ఱਆʹ͍͚͹͍͍ͩͨἧ͏ όεόεΞϯυόε ݪ෇΋ࢹ໺ Ί͠ʢಛʹڕʣ͕͏·͍ ຀ͷ͏ͲΜɺࢿ͞Μ͏ͲΜ Ӻલʹ͍͚͹͍͍ͩͨἧ͏ ి˒ं ΫϩεόΠΫ΋ࢹ໺ ͏·͍Ί͠΋ۚ࣍ୈ ݽಠͷάϧϝͷళʹ͍͚Δ

ฏ௩໌ଠࢠͷັྗ ͋Δ౦ژਓʮ෱Ԭͷ໌ଠࢠ ͸ɺͲ͜ͷϝʔΧʔ΋ಉ͡ ຯ͕͢Δʯ
 ↓ ൿ఻ͷλϨͱ10ஈ֊ͷຯ෇ ͚ɾਏ͞ʹΑΓ൒͹ϠέΫ ιͳ͕Βѹ౗తࠩผԽʹ੒ ޭ ฏ௩໌ଠࢠţŪŖžŖ http://www.hira-tsuka.co.jp/hpgen/HPB/entries/4.html

΋͘͡ ֎෦αʔϏεͱͷ͓෇߹͍ 
 - Web API x ElastiCache ϚωʔδϟɺϫʔΧʔͷεϧʔϓοτΛ͍͋͛ͨ
 - Akka Worker x Amazon ECS ΤϯδχΞɺDDoSʹک͑ͳ͍҆Β͔ͳ຾ΓΛ
 - AWS Lambda x AWS WAF

Web API × ElastiCache

ͱ͋ΔҊ݅ͷ ཁ݅ఆٛձٞʹͯ ؆୯ͳήʔϜͰ ϙΠϯτήοτ εϚʔτϑΥϯΞϓϦ

“ϙΠϯτͷ؅ཧ͚ͩͲɺ
 ͏͕ͪ࡞ͬͨϚΠΫϩαʔϏε͕͋ΔΜͩɻ
 ͥͻ࢖ͬͯ͘Εͳ͍͔ͳʁ” – ಉ͡ձࣾͷผ෦ॺͷϙΠϯτ؅ཧγεςϜΛ࡞ͬͨਓ ※ϑΟΫγϣϯͰ͢

ϙΠϯτ؅ཧγεςϜͷ ϞόΠϧόοΫΤϯυΛ࡞Δ

ϙΠϯτ؅ཧγεςϜͷ ϞόΠϧόοΫΤϯυΛ࡞Δ ͕͜͜ ผαʔϏε

Կ͕՝୊͔ 1೔1ճͷϓϨθϯτͰԾ૝௨՟Λ෇༩͢Δ Ծ૝௨՟αʔϏε͸ႈ౳ੑ͕୲อ͞Ε͍ͯͳ͍ ෳ਺୺຤Ͱಉ࣌ʹϓϨθϯτड͚औΓΛ͢Δͱ…

ElastiCacheΛ࢖͏ Redis / Memcached ߴύϑΥʔϚϯε ϚωʔδυαʔϏε

ElastiCacheΛ࢖͏ Redis / Memcached ߴύϑΥʔϚϯε ϚωʔδυαʔϏε

ϚϧναʔόͰͷϩοΫ setnx ͳ͚Ε͹Ωʔ࡞੒ɺ͋Ε͹ࣦഊ ઌউͪͰsetnxͰ͖ͨϦΫΤετ͚͕ͩσΠϦʔϓ ϨθϯτΛड͚औΕΔ

ຊ౰ʹϩοΫͰ͖ͯΔͷʁ έʔε ࣮ߦ͞ΕΔ ࣮ߦ͞Εͳ͍ ϩοΫ੒ޭ ɾϢʔβҾ౰ ɾsetnx ɾԾ૝௨՟ Ճࢉ ɾϩοΫղআ ɾͳ͠ ϩοΫࣦഊ ɾϢʔβҾ౰ ɾsetnx ɾԾ૝௨՟ Ճࢉ ɾϩοΫղআ

ຊ౰ʹϩοΫͰ͖ͯΔͷʁ έʔε ࣮ߦ͞ΕΔ ࣮ߦ͞Εͳ͍ ϩοΫ੒ޭ ɾϢʔβҾ౰ ɾsetnx ɾԾ૝௨՟ Ճࢉ ɾϩοΫղআ ɾͳ͠ ϩοΫࣦഊ ɾϢʔβҾ౰ ɾsetnx ɾԾ૝௨՟ Ճࢉ ɾϩοΫղআ

Unit Test Code @ Specs2 "locked" should { trait Before { self: CommonBefore => mockLockComponent.lock(anyString) returns Left(new IllegalStateException()) } class Context extends CommonContext with Before "throw DailyPresentConflictError" in new Context { val result = service.present(mockUserName) await(result) must throwA[DailyPresentConflictError] } "no update process" in new Context { val result = service.present(mockUserName) await(result) must throwA[Throwable] there was one(mockUserRepository).findByName(anyString) andThen one(mockLockComponent).lock(anyString) andThen no(mockPointRepository).append(anyString, anyInt) andThen no(mockLockComponent).unlock(anyString) } } DailyPresentServiceSpec.scala

Unit Test Code @ Specs2 "locked" should { trait Before { self: CommonBefore => mockLockComponent.lock(anyString) returns Left(new IllegalStateException()) } class Context extends CommonContext with Before "throw DailyPresentConflictError" in new Context { val result = service.present(mockUserName) await(result) must throwA[DailyPresentConflictError] } "no update process" in new Context { val result = service.present(mockUserName) await(result) must throwA[Throwable] there was one(mockUserRepository).findByName(anyString) andThen one(mockLockComponent).lock(anyString) andThen no(mockPointRepository).append(anyString, anyInt) andThen no(mockLockComponent).unlock(anyString) } } DailyPresentServiceSpec.scala ϩοΫ࣌Ͱ΋ ࣮ߦ͞ΕΔ͸ͣ

Unit Test Code @ Specs2 "locked" should { trait Before { self: CommonBefore => mockLockComponent.lock(anyString) returns Left(new IllegalStateException()) } class Context extends CommonContext with Before "throw DailyPresentConflictError" in new Context { val result = service.present(mockUserName) await(result) must throwA[DailyPresentConflictError] } "no update process" in new Context { val result = service.present(mockUserName) await(result) must throwA[Throwable] there was one(mockUserRepository).findByName(anyString) andThen one(mockLockComponent).lock(anyString) andThen no(mockPointRepository).append(anyString, anyInt) andThen no(mockLockComponent).unlock(anyString) } } DailyPresentServiceSpec.scala ϩοΫ࣌ ࣮ߦ͞Εͳ͍͸ͣ

Akka × Amazon ECS

“͍΍͊Ϣʔβ૿͖͑ͯͨͶʂͱ͜ΖͰ೔࣌Ͱಈ͍ ͍ͯΔ͋ͷόον͊͞ɺࠓ6͔͔࣌ؒͬͯΔ͡Ό Μɺ1ඵʹҰճ͡Όͳͯ͘0.5ඵʹ1ճʹͯ͠Αɻ ͑ʁσϓϩΠ͕ཁΔͷʁ͡Ό͋σϓϩΠ͠Αʂ” – ͝ػݏͳϚωʔδϟ ※ϑΟΫγϣϯͰ͢

όονϫʔΧʔͷঢ়گ 1ඵʹ1ճϙʔϦϯά

όονϫʔΧʔͷঢ়گ 1ඵʹ1ճϙʔϦϯά ϙΠϯτ ֫ಘཤྺ

όονϫʔΧʔͷঢ়گ 1ඵʹ1ճϙʔϦϯά => 0.5ඵʹ1ճʁ ϙΠϯτ ֫ಘཤྺ

σϓϩΠͨ͘͠ͳ͍

ൃ૝Λม͑ͯΈΔ ϙʔϦϯάස౓Λ্͛Δ ཁ͢Δʹεϧʔϓοτ͕େ ͖͘ͳΕ͹OK ॲཧ୆਺Λ૿΍͢

Կ͕՝୊͔ ೔࣌ͷόονॲཧʹ͔͔Δ͕࣌ؒ૿͖͑ͯͨͨΊɺ ࣌ؒΛ୹ॖ͍ͨ͠ ϙʔϦϯάִؒΛখ͘͢͞Δ͜ͱͰॲཧ࣌ؒͷ୹ ॖ͕ݟࠐΊΔ͕ɺमਖ਼σϓϩΠͨ͘͠ͳ͍ EC2ͷ୆਺Λ૿΍͢͜ͱͰεϧʔϓοτΛେ͖͘ ͠Α͏

Կ͕՝୊͔ ೔࣌ͷόονॲཧʹ͔͔Δ͕࣌ؒ૿͖͑ͯͨͨΊɺ ࣌ؒΛ୹ॖ͍ͨ͠ ϙʔϦϯάִؒΛখ͘͢͞Δ͜ͱͰॲཧ࣌ؒͷ୹ ॖ͕ݟࠐΊΔ͕ɺσϓϩΠͨ͘͠ͳ͍ EC2ͷ୆਺Λ૿΍͢͜ͱͰεϧʔϓοτΛେ͖͘ ͠Α͏ EC2ΠϯελϯεΛ૿΍͢ ՝্ֹ͕͕ۚͬͯ͠·͏

Amazon ECSΛ࢖͏ Kubernetes ͷ AWS൛ DockerίϯςφʹରԠͯ͠ ͓ΓɺEC2ΠϯελϯεͰ ίϯςφΫϥελΛ૊Ή͜ ͱ͕Ͱ͖Δ ࣮ߦ͢Δίϯςφͷ਺΍ Auto Scaling ͷઃఆ͕Մೳ

Amazon ECS Λಋೖ͢Δ

Akka Worker ͷ
 Docker Image ࡞੒ lazy val news = project.in(file("./news")) .enablePlugins(PlayScala) .enablePlugins(DockerPlugin) .settings(commonSettings: _*) .settings( name := "ecn-worker-news", version := "0.1.0" ) .settings(libraryDependencies ++= Seq( )) .settings( maintainer in Docker := "Yusuke Wada ", dockerExposedPorts in Docker := Seq(9000, 9443), dockerRepository := Some("cmwadayusuke") ) .dependsOn(library % “test->test;compile->compile") $ ./activator docker:publish build.sbt shell

Akka Worker ͷ Docker Image ࡞੒ lazy val news = project.in(file("./news")) .enablePlugins(PlayScala) .enablePlugins(DockerPlugin) .settings(commonSettings: _*) .settings( name := "ecn-worker-news", version := "0.1.0" ) .settings(libraryDependencies ++= Seq( )) .settings( maintainer in Docker := "Yusuke Wada ", dockerExposedPorts in Docker := Seq(9000, 9443), dockerRepository := Some("cmwadayusuke") ) .dependsOn(library % “test->test;compile->compile") $ ./activator docker:publish build.sbt shell Docker༻ͷ ৘ใΛఆٛ

Akka Worker ͷ Docker Image ࡞੒ lazy val news = project.in(file("./news")) .enablePlugins(PlayScala) .enablePlugins(DockerPlugin) .settings(commonSettings: _*) .settings( name := "ecn-worker-news", version := "0.1.0" ) .settings(libraryDependencies ++= Seq( )) .settings( maintainer in Docker := "Yusuke Wada ", dockerExposedPorts in Docker := Seq(9000, 9443), dockerRepository := Some("cmwadayusuke") ) .dependsOn(library % “test->test;compile->compile") $ ./activator docker:publish build.sbt shell activator (sbt ϥούʔ)ͷ ίϚϯυΛ࣮ߦ

ίϯςφ਺Λ੍ޚ͢Δ αʔϏεͱ͍͏ઃఆ߲໨Ͱ ίϯςφ਺ͷ੍ޚ͕Մೳ Կ୆ͷEC2ΫϥελΛ૊Ή ͔ɺͱ͍͏͜ͱ΋ઃఆՄೳ 1୆ͷEC2্ʹෳ਺ίϯςφ Λ্ཱͪ͛Δ͜ͱͰεϧʔ ϓοτ͕͕͋Δ͸ͣ

ίϯςφ਺vsεϧʔϓοτ SQSͷϝοηʔδΛಡ ΜͰDynamoDBʹॻ͘ ϝοηʔδ਺͸100 100ϝοηʔδফԽͷ εϧʔϓοτΛܭଌ ॲཧϝοηʔδ਺ / ඵ 0 1.25 2.5 3.75 5 1ίϯςφ 2 3 4

ίϯςφ਺vsεϧʔϓοτ SQSͷϝοηʔδΛಡ ΜͰDynamoDBʹॻ͘ ϝοηʔδ਺͸100 100ϝοηʔδফԽͷ εϧʔϓοτΛܭଌ ॲཧϝοηʔδ਺ / ඵ 0 1.25 2.5 3.75 5 1ίϯςφ 2 3 4 ઢܗʹ εϧʔϓοτΞοϓ

ίϯςφ਺vsεϧʔϓοτ SQSͷϝοηʔδΛಡ ΜͰDynamoDBʹॻ͘ ϝοηʔδ਺͸100 100ϝοηʔδফԽͷ εϧʔϓοτΛܭଌ ॲཧϝοηʔδ਺ / ඵ 0 1.25 2.5 3.75 5 1ίϯςφ 2 3 4 EC2Πϯελϯε͸Ұ୆ ՝ֹۚ͸มΘΒͳ͍

AWS LAMBDA × AWS WAF

“DDoSΛ৯ΒͬͨɻೝূγεςϜ͕མͪͨɻ
 Τϯυ͔ΒʮϩάΠϯͰ͖ͳ͍ʯͱ͍͏໰͍߹Θ ͕͖͍ͤͯΔΒ͍͠ɻ͓Εͨͪ͸΋͏ͩΊͩɻ” – ઈ๬ͷϚωʔδϟ ※ϑΟΫγϣϯͰ͢

Կ͕՝୊͔ DDoSΛड͚ͨɻ
 ߈ܸऀ͸User-AgentΛِ૷͍ͯ͠Δ ͻͱ·ͣ Network ACL Ͱ౰֘IPΞυϨεΛःஅ͠ ͕ͨɺ߈ܸऀ͸͙͢ʹΞΫηεݩIPΞυϨεΛม ͑ͯ͘Δ Πλνͬ͜͝Ͱӡ༻ΤϯδχΞ΋ফ໣

“ͱΓ͋͑ͣ AWS WAF ΛೖΕ·͠ΐ͏ɻ
 AWS Lambda ͱ૊Έ߹ΘͤΕ͹
 ࣗಈ๷ޚػߏ͕ͭ͘Εͦ͏Ͱ͢ɻ” –ΤϯδχΞ ※ϑΟΫγϣϯͰ͢

ALB + EC2 ͱ͍͏ߏ੒

AWS WAF ࣗಈઃఆͷ࡞ઓ

ᶃ AWS WAF ͷಋೖ

ᶃ AWS WAF ͷಋೖ AWS WAF ΛηοτΞοϓɻःஅର৅ͷ
 IP Addresses ͸ͻͱ·ۭͣʹ
 →͜͜Λ AWS Lambda Ͱࣗಈઃఆ AWS WAF Λ CloudFront ʹؔ࿈͚ͮΔ

ᶄ fluentdͰΞΫηεϩά

ᶄ fluentdͰΞΫηεϩά nginx X-Forwarded-For Ͱ
 ΞΫηεݩIPΛग़ྗ͢ΔΑ͏ઃఆ fluentd nginxͷΞΫηεϩάΛύʔε ύʔεޙͷϩάΛS3΁Ξοϓ

tg-agent.conf.j2ʢҰ෦ ʣ type s3 s3_bucket "webapi-access-log" path "webapi/access/" s3_object_key_format %{path}%{time_slice}/access_%{index}.% {file_extension} buffer_path /var/log/td-agent/s3/webapi_access time_slice_format %Y/%m/%d/%H time_slice_wait 1m utc buffer_chunk_limit 512k format json include_time_key true time_key log_time tg-agent.conf.j2

tg-agent.conf.j2ʢҰ෦ ʣ type s3 s3_bucket "webapi-access-log" path "webapi/access/" s3_object_key_format %{path}%{time_slice}/access_%{index}.% {file_extension} buffer_path /var/log/td-agent/s3/webapi_access time_slice_format %Y/%m/%d/%H time_slice_wait 1m utc buffer_chunk_limit 512k format json include_time_key true time_key log_time tg-agent.conf.j2 1࣌ؒ΋͘͠͸ 512KB͝ͱʹϩάΛ S3΁

S3όέοτʹΞοϓ͞ΕΔIPΞυϨεϦετ {"host":"5.24.105.49","log_time":"2017-01-16T00:07:40Z"} {"host":"245.52.215.137","log_time":"2017-01-16T00:07:42Z"} {"host":"9.13.105.49","log_time":"2017-01-16T00:07:49Z"} {"host":"4.44.105.49","log_time":"2017-01-16T00:07:50Z"} {“host":"678.124.40.68","log_time":"2017-01-16T00:08:07Z"} {"host":"5.24.105.49","log_time":"2017-01-16T00:07:40Z"} {"host":"245.52.215.137","log_time":"2017-01-16T00:07:42Z"} {"host":"9.13.105.49","log_time":"2017-01-16T00:07:49Z"} {"host":"4.44.105.49","log_time":"2017-01-16T00:07:50Z"} {“host":"678.124.40.68","log_time":"2017-01-16T00:08:07Z"} access.log

ᶅ AWS Lambda ߏங

ᶅ AWS Lambda ߏங S3όέοτ΁ͷϩάసૹΛτϦΨʔʹىಈ͢Δ

Lambda Function ຊମΛॻ͘ def handle(event, context): # Get the object from the event and show its content type bucket = event['Records'][0]['s3']['bucket']['name'] key = urllib.unquote_plus(event['Records'][0]['s3']['object'] ['key'].encode('utf8')) try: # open file, unzip s3.download_file(bucket, key, '/tmp/file.gz') f = gzip.open('/tmp/file.gz', 'rb') content = f.read() f.close # create black list ip_address_list = to_ip_address_list(content.decode('utf-8')) black_list = to_ip_black_list(ip_address_list) main.py

Lambda Function ຊମΛॻ͘ def handle(event, context): # Get the object from the event and show its content type bucket = event['Records'][0]['s3']['bucket']['name'] key = urllib.unquote_plus(event['Records'][0]['s3']['object'] ['key'].encode('utf8')) try: # open file, unzip s3.download_file(bucket, key, '/tmp/file.gz') f = gzip.open('/tmp/file.gz', 'rb') content = f.read() f.close # create black list ip_address_list = to_ip_address_list(content.decode('utf-8')) black_list = to_ip_black_list(ip_address_list) main.py IPΞυϨε͚ͩ நग़

Lambda Function ຊମΛॻ͘ def handle(event, context): # Get the object from the event and show its content type bucket = event['Records'][0]['s3']['bucket']['name'] key = urllib.unquote_plus(event['Records'][0]['s3']['object'] ['key'].encode('utf8')) try: # open file, unzip s3.download_file(bucket, key, '/tmp/file.gz') f = gzip.open('/tmp/file.gz', 'rb') content = f.read() f.close # create black list ip_address_list = to_ip_address_list(content.decode('utf-8')) black_list = to_ip_black_list(ip_address_list) main.py ूܭɾ͖͍͠஋ ൑ఆ

if len(black_list): # create ip set for waf update_ip_set=map(to_ip_set_for_waf, black_list) # create sns settings sns_settings=to_sns_notification_settings(black_list) # waf setting (side-effect function) waf_block(update_ip_set) # notification (side-effect functions) sns_notification(sns_settings) if slack_notification_enabled: slack_notification(black_list) else: pass else: print(u'nothing to do.') pass return black_list except Exception as e: print(e) raise e

if len(black_list): # create ip set for waf update_ip_set=map(to_ip_set_for_waf, black_list) # create sns settings sns_settings=to_sns_notification_settings(black_list) # waf setting (side-effect function) waf_block(update_ip_set) # notification (side-effect functions) sns_notification(sns_settings) if slack_notification_enabled: slack_notification(black_list) else: pass else: print(u'nothing to do.') pass return black_list except Exception as e: print(e) raise e AWS WAF API ίʔϧ

if len(black_list): # create ip set for waf update_ip_set=map(to_ip_set_for_waf, black_list) # create sns settings sns_settings=to_sns_notification_settings(black_list) # waf setting (side-effect function) waf_block(update_ip_set) # notification (side-effect functions) sns_notification(sns_settings) if slack_notification_enabled: slack_notification(black_list) else: pass else: print(u'nothing to do.') pass return black_list except Exception as e: print(e) raise e ௨஌

ಉҰIP͔ΒͷΞΫηε͕Ұఆͷ͖͍͠஋ʢLambda Function ͷ؀ڥม਺ͰઃఆͰ͖ΔΑ͏ʹͨ͠ʣʹ ୡ͢ΔͱϒϥοΫϦετ൑ఆ͠ɺ AWS WAF ͷϒ ϩοΫIPΛࣗಈͰߋ৽͢Δ

λΠϜϥΠϯ 08:00 
 ىচɺΞϥʔτ·ΈΕͷSlack 10:00 
 ग़ࣾɺϚωʔδϟʔ૵ന 12:00 
 ΞʔΩςΫνϟܾఆɺ࣮૷։࢝ 15:00 
 ࣮૷׬ྃɺಈ࡞ݕূ։࢝ 17:00 
 ಈ࡞ݕূ׬ྃɺ
 εςʔδϯά؀ڥʹσϓϩΠ ཌ11:00 ຊ൪ϦϦʔε ཌ18:00 υϠؼ୐

͜Μͳײ͡Ͱ ָ͘͠΍ͬͯ·͢ ࠷৽ͷٕज़Λ࣮Ҋ݅΁ੵۃతʹ౤ೖ͍ͨ͠ʂ AWSɺαʔόʔαΠυʹؔ͢Δ༇շͳΤϯδχΞͱҰॹʹಇ͖ ͍ͨʂ Developers.IOͰϒϩάॻ͖͍ͨʂ εΩϧ σβΠϯ / iOS / Android Java / Scala / Ruby / node.js / Go && AWS ϞόΠϧόοΫΤϯυΤϯδχΞ ― https://classmethod.jp/recruit/jobs/aws-web-app-mobile-engineer/

Scalaͱ͍͏ݴޠʹ͍ͭͯ

Scalaͱ͍͏ݴޠʹ͍ͭͯ