Upgrade to Pro — share decks privately, control downloads, hide ads and more …

投稿監視マイクロサービスの継続的なデプロイと構成変更の実現手段の紹介

 投稿監視マイクロサービスの継続的なデプロイと構成変更の実現手段の紹介

投稿監視マイクロサービスの継続的なデプロイと構成変更の実現手段の紹介

takuya542

June 16, 2017
Tweet

More Decks by takuya542

Other Decks in Programming

Transcript

  1. Copyright © 2009-2017 eureka, inc. All rights reserved. Takuya Onda

    / eureka, Inc. 2017-06-16 @Minami Aoyama Night #3 ౤ߘ؂ࢹϚΠΫϩαʔϏεͷܧଓతͳσϓϩΠͱ ߏ੒มߋͷ࣮ݱखஈͷ঺հ
  2. Agenda ▪ 1. About Us ▪ 2. Introduction of our

    Text Moderation System ▪ 3. Continuous Delivery ▪ 4. Continuous Configuration Management
  3. 2017.01 2016.01 2016.09 2015.05 2017 - 2015 2014 - 2008

    Our History "Couples" gets over 4 million downloads. "Pairs" gets over 5 million user and 32 million matchings. Junya Ishibashi is elected CEO, succeeding Yu Akasaka, who retained chairman title. IAC/InterActiveCorp acquires eureka. 2014.06 2014.05 2013.10 2012.10 2011.09 2010.08 2008.11 "Pairs" gets over 1 million user and 3.3 million matchings. eureka launches communication app for couple, "Couples". eureka launches online-dating service "Pairs" in Taiwan. eureka launches online-dating service "Pairs" in Japan. eureka starts smartphone and Facebook app services. eureka starts blog marketing services.
 eureka is founded in Tokyo.
  4. Tradeoff between Safety and Cost Expense ▪ Service Safety –

    Brand image can be ruined easily – Once we lose user’s trust, service growth will be disrupted ▪ Cost – The more messages we process, the more people 
 we need
  5. Technical Overview ▪ Machine Leaning – Supervised Leaning By Manual

    Text Moderation Data ▪ Microservice – Independent Deployment Pipeline ▪ Cloud – AWS ALB / EC2 / Aurora – CodeBuild / CodeDeploy / CodePipeline
  6. Result ▪ Cost Reduction – Massive labor cost turned into

    much smaller AWS operational cost ▪ Service KPI – Still under observation
  7. Why we need Continuous Delivery ▪ Reducing Time to Value

    – You write code. You commit it. Then what? – Your code doesn’t produce any benefit until you launch ▪ Service Quality – Dev : It’s not my code! Somebody caused it! ▪ Quick Feedback – We don’t know what happens until we launch new feature
  8. Automation & Reproducibility ▪ No Manual Deployment – It should

    be automated – It should be reproduced – Your build it, You run it
  9. Branch Based Deployment ▪ Keep Master-branch always be releasable –

    Check In = Deploy ( Should be Automated ) – Rollback = Revert & Deploy – Don’t leave disruptive commits on master branch
  10. Reliable Staging Environment ▪ Test passed on staging = release

    immediately – Twelve-Factor of App ( App should not care where it runs ) – It should be same middleware / Orchestration process
  11. System Architecture ▪ Fully automated – Deploy when you merge

    master-branch – Same as Staging (When you merge stage-branch) ▪ No centralized build / deploy server – Use AWS Managed Service – No need to monitor Build / Deploy pipeline modules
  12. Result ▪ Fully automated deployment process – Check In =

    Deploy – No environmental differences ( if you check-in stage branch,It`s shipped to staging environment) ▪ No centralized build/deploy server – Use AWS Managed Service
  13. Why we need Continuous Configuration Management ▪ Reduce security disruption

    – Need frequent Middleware Update ▪ Accelerate our Development – Dev : How long do we have to keep using PHP5.xx??
  14. Code-based management & reusability ▪ No Manual Configuration – It

    should be managed on Code – It should be standardized – Is your ansible-role really reusable??
  15. Scheduled(automated) & frequent apply ▪ How to fight against config-changes

    phobia? – When you apply your playbook last time?? – Can you really trust your playbook? – More frequently you apply, less scared you feel
  16. Treat servers as cattle ▪ Be disposable – Since we

    use AWS, Why not? – Be Stateless ( Logs / Cache / Etc,,) – Don’t modify server outside of config-management- pipeline
  17. System Architecture ( Packer Image Transition ) • env :

    prod • domain : stage-hoge.eureka.jp • Nginx : • log_prefix : stage-hoge • upstream : hoge • mackerel_check_metrics : • nginx • port : 80 • log : /var/log/nginx/xxx
  18. System Architecture ( Packer Image Transition ) ▪ Fully codenized

    machine-image (AMI) – Testable ( serverspec ) ▪ Provisioning same middleware – Control configuration diff by variables – Exɿservice / region / env
  19. System Architecture ( apply new configuration ) ▪ Terraforming &

    Blue Green – Modify launch-configuration and rotate EC2 ▪ Scheduled rotation – Running EC2 always be launched from clean AMI – Avoid to leave manual config modification ▪ Not Fully Automated – Need to apply by Dev – Pipelining between Packer-build result and Terraform configuration (AMI ID)
  20. Summary ▪ Text Moderation System – Use Machine Leaning –

    Build as Micro Service – Build on Cloud ▪ Deploy and Configuration Management – Why we need Continues Improvement – Continuous Delivery – Continuous Configuration Management