Open source application and Ecosystem on Serverless Framework / Serverlessconf Tokyo 2017

Open source application and Ecosystem on Serverless Framework / Serverlessconf Tokyo 2017

69b93af68320a590f607c296e8edff73?s=128

Ken’ichiro Oyama

November 03, 2017
Tweet

Transcript

  1. Open source application and Ecosystem on Serverless Framework Kenʼichiro Oyama

    Fusic Co.,Ltd. 2017.11.3 1 Serverlessconf Tokyo 2017
  2. Who 2 Serverlessconf Tokyo 2017

  3. k1LoW   Kenʼichiro Oyama   @k1LoW   Fusic Co.,Ltd.  

    Engineer / TechLead / KIBAN-Unit   Creator of awspec   https://github.com/k1LoW/awspec   Favorite Serverless Framework Plugins   serverless-prune-plugin / serverless-s3-remover 3 Serverlessconf Tokyo 2017
  4. Agenda   Open source application using Serverless Framework   Serverless

    Ninja Tips   Ecosystem on Serverless Framework 4 Serverlessconf Tokyo 2017
  5. Open source application using Serverless Framework 5 Serverlessconf Tokyo 2017

  6. Serverless Architecture and Serverless OSS   Currently, in Japan, there

    are many case studies of serverless architecture.   I believe, “serverless” becomes not only “the design concept” but also “one of OSS implementation methods”.   Because, Serverless Framework is a very powerful tool. 6 Serverlessconf Tokyo 2017
  7. My “Serverless OSS” using Serverless Framework 7 Serverlessconf Tokyo 2017

  8. faultline 8 Serverlessconf Tokyo 2017

  9. faultline   Error tracking tool on AWS managed services.  

    Similar SaaS: Airbrake, Bugsnag, Rollbar, etc   https://github.com/faultline 9 Serverlessconf Tokyo 2017
  10. faultline architecture 10 Serverlessconf Tokyo 2017

  11. In Fusic 11 Serverlessconf Tokyo 2017 13 Projects 3 Languages

    68k Errors 1.26GB Raw logs
  12. 9< Months 12 Serverlessconf Tokyo 2017

  13. 13 Serverlessconf Tokyo 2017 1 Down 0 Recovery ※DynamoDB Capacity

    ※faultline have NO recovery server
  14. utsusemi 14 Serverlessconf Tokyo 2017

  15. utsusemi   A tool to generate a “API-ready” “updatable” static

    website by crawling the original site.   https://github.com/k1LoW/utsusemi 15 Serverlessconf Tokyo 2017
  16. utsusemi architecture 16 Serverlessconf Tokyo 2017

  17. After all, What is Serverless Framework?   CFn++   Programable

    CFn   Service as Code for me 17 Serverlessconf Tokyo 2017
  18. Application Deployment with Serverless Framework 18 Serverlessconf Tokyo 2017 Cloud

    Service Resource Middleware Application code App on IaaS Serverless App
  19. Application Deployment with Serverless Framework 19 Serverlessconf Tokyo 2017 Cloud

    Service Resource Middleware Application code App on IaaS Serverless App
  20. Application Deployment with Serverless Framework 20 Serverlessconf Tokyo 2017 Cloud

    Service Resource Middleware Application code Cloud Service Resource Application code App on IaaS Serverless App
  21. Application Deployment with Serverless Framework 21 Serverlessconf Tokyo 2017 Cloud

    Service Resource Middleware Application code Cloud Service Resource Application code App on IaaS Serverless App
  22. Abstract as middleware or service using Serverless Framework   Applications

    built on manged services can abstract as one middleware or one service.   ex. diamond is already one time series DB service.   ex. faultline is already one error tracking service.   Serverless Framework provides `sls deploy` like `make install`   one command bulid ≒ one command deployment 22 Serverlessconf Tokyo 2017
  23. faultline architecture 23 Serverlessconf Tokyo 2017

  24. faultline as service 24 Serverlessconf Tokyo 2017

  25. We can provide open source middleware and service with Serverless

    Framework !! 25 Serverlessconf Tokyo 2017
  26. Serverless⚡Ninja Tips 26 Serverlessconf Tokyo 2017

  27. Serverless Ninja Tips   Serverless Architecture is not general architecture.

      ex. Managed services (FaaS / Object Storage / NoSQL)   Need new tips   I have ⚡⚡⚡⚡⚡tips.   POST with config   Reversed Timestamp ID   Instant Job Queue   S3 Object Tagging   Env Sync <- New! 27 Serverlessconf Tokyo 2017
  28. POST with config   Include settings in API request parameters

    wherever possible   ex. Slack incoming webhook URL / channel   ex. GitHub Token   Do NOT manage states as much as you can   If it is SECRET, use KMS encryption 28 Serverlessconf Tokyo 2017
  29. Reversed Timestamp ID   Implement "SELECT * FROM posts ORDER

    BY create_at DESC" on S3   (Math.pow(2, 53) - 1) – unixtime   Number.MAX_SAFE_INTEGER – unixtime   /projects/{project}/errors/{message}/ occurrences/{reversedUnixtime}.json 29 Serverlessconf Tokyo 2017
  30. Instant Job Queue   When invoking functions to fan-out recursively

      ex. Crawling web 30 Serverlessconf Tokyo 2017
  31. S3 Object Tagging   Use AWS “S3 Object tagging” for

    KVS   ex.   Metadata of file ( s3://bucket/path/to/index.html )   ex. Content-Type / Etag   Details of user profile ( s3://bucket/group-id/user-id.jpg )   ex. Name / address / telno   Labels of issue data ( s3://bucket/project-id/issue.json ) 31 Serverlessconf Tokyo 2017
  32. Env Sync   Synchronize the environment variable of the deployment

    environment and the environment variable of the function execution environment.   The Twelve-Factor App 32 Serverlessconf Tokyo 2017  provider:   name: aws   environment: # env of AWS Lambda function   STAGE: ${env:STAGE, ’develop’} # use env of the deployment environment
  33. Ecosystem on Serverless Framework 33 Serverlessconf Tokyo 2017

  34. Serverless user   Applications built on manged services can abstract

    as one middleware or one service.   Serverless Framework provides `sls deploy` like `make install`   I think that more people will USE open source “serverless middleware” and “serverless service”.   Like Redmine for Ruby on Rails   But, user does not know Serverless Framework 34 Serverlessconf Tokyo 2017
  35. Need ecosystem like a 'Deploy to Heroku' Button 35 Serverlessconf

    Tokyo 2017
  36. like a 'Deploy to Heroku' Button   “Deploy with Serverless”

    project ⚡⚡⚡   by Rafal Wilinski   https://github.com/RafalWilinski/deploy-with-serverless   “.slsdeploy.yml” project [PoC]   by Me   https://github.com/k1LoW/slsdeploy 36 Serverlessconf Tokyo 2017
  37. .slsdeploy.yml   Put .slsdeploy.yml to your sls app GitHub repo

      Define environment variables required for .slsdeploy.yml   Make your sls app correspond to “Env Sync” 37  ---  environment:   UTSUSEMI_TARGET_HOST:   description: Crawl target host   type: text   UTSUSEMI_WORKER_PROCESS:   description: Crawl worker process   type: number   default: 2   UTSUSEMI_BUCKET_NAME:
  38. DEMO ( If I have time ) 38 Serverlessconf Tokyo

    2017
  39. User simple deploy ecosystem!! 39 Serverlessconf Tokyo 2017

  40. Conclusion   Applications built on manged services can abstract as

    one middleware or one service.   Serverless Framework provides `sls deploy` like `make install`   If a mechanism that makes it easier to deploy is provided, the user can obtain a “serverless middleware” and “serverless service” more easily.   I would like to see more open source “serverless middleware” and “serverless service” 40 Serverlessconf Tokyo 2017
  41. 41 Weʼre Hiring Engineers! > https://fusic.github.io Thank you! Serverlessconf Tokyo

    2017