Serverless Server Side Swift

Yuki Takei(@noppoMan722)
Builderscon Tokyo 2017

Transcript

1. Serverless Server Side Swift Yuki Takei(@noppoMan722) Builderscon Tokyo 2017
2. Who am I? Yuki Takei Freelance Software Engineer • Job: Backend, Infrastructure, Data mining/analysis, iOS • Organizer of Tokyo Server Side Swift Meetup noppoMan noppoman722
5. What is Swift? • Released from Apple on 2014 • For making app on Apple Platforms such as iOS • (Strong)static typing programming language • Swift4 will be released soon.
6. Language Features • No Virtual Machine and Interpreter • Adopting Reference counting to manage memory • Safe • Faster • Expressive
7. “Go vs Node vs Rust vs Swift” sited from https://grigio.org/go-vs-node-vs-rust-vs-swift/
8. 1. Web Development with Swift 2. Serverless × Swift 3. Introduction of Hexaville 4. Create Practical Web App with Hexaville 5. Conclusion Objectives
9. 1. Web Development with Swift
10. In December 2015 1. Web Development with Swift
12. 1. Web Development with Swift
13. 1. Web Development with Swift Swift works on Linux
14. That’s mean, We can create web apps with Swift It’s no longer only for the Apple Platforms 1. Web Development with Swift
16. Why do we web development with Swift? • Isomorphic development (iOS and Backend) • Faster/Easy to deploy (No VM and Interpreter) • Can reduce run-time errors with Strong Static Typing and Null Safety • Easy to handle parallelism and asynchronously (GCD) 1. Web Development with Swift
17. Swift has been used in iOS Production for 3 years 1. Web Development with Swift
18. 1. Development Environment 2. Hello World on Linux 3. Web Servers/Frameworks 4. About Deployment How do we web development with Swift? 1. Web Development with Swift
19. 1. Development Environment Swift Package Manager Xcode Mac 1. Web Development with Swift
20. The Key is Swift Package Manager > 1. Development Environment 1. Web Development with Swift
21. Swift Package Manager (SPM) • SPM is a official package management system for Swift. • Post Carthage and CocoaPods • Use to configure application settings and manage dependencies (It’s like npm, sbt) • Also includes cross platform build system (llbuild) > 1. Development Environment 1. Web Development with Swift
22. HexavilleBench$ swift package OVERVIEW: Perform operations on Swift packages USAGE: swift package [options] subcommand SUBCOMMANDS: clean Delete build artifacts describe Describe the current package dump-package Print parsed Package.swift as JSON edit Put a package in editable mode generate-xcodeproj Generates an Xcode project init Initialize a new package reset Reset the complete cache/build directory resolve Resolve package dependencies show-dependencies Print the resolved dependency graph tools-version Manipulate tools version of the current package unedit Remove a package from editable mode update Update package dependencies swift package command > 1. Development Environment 1. Web Development with Swift
23. HexavilleBench$ swift build -h OVERVIEW: Build sources into binary products USAGE: swift build [options] OPTIONS: --build-path Specify build/cache directory [default: ./.build] --build-tests Build the both source and test targets --configuration, -c Build with configuration (debug|release) [default: debug] --disable-prefetching --disable-sandbox Disable using the sandbox when executing subprocesses --enable-prefetching --package-path Change working directory before any other operation --show-bin-path Print the binary output path --verbose, -v Increase verbosity of informational output -Xcc Pass flag through to all C compiler invocations -Xcxx Pass flag through to all C++ compiler invocations -Xlinker Pass flag through to all linker invocations -Xswiftc Pass flag through to all Swift compiler invocations --help Display available options > 1. Development Environment 1. Web Development with Swift swift build command
24. Generate Initial CLI(Server) Project with SPM /Users/yuki/Desktop % mkdir HelloSPM && cd HelloSPM /Users/yuki/Desktop % swift package init —type executable Creating executable package: HelloSPM Creating Package.swift Creating .gitignore Creating Sources/ Creating Sources/main.swift Creating Tests/ /Users/yuki/Desktop/HelloSPM % tree . ├── Package.swift ├── Sources │ └── main.swift └── Tests 2 directories, 2 files > 1. Development Environment 1. Web Development with Swift
25. swift build and execute yuki$ cd HelloSPM HelloSPM$ swift build HelloSPM$ ./.build/debug/HelloSPM Hello, world! > 1. Development Environment 1. Web Development with Swift
26. 2. Hello World on Linux 1. Web Development with Swift
27. 2. Hello World on Linux 1. Web Development with Swift
28. Linux version of Swift can download from swift.org > 2. Hello World on Linux 1. Web Development with Swift
29. Dockerfile > 2. Hello World on Linux 1. Web Development with Swift
30. Build and Run /Users/yuki/Desktop/HelloSPM % docker build -t hello-spm . ……… Removing intermediate container 2a1e07e3bb29 Successfully built 2b7cd7f22a1b Successfully tagged hello-spm: Build > 2. Hello World on Linux 1. Web Development with Swift
31. Build and Run /Users/yuki/Desktop/HelloSPM % docker build -t hello-spm . ……… Removing intermediate container 2a1e07e3bb29 Successfully built 2b7cd7f22a1b Successfully tagged hello-spm: Build Run /Users/yuki/Desktop/HelloSPM % docker run -it -t hello-spm Hello, world! > 2. Hello World on Linux 1. Web Development with Swift
32. 3. Web Frameworks > 2. Hello World on Linux 1. Web Development with Swift
33. • There are several Frameworks (Fullstack - Minimal) • HTTP Server is not provided in Swift Standard Lib • So almost WebFrameworks has Web Server • Covering ORM / Database Driver, JWT and Template engines etc… 3. Web Frameworks > 3. Web Frameworks 1. Web Development with Swift
34. Major Frameworks In Swift > 3. Web Frameworks 1. Web Development with Swift
35. - Vapor - > 3. Web Frameworks 1. Web Development with Swift • Developer: Vapor • Type: HTTPServer and FullStack MVC Framework • GitHub Star: 10K • Documentation: Very GOOD > 3. Vapor
36. Controller > 3. Web Frameworks 1. Web Development with Swift > 3. Vapor
37. Models > 3. Web Frameworks 1. Web Development with Swift > 3. Vapor
38. > 3. Web Frameworks 1. Web Development with Swift > 3. Vapor Learning
39. - Kitura - 1. Web Development with Swift • Developer: IBM • Type: HTTPServer and Lightweight Web Framework • GitHub Star: 6K • Documentation: GOOD? > 3. Kitura
40. Example Code of Kitura HTTP Server > 3. Web Frameworks 1. Web Development with Swift
41. > 3. Web Frameworks 1. Web Development with Swift Actually… I tried to create WAF for Swift a year ago…
42. > 3. Web Frameworks 1. Web Development with Swift Stop developing due to unpopular
43. 4. About Deployment 1. Web Development with Swift
44. • Swift doesn’t require VM and Interpreter • Just uploading ELF (required runtime libraries or static link) • Working well with Ubuntu and RedHat Distributions • Working well with ECS, Kubernetes and other Container Services 4. About Deployment 1. Web Development with Swift
45. Works on well known Cloud Platforms > 4. About Deployment 1. Web Development with Swift
46. Swift web development is getting Real! Web Development with Swift
47. But, bit early…. right? Web Development with Swift
48. I have good news! Web Development with Swift
49. You can small start Swift Web Development with Serverless Computing Web Development with Swift
50. 2. Serverless × Swift
51. ”Serverless computing is a cloud computing execution model in which the cloud provider dynamically manages the allocation of machine resources, and bills based on the actual amount of resources consumed by an application, rather than billing based on pre- purchased units of capacity.” Serverless computing from wikipedia ~Serverless computing~ 2. Serverless × Swift
52. Advantages of Serverless Computing • Lower cost • Free from Server Management / Scaling • Mobile engineers can easily develop server side • Faster innovation 2. Serverless × Swift
54. 2. Serverless × Swift This time I’ll focus on AWS's Lambda and API Gateway to tell you about Serverless and compatibility of Swift
55. Lambda 2. Serverless × Swift • AWS’s Serverless Service • Triggered by Internal Events • Doesn’t have HTTP Endpoint
56. 2. Serverless × Swift APIGateway • Gateway and Frontend for the AWS Services • Provides HTTPS Endpoint • Can choose lambda as backend service • Client -> API Gateway -> Lambda
57. Anyways, Can we use Swift on AWS Lambda? 2. Serverless × Swift
58. 2. Serverless × Swift Environment of AWS Lambda • Amazon Linux • Only supports 64Bit Binary • Supporting Languages: Node.js, Python, Java and .NET Core
59. 2. Serverless × Swift Environment of AWS Lambda • Amazon Linux • Only supports 64Bit Binary • Supporting Languages: Node.js, Python, Java and .NET Core
60. W h a t t h e f ̋ c k !?
61. Calm down!
62. Swift become ELF(64bit) after building
63. Swift code ELF Lambda Upload Build Swift works on Lambda with following flow 2. Serverless × Swift
64. • Need Linux to build swift • Different development and work flow from conventional web development • Must execute Swift from supported languages • Anyways, It’s bother to deploy APIGateway + Lambda But… 2. Serverless × Swift
65. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy 2. Serverless × Swift ᶆ’ mapping
66. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with Uploaded Zip URL ᶆ Resource and Function Mapping Request ᶇ Deploy Request Mapping Function and Resource Serverless × Swift It’s hard….
67. I thought we can automate such bother things and focus to develop web apps with Swift 2. Serverless × Swift
68. Then I tried to develop ….. 2. Serverless × Swift
70. 3. Introduction of Hexaville
71. • 1. What is Hexaville? • 2. Installation • 3. Deploy to AWS • 4. How is Swift executed on Lambda? • 5. How to Debug? 3. Introduction of Hexaville Objectives
72. • Framework and libraries for building serverless applications that is written in Swift • Currently only supports AWS APIGateway+Lambda • Don’t need to deep knowledge for AWS • Cloud Native 1. What is Hexaville? 3. Introduction of Hexaville
73. What Hexaville solved / will solve • Fully automate deployment of Swift program to serverless environment (Routing, code deploy etc..) • Automatically create Swift execution environment on Lambda • Normal Development/Deployment work flow • iOS Developer can write backend and client with single language. can share code. > 1. What is Hexaville? 3. Introduction of Hexaville
74. Hexaville has Two layers > 1. What is Hexaville? 3. Introduction of Hexaville Hexaville Framework (Web Application Framework) Hexaville (Command Line Interface) Your Application Code Deploy
75. HexavilleFramework • Web Application Framework layer for Hexaville • Executed on Lambda • Provides Middleware and Router features • Extendable with Middlewares and plugins > 1. What is Hexaville? 3. Introduction of Hexaville
76. Basic Usage and Routing > 1. What is Hexaville? 3. Introduction of Hexaville
77. Your Own Middleware > 1. What is Hexaville? 3. Introduction of Hexaville
78. 2. Installation 3. Introduction of Hexaville
79. > 2. Installation 3. Introduction of Hexaville
80. /Users/yuki/WorkingsPlace/program/polyrhythm/FlieriOS % hexaville Available commands: - generate Generate initial project - deploy Deploy your application to the specified cloud provider - routes Show routes and endpoint for the API - help Prints this help information - version Prints the current version of this app hexaville command > 2. Installation 3. Introduction of Hexaville
81. /Users/yuki/WorkingsPlace/program/polyrhythm/FlieriOS % hexaville Available commands: - generate Generate initial project - deploy Deploy your application to the specified cloud provider - routes Show routes and endpoint for the API - help Prints this help information - version Prints the current version of this app hexaville command > 2. Installation 3. Introduction of Hexaville
82. /Users/yuki/Desktop % hexaville generate Hello /Users/yuki/Desktop % tree Hello Hello ├── Hexavillefile.yml ├── Package.swift └── Sources ├── RandomNumberGenerateMiddleware.swift └── main.swift 1 directory, 4 files The initial directory structure > 2. Installation 3. Introduction of Hexaville
83. Hexavillefile.yml - Configuration file for the Hexaville Package.swift - SPM manifest file that have dependencies, app settings Sources/main.swift - entrypoint Sources/RandomNumberGenerateMiddleware.swift - An example code of Middleware > 2. Installation 3. Introduction of Hexaville
84. A Project created with hexaville generate is ready to be deployed to lambda immediately. > 2. Installation 3. Introduction of Hexaville
85. 2. Deploy to AWS 3. Introduction of Hexaville
86. Configure Your Hexavillefile.yml > 3. Deploy to AWS 3. Introduction of Hexaville
87. /Users/yuki/Desktop/Hello % hexaville deploy Hello Hexavillefile: /Users/yuki/Desktop/Hello/Hexavillefile.yml Start to build swift... Building application.... Generating Routing Manifest file.... Generated Routing Manifest file Sending build context to Docker daemon 351.2kB Step 1/11 : FROM ubuntu:14.04 ---> 4a2820e686c4 Step 2/11 : RUN apt-get update -y ---> Using cache ---> 00c2bd676a0a 3. Introduction of Hexaville > 3. Deploy to AWS ‘hexaville deploy’ command
88. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy ᶆ’ mapping > 4. How is Swift executed on lambda? 3. Introduction of Hexaville
89. Deployment Flow of Lambda + ApiGateway Code Zip S3 Lambda APIGateway ᶄ Upload ᶃ Archive ᶅ Create Function with S3 URL ᶆ Resource(Path) and Function Mapping ᶇ Deploy ᶆ’ mapping > 4. How is Swift executed on lambda? 3. Introduction of Hexaville Totally Automated!!
90. ###################################################### Information ApplicationName: Hello Endpoint: https://xxx.execute-api.ap-northeast-1.amazonaws.com/staging Stage: staging ###################################################### All Done. Deployment done 3. Introduction of Hexaville > 3. Deploy to AWS
91. Access / from Browser 3. Introduction of Hexaville > 3. Deploy to AWS
92. it could deploy swift web application in only 5 minutes without Signing in AWS 3. Introduction of Hexaville > 3. Deploy to AWS
93. 4. How is Swift executed on lambda? 3. Introduction of Hexaville
94. Hexaville ᶃ Build (call hexaville deploy) Hexaville Framework Web App Code Your Web App Ubuntu Deployment Flow of Hexaville > 4. How is Swift executed on lambda? 3. Introduction of Hexaville
95. Hexaville ELF Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
96. Hexaville ELF - index.js - *.so ᶅArchive Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
97. Hexaville ELF AWS - index.js - *.so ᶅArchive ᶆ Deploy Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file > 4. How is Swift executed on lambda? 3. Introduction of Hexaville ᶃ Build (call hexaville deploy)
98. Hexaville ELF ᶃ Build AWS - index.js - *.so ᶅArchive ᶆ Deploy Hexaville Framework Web App Code Your Web App Ubuntu ᶄ Put the ELF into the Host Machine Deployment Flow of Hexaville Zip file Introduction of Hexaville > 2. How is Swift executed on lambda?
99. The Exposure of a trick > 4. How is Swift executed on lambda? 3. Introduction of Hexaville
100. 5. How to Debug? 3. Introduction of Hexaville
101. 5. How to Debug? 3. Introduction of Hexaville • Local Debug • Remote Debug There are two types of debugging
102. HexavilleFramework is also executable > 5. How to Debug? 3. Introduction of Hexaville
103. Local Debug > 5. How to Debug? /Users/yuki/Desktop/ % cd Hello /Users/yuki/Desktop/Hello % swift build /Users/yuki/Desktop/Hello % ./.build/debug/Hello Available commands: - gen-routing-manif Generate routing manifest file - execute Execute the specified resource. ex. execute GET / - serve Start Hexaville Builtin Server - help Prints this help information - version Prints the current version of this app 3. Introduction of Hexaville
104. Local Debug /Users/yuki/Desktop/ % cd Hello /Users/yuki/Desktop/Hello % swift build /Users/yuki/Desktop/Hello % ./.build/debug/Hello Available commands: - gen-routing-manif Generate routing manifest file - execute Execute the specified resource. ex. execute GET / - serve Start Hexaville Builtin Server - help Prints this help information - version Prints the current version of this app > 5. How to Debug? 3. Introduction of Hexaville
105. Local Debug /Users/yuki/Desktop/Hello % ./.build/debug/Hello serve Hexaville Builtin Server started at 0.0.0.0:3000 Serve builtin web server > 5. How to Debug? 3. Introduction of Hexaville
106. Local Debug Access the resource with curl /Users/yuki/Desktop/Hello % curl localhost:3000 <html><head><title>Hexaville</title></head><body>Welcome to Hexaville!</body></html> > 5. How to Debug? 3. Introduction of Hexaville /Users/yuki/Desktop/Hello % ./.build/debug/Hello serve Hexaville Builtin Server started at 0.0.0.0:3000 Serve builtin web server
107. Debuggable with Xcode and LLDB > 5. How to Debug? 3. Introduction of Hexaville
108. Remote Debug > 5. How to Debug? 3. Introduction of Hexaville Cloud Watch logs
109. $ hexaville tail -f your-project-name Now I’m considering to implement… > 5. How to Debug? 3. Introduction of Hexaville
110. Customize Error handling and Logging let app = HexavilleFramework() app.catch { error in print(error) // stdout will be output to Cloud Watch Logs // or // SendLog to Fluentd or something switch error { case FooError.notFound: return Response(status: .notFound) case JWTAuthenticationMiddleware.authrozationHeaderIsMissing: return Response(status: .unauthorized) default: return Response(status: .internalServerError) } } try app.run() > 5. How to Debug? 3. Introduction of Hexaville
111. 4. Create Practical Web App with Hexaville
112. https://github.com/noppoMan/HexavilleTODOExample 4. Create Practical Web App with Hexaville
113. OAuth2 Overview Callback 4. Create Practical Web App with Hexaville TODO Page(Hexaville) login page (Hexaville)
114. Stacks 4. Create Practical Web App with Hexaville • Hexaville: v0.2.0 • HexavilleFramework: v0.1.12 • Session Storage: DynamoDB • Main Data Base: DynamoDB
115. Demo (Movie) 4. Create Practical Web App with Hexaville
116. 4. Create Practical Web App with Hexaville
117. Plugins that are used in this time 4. Create Practical Web App with Hexaville
118. HexavilleAuth 4. Create Practical Web App with Hexaville
119. HexavilleAuth • OAuth1 • OAuth2 Authorization Methods Authentication Methods • Email and Password (not implemented yet) 4. Create Practical Web App with Hexaville
120. Supported SNS Platforms • Facebook • Github • Google • Instagram OAuth1 OAuth2 • Twitter 4. Create Practical Web App with Hexaville
121. import HexavilleAuth let facebookProvider = FacebookAuthorizationProvider( path: "/auth/facebook", consumerKey: ProcessInfo.processInfo.environment[“FACEBOOK_APP_ID"]! ?? "", consumerSecret: ProcessInfo.processInfo.environment[“FACEBOOK_APP_SECRET"]!, callbackURL: CallbackURL(baseURL: “http://foo.bar”, path: "/auth/facebook/callback"), scope: "public_profile,email" ) { credentianl, user, req, ctx in return Response(body: "access token is: \(credentianl.accessToken)") } var auth = HexavilleAuth() auth.add(facebookProvider) app.use(session) app.use(HexavilleAuth.AuthenticationMiddleware()) app.use(auth) // Available /auth/facebook and /auth/facebook /callback app.use(auth) { request, context in print(context.isAuthenticated()) } Code Example 4. Create Practical Web App with Hexaville
122. Session 4. Create Practical Web App with Hexaville
123. • DynamoDB (noppoMan) DataBase Clients 4. Create Practical Web App with Hexaville Using in TODO Example • Redis (IBM-Swift, Vapor) • MySQL (noppoMan, IBM-Swift) • Postgres (IBM-Swift) • MongoDB (OpenKitten) Other Available DB Clients
124. One more thing
125. • RDBMS and Serverless • Continuous Integration 4. Create Practical Web App with Hexaville Hexaville on Production > Hexaville on production
126. • Almost developers loves RDBMS even now • Our program is not a daemon on Serverless • —> Can’t keep connection for RDBMS • 1 HTTP Request = 1 Connection of RDBMS • —> Your RDBMS will hang up RDBMS and Serverless 4. Create Practical Web App with Hexaville > Hexaville on production
128. Hexaville provides following Pooling Connections Mechanism > Hexaville on production 4. Create Practical Web App with Hexaville
129. • Hexaville can easily do CI • Provides Prebuilt Binary and Installable just one line command • Doesn’t need to install Swift Language and Runtime • just 3 steps: git push -> test -> hexaville deploy Continuous Integration and Hexaville >Continuous Integration and Hexaville 4. Create Practical Web App with Hexaville
130. Hexaville is available on major CI Services 4. Create Practical Web App with Hexaville
131. 5. Conclusion
132. Swift Server Performance 5. Conclusion Processed Num of Requests in 30s 0 17500 35000 52500 70000 Prorsum(Swift 3.1) Kitura(Swift 3.1) Go 1.7 HTTP Server Node.js4.3 http 14,769 64,768 17,144 29,436 ɾwrk -d 30s -t 4 -c 20 ɾResponded with the 10 length of random JSON array ɾMachine: MacOS Sierra, 8 logical cores, 8GB RAM
133. http://qiita.com/applideveloper/items/0ac1a38249cccdc3f2d1 Akka HTTP (Scala) Prorsum (Swift 3.1)
134. 5. Conclusion wrk -d 30s -t 4 -c 20 Request/sec 0 25 50 75 100 Simple JSON Hexaville(Swift4 -O) Node.js 4.3 Node.js4.3 vs Hexaville on Lambda(512 RAM)
135. 5. Conclusion wrk -d 30s -t 4 -c 20 Request/sec 0 25 50 75 100 Simple JSON SHA256(1720 length string) Hexaville(Swift4 -O) Node.js 4.3 Node.js4.3 vs Hexaville on Lambda(512 RAM)
136. Real Product that use Hexaville 5. Conclusion
139. Thanks!