APIからFirebase Realtime Databaseへ - クライアントサイドから見るサーバーレスアーキテクチャー

2018/11/3 serverlessconf Tokyo
Firebase Realtime Database / Cloud Firestoreの話

Transcript

1. Replace API to Firebase Database - The view from client side Daiki Matsudate / @d_date 2017/11/03 / Serverlessconf Tokyo 2017
2. Daiki Matsudate iOS Mobile App Engineer @d_date
3. Tokyo 2018/3/1 - 3
4. Firebase
5. Firebase
6. Cloud Messaging
9. 31 Oct. at Amsterdam #FirebaseSummit
10. • Prediction • A/B Testing • Integrate Crashlytics into Firebase #FirebaseSummit
11. #FirebaseSummit
12. Agenda • Firebase Realtime Database • Feature • Introduce to Mobile App • Architecture • Testing • ???
13. Firebase Database
14. • NoSQL cloud database • Realtime Data Sync with JSON Tree • Available data on offline Firebase Database
15. Firebase Database
16. Replace API to Realtime Database Case Study
17. Replace API to Realtime Database API Client (Native App) Server
18. Client (Native App) Server Firebase Database Replace API to Realtime Database
19. Replace API to Realtime Database • use func observeSingleEvent(:_) in FirebaseDatabase
20. Replace API to Realtime Database • Reduce building API man-hours • No more performance tunings • Easy to handle data in offline Purpose
21. Client (Native App) Server Firebase Database Replace API to Realtime Database
22. Client (Native App) Firebase Database Replace API to Realtime Database
23. NoSQL
24. NoSQL = Denormalization
25. Denormalization in NoSQL { "users": { "user1": { "name": "Alice" }, "user2": { "name": "Bob" } }, "links": { "link1": { "title": "Example", "href": "http://example.org", "submitted": "user1" } }, "comments": { "comment1": { "link": "link1", "body": "This is awesome!", "author": "user2" } } }
26. Denormalization in NoSQL { "users": { "user1": { "name": "Alice" }, "user2": { "name": "Bob" } }, "links": { "link1": { "title": "Example", "href": "http://example.org", "submitted": "user1" } }, "comments": { "comment1": { "link": "link1", "body": "This is awesome!", "author": "user2" } } }
27. Denormalization in NoSQL { "users": { "user1": { "name": "Alice" }, "user2": { "name": "Bob" } }, "links": { "link1": { "title": "Example", "href": "http://example.org", "submitted": "user1" } }, "comments": { "comment1": { "link": "link1", "body": "This is awesome!", "author": "user2" } } } Join data in client
29. JOIN data in client in Swift self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) })
30. Not Swifty!! self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) }) “Stringly“ typed Pyramid of Death
31. Easy to replace from API? self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) }) Need reference of database
33. To make friend with NoSQL in Swift
34. To make friend with NoSQL in Swift • Re-Design Architecture • Introduce Reactive Programming
35. Re-Design Architecture
41. !Architecture → Ideal State "Ideal State → Architecture
42. Ideal State • Data from Firebase ≠ Data in app -> Need translation • Easy to replace from API to Firebase -> I/F is same as API, then want to replace DAO
43. Replaceable Data Access Object Entity Data Access to API Data Access Interface (Repository) Server
44. Replaceable Data Access Object Entity Data Access to Firebase Data Access Interface (Repository)
45. Translation Entity Data Access Interface Data Access to Firebase Interactor Model View
46. Requirement for Architecture Entity Data Access Interface Data Access to Firebase Interactor Model View
47. What is this architecture? Entity Data Access Interface Data Access to Firebase Interactor Model Presenter View
49. How to resolve Pyramid of Death self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) }) Pyramid of Death
51. How to resolve Pyramid of Death self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) })
52. protocol Entity: Codable {} struct UserEntity: Entity { let name: String } struct LinkEntity: Entity { let title: String let href: URL let submitted: String } struct CommentEntity: Entity { let link: String let body: String let author: String } Entity Entity Data Access Data Access Interactor Model Presenter View
53. Data Access + RxSwift extension DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in self.observeSingleEvent(of: .value, with: { (snapshot) in guard let jsonObject = snapshot.value else { observer(.error(ProviderError.valueNotExist)) return } do { let data = try JSONSerialization.data(withJSONObject: jsonObject) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch(let error) { observer(.error(error)) } }) return Disposables.create() }) } }
54. Data Access + RxSwift extension DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in self.observeSingleEvent(of: .value, with: { (snapshot) in guard let jsonObject = snapshot.value else { observer(.error(ProviderError.valueNotExist)) return } do { let data = try JSONSerialization.data(withJSONObject: jsonObject) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch(let error) { observer(.error(error)) } }) return Disposables.create() }) } } If value not exist, invoke error
55. Data Access + RxSwift extension DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in self.observeSingleEvent(of: .value, with: { (snapshot) in guard let jsonObject = snapshot.value else { observer(.error(ProviderError.valueNotExist)) return } do { let data = try JSONSerialization.data(withJSONObject: jsonObject) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch(let error) { observer(.error(error)) } }) return Disposables.create() }) } } If fail to serialize JSON, invoke error
56. Data Provider class DataProvider { static let shared = DataProvider() private init() {} lazy var ref: DatabaseReference = { return Database.database().reference() }() func getUser(key: String) -> Single<UserEntity> { return ref.child("users").child(key).get() } func getLink(key: String) -> Single<LinkEntity> { return ref.child("links").child(key).get() } func getComments() -> Single<CommentEntity> { return ref.child("comments").get() } }
57. Data Provider class DataProvider { static let shared = DataProvider() private init() {} lazy var ref: DatabaseReference = { return Database.database().reference() }() func getUser(key: String) -> Single<UserEntity> { return ref.child("users").child(key).get() } func getLink(key: String) -> Single<LinkEntity> { return ref.child("links").child(key).get() } func getComments() -> Single<CommentEntity> { return ref.child("comments").get() } } Singleton in Swift
58. Data Provider class DataProvider { static let shared = DataProvider() private init() {} lazy var ref: DatabaseReference = { return Database.database().reference() }() func getUser(key: String) -> Single<UserEntity> { return ref.child("users").child(key).get() } func getLink(key: String) -> Single<LinkEntity> { return ref.child("links").child(key).get() } func getComments() -> Single<CommentEntity> { return ref.child("comments").get() } } Retain DatabaseReference
59. Data Provider class DataProvider { static let shared = DataProvider() private init() {} lazy var ref: DatabaseReference = { return Database.database().reference() }() func getUser(key: String) -> Single<UserEntity> { return ref.child("users").child(key).get() } func getLink(key: String) -> Single<LinkEntity> { return ref.child("links").child(key).get() } func getComments() -> Single<CommentEntity> { return ref.child("comments").get() } } Get Entity from Database, then return Single<Entity>
60. Data Access struct UserRepository { let dataStore: LinkDataStore func get(userId: String) -> Single<UserEntity> { return dataStore.get(key: userId) } } struct UserDataStore { let provider = DataProvider.shared func get(key: String) -> Single<UserEntity> { return provider.getUser(key: key) } } Entity Data Access Data Access Interactor Model Presenter View
61. Data Access struct UserRepository { let dataStore: LinkDataStore func get(userId: String) -> Single<UserEntity> { return dataStore.get(key: userId) } } struct UserDataStore { let provider = DataProvider.shared func get(key: String) -> Single<UserEntity> { return provider.getUser(key: key) } } I/F for data access Implementation for data access Entity Data Access Data Access Interactor Model Presenter View
62. Interactor (UseCase) struct UseCase { let commentsRepository: CommentRepository let usersRepository: UserRepository let linksRepository: LinkRepository func getComments() -> Single<[CommentModel]> { return commentsRepository.get() .map { (comment) -> [CommentModel] in let link = self.getLink(linkId: comment.link) let author = self.getUser(userId: comment.author) return Single<(LinkModel, UserModel)>.zip(link, author) { ($0, $1) }).map(translator: CommentTranslator()) } } } Entity Data Access Data Access Interactor Model Presenter View
63. Before self.ref.child("comments").observeSingleEvent(of: .value, with: { [weak self] (snapshot) in guard let strongSelf = self else { return } guard let comments = snapshot.value as? [String: Any] else { return } comments.forEach({ (key, value) in guard let comment = value as? [String: Any] else { return } let linkKey: String = comment["link"] as! String // "link1" let authorKey: String = comment["author"] as! String // "user2" strongSelf.ref.child("links").child(linkKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let link = snapshot.value as? [String: Any] else { return } let submittedKey: String = link["submitted"] as! String // "user1" strongSelf.ref.child("users").child(submittedKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Alice" print(name) }) }) strongSelf.ref.child("users").child(authorKey).observeSingleEvent(of: .value, with: { (snapshot) in guard let user = snapshot.value as? [String: Any] else { return } let name: String = user["name"] as! String // “Bob” print(name) }) }) })
64. Presenter class Presenter { let useCase: UseCase init(useCase: UseCase) { self.useCase = useCase } func getComments() -> Single<[CommentModel]> { return useCase.getComments() } } Entity Data Access Data Access Interactor Model Presenter View
65. View class ViewController: UIViewController { weak var presenter: Presenter! private let disposeBag = DisposeBag() func inject(presenter: Presenter) { self.presenter = presenter } override func viewDidLoad() { super.viewDidLoad() presenter.getComments() .subscribe(onSuccess: { (models) in //Handling model }, onError: { (error) in //Handling Error }).disposed(by: disposeBag) } } Entity Data Access Data Access Interactor Model Presenter View
66. View class ViewController: UIViewController { weak var presenter: Presenter! private let disposeBag = DisposeBag() func inject(presenter: Presenter) { self.presenter = presenter } override func viewDidLoad() { super.viewDidLoad() presenter.getComments() .subscribe(onSuccess: { (models) in //Handling model }, onError: { (error) in //Handling Error }).disposed(by: disposeBag) } } Entity Data Access Data Access Interactor Model Presenter View Focus on UI Handling
67. Side Effect
68. Side Effect Entity Data Access Interface Data Access to Firebase Interactor Model Presenter View
69. Side Effect - Easy Unit Testing Entity Data Access Interface Mock Object Interactor Model Presenter View Easy to replace
70. class DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in guard let bundlePath = Bundle(for: type(of: self)).path(forResource: "Stub", ofType: "bundle") else { observer(.error(SerializeError.missingBundle)) return Disposables.create() } let jsonName = String(describing: T.self) guard let path = Bundle(path: bundlePath)?.path(forResource: jsonName, ofType: "json") else { observer(.error(SerializeError.missingJson(jsonName: jsonName))) return Disposables.create() } do{ let data = try Data(contentsOf: URL(fileURLWithPath: path)) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch { observer(.error(error)) } return Disposables.create() }) } } Side Effect - Easy Unit Testing
71. class DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in guard let bundlePath = Bundle(for: type(of: self)).path(forResource: "Stub", ofType: "bundle") else { observer(.error(SerializeError.missingBundle)) return Disposables.create() } let jsonName = String(describing: T.self) guard let path = Bundle(path: bundlePath)?.path(forResource: jsonName, ofType: "json") else { observer(.error(SerializeError.missingJson(jsonName: jsonName))) return Disposables.create() } do{ let data = try Data(contentsOf: URL(fileURLWithPath: path)) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch { observer(.error(error)) } return Disposables.create() }) } } Side Effect - Easy Unit Testing FackObject
72. class DatabaseReference { func get<T: Entity>() -> Single<T> { return Single.create(subscribe: { (observer) in guard let bundlePath = Bundle(for: type(of: self)).path(forResource: "Stub", ofType: "bundle") else { observer(.error(SerializeError.missingBundle)) return Disposables.create() } let jsonName = String(describing: T.self) guard let path = Bundle(path: bundlePath)?.path(forResource: jsonName, ofType: "json") else { observer(.error(SerializeError.missingJson(jsonName: jsonName))) return Disposables.create() } do{ let data = try Data(contentsOf: URL(fileURLWithPath: path)) let entity = try JSONDecoder().decode(T.self, from: data) observer(.success(entity)) } catch { observer(.error(error)) } return Disposables.create() }) } } Return Entity created from Stub JSON Side Effect - Easy Unit Testing FackObject
73. Tips - Realtime update or not lazy var realtimeRef: DatabaseReference = { let ref = Database.database().reference() ref.keepSynced(true) return ref }() func getFavorite(userId: String) -> Single<Void> { return realtimeRef.child(userId).child("favorites").get() }
74. Tips - Realtime update or not lazy var realtimeRef: DatabaseReference = { let ref = Database.database().reference() ref.keepSynced(true) return ref }() func getFavorite(userId: String) -> Single<Void> { return realtimeRef.child(userId).child("favorites").get() } Reflect database update immediately
75. Pros/Cons • Reduce building API man-hours • No more performance tunings • Easy to handle data in offline Pros
76. Pros/Cons • Reduce building API man-hours • No more performance tunings • Easy to handle data in offline Pros #
77. Pros/Cons • Must have join logic in client side • Need more money $ to build in some case • Data cache system doesn’t suit for your app in some case • Sorting and Query are hard to use • Scaling requires sharding (100,000 concurrent connections & 1,000 writes/sec.) Cons
78. Pros/Cons • Must have join logic in client side • Need more money $ to build in some case • Data cache system doesn’t suit for your app in some case • Sorting and Query are hard to use • Scaling requires sharding (100,000 concurrent connections & 1,000 writes/sec.) Cons %
79. Alternatives • Realtime Database + Cloud Functions • Building API server (Old way) • ???
81. One more thing…
82. Cloud Firestore
83. • Stores data in documents organized in collections • Sending data with protobuf, which have more types • Introduced reference type, so requires less denormalization and data flattening • Indexed queries with compound sorting and filtering Cloud Firestore
84. Cloud Firestore
85. Cloud Firestore Reference type
86. What will be changed on architecture Entity Data Access Interface Data Access to Firebase Interactor Model Presenter View
87. What will be changed on architecture Entity Data Access Interface Data Access to Firebase Interactor Model Presenter View
88. Recap • Serverless architecture ( mBaaS / SaaS and etc. ) is also effective to use on native apps • API server can be replaced by Firebase Database / Cloud firestore in some case • Replaceable DAO and separate Entity and Model in app is important • With side effect, easy to test modules
89. If you have Questions… Daiki Matsudate d.matsudate@gmail.com twitter: @d_date GitHub: @d-date
90. Enjoy! Your serverless
91. References • Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin • https://firebase.google.com/docs/firestore/rtdb-vs-firestore • https://firebase.google.com/docs/firestore/firestore-for- rtdb • https://speakerdeck.com/bpyamasinn/api-wo-firebase- realtime-database-niyi-xing-siteqi-fu-itakoto-ver2