Slide 1
Slide 1 text
What’s new in Firebase
Daiki Matsudate / @d_date
2018/05/28 FJUG #4
Slide 2
Slide 2 text
Daiki Matsudate
@d_date
Slide 3
Slide 3 text
iOSΞϓϦ։ൃऀͷͨΊͷFirebaseೖ
Slide 4
Slide 4 text
https://yakiringo.booth.pm/
items/835645
iOSΞϓϦ։ൃऀͷͨΊͷFirebaseೖ
Slide 5
Slide 5 text
։࢝5࣌ؒͰཱʂiOSΞϓϦઃܭύλʔϯೖ
Slide 6
Slide 6 text
։࢝5࣌ؒͰཱʂiOSΞϓϦઃܭύλʔϯೖ
https://peaks.cc/iOS_architecture
Slide 7
Slide 7 text
Google I/O
Slide 8
Slide 8 text
Yosemite Bishop
Slide 9
Slide 9 text
Hakone
Slide 10
Slide 10 text
Part1: What’s new in iOS SDK v5.0.0
Daiki Matsudate / @d_date
2018/05/28 FJUG #4
Slide 11
Slide 11 text
v5.0.0 Released!
Slide 12
Slide 12 text
v5.0.0
• Support iOS version (iOS7.0 and later -> iOS 8.0 and later)
• Performance Monitoring graduated beta
• Phone Auth Testing
• Firebase Testlab for iOS
• MLKit
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
Graduated from beta!
Slide 15
Slide 15 text
- Automatic Trace
Measuring performances about app
• Automatic trace
• App start
• Network
• App in foreground / background
•
• Screen ← NEW!
Slide 16
Slide 16 text
- Screen Trace
iOS
keyWindowʹ͋ΔUIViewControllerͷ
ɾStart: viewDidAppear:͕ݺΕͨͱ͖
ɾStop: viewDidDisappear:͕ΑΕͨͱ͖ɻ
※Container View ControllerͰΩϟϓνϟ͠ͳ͍
Slide 17
Slide 17 text
Android
ͯ͢ͷActivityͷ
ɾStart: onActivityStarted()͕ݺΕͨͱ͖
ɾStop: onActivityStopped()͕ݺΕͨͱ͖
- Screen Trace
Slide 18
Slide 18 text
https://www.youtube.com/watch?v=e-8fiv-vteQ&t=1671s
Slide 19
Slide 19 text
https://www.youtube.com/watch?v=e-8fiv-vteQ&t=1671s
Slide 20
Slide 20 text
Appendix: Performance Monitoring - Custom Trace
let trace = Performance.startTrace(name: "request_trace")
let contents: String
do {
contents = try String(contentsOfFile: fileName, encoding: .utf8)
} catch {
print("Log file doesn't exist yet")
contents = ""
}
let fileLength = contents.lengthOfBytes(using: .utf8)
trace?.incrementMetric("log_file_size", by: Int64(fileLength))
Slide 21
Slide 21 text
let trace = Performance.startTrace(name: "request_trace")
let contents: String
do {
contents = try String(contentsOfFile: fileName, encoding: .utf8)
} catch {
print("Log file doesn't exist yet")
contents = ""
}
let fileLength = contents.lengthOfBytes(using: .utf8)
trace?.incrementMetric("log_file_size", by: Int64(fileLength))
1. Call startTrace with label name
2. Set your metric with parameter
Appendix: Performance Monitoring - Custom Trace
Slide 22
Slide 22 text
let target = "https://www.google.com/images/branding/googlelogo/2x/
googlelogo_color_272x92dp.png"
guard let targetUrl = URL(string: target) else { return }
guard let metric = HTTPMetric(url: targetUrl, httpMethod: .get) else { return }
metric.start()
var request = URLRequest(url:targetUrl)
request.httpMethod = "GET"
let task = URLSession.shared.dataTask(with: request) {
data, response, error in
if let httpResponse = response as? HTTPURLResponse {
metric.responseCode = httpResponse.statusCode
}
metric.stop()
ɾSpecified network trace
Appendix: Performance Monitoring - Custom Trace
Slide 23
Slide 23 text
let target = "https://www.google.com/images/branding/googlelogo/2x/
googlelogo_color_272x92dp.png"
guard let targetUrl = URL(string: target) else { return }
guard let metric = HTTPMetric(url: targetUrl, httpMethod: .get) else { return }
metric.start()
var request = URLRequest(url:targetUrl)
request.httpMethod = "GET"
let task = URLSession.shared.dataTask(with: request) {
data, response, error in
if let httpResponse = response as? HTTPURLResponse {
metric.responseCode = httpResponse.statusCode
}
metric.stop()
1. Initialize HTTPMetric with
targetUrl
3. Call stop() when finished downloading
2.Call start()
ɾSpecified network trace
Appendix: Performance Monitoring - Custom Trace
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
https://www.youtube.com/watch?v=e-8fiv-vteQ&t=1671s
Slide 26
Slide 26 text
Phone Auth Testing
Slide 27
Slide 27 text
Set phone number and verification code for testing
Phone Auth Testing
Slide 28
Slide 28 text
Auth.auth().createUser(withEmail: email, password: password) { (user, error) in
self.hideSpinner {
guard let email = user.email, error == nil else {
self.showMessagePrompt(error!.localizedDescription)
return
}
print("\(email) created")
self.navigationController!.popViewController(animated: true)
}
}
Breaking change
Slide 29
Slide 29 text
Auth.auth().createUser(withEmail: email, password: password) { (user, error) in
self.hideSpinner {
guard let email = user.email, error == nil else {
self.showMessagePrompt(error!.localizedDescription)
return
}
print("\(email) created")
self.navigationController!.popViewController(animated: true)
}
}
Breaking change
Slide 30
Slide 30 text
Auth.auth().createUser(withEmail: email, password: password) { (authResult, error) in
self.hideSpinner {
guard let email = authResult?.user.email, error == nil else {
self.showMessagePrompt(error!.localizedDescription)
return
}
print("\(email) created")
self.navigationController!.popViewController(animated: true)
}
}
Breaking change
Slide 31
Slide 31 text
Auth.auth().createUser(withEmail: email, password: password) { (authResult, error) in
self.hideSpinner {
guard let email = authResult?.user.email, error == nil else {
self.showMessagePrompt(error!.localizedDescription)
return
}
print("\(email) created")
self.navigationController!.popViewController(animated: true)
}
}
Breaking change
Slide 32
Slide 32 text
authResult: AuthDataResult
user: User
additionalUserInfo: AdditionalUserInfo
profile: [String: NSObject]?
isNewUser: Bool
username: String?
Breaking change
Slide 33
Slide 33 text
Goodbye Twitter Kit SDK!
https://blog.twitter.com/developer/en_us/topics/tools/2018/discontinuing-support-for-twitter-kit-sdk.html
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
• Run test in physical and virtual devices
• Available in
• android
• iOS
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
• Run Unit test and UI test with XCTest
• Firebase Console
• gcloud CLI
Firebase Test Lab for iOS
Slide 38
Slide 38 text
https://firebase.google.com/docs/test-lab/ios/available-testing-devices
Available iOS devices
Slide 39
Slide 39 text
Firebase Test Lab for iOS
Slide 40
Slide 40 text
https://g.co/firebase/TestLabSignUp
Signup Test Lab for iOS
Slide 41
Slide 41 text
Summary
• Automatic screen trace in performance monitoring → NEW!
• Phone auth testing with whitelist phone number → NEW!
• Test Lab for iOS → NEW!
• MLKit → NEW!
Slide 42
Slide 42 text
Resources
• Session:
• What’s new in Firebase https://www.youtube.com/watch?
v=e-8fiv-vteQ&t=57s
• Release note:
• iOS: https://firebase.google.com/support/release-notes/ios
• Android: https://firebase.google.com/support/release-notes/
android
Slide 43
Slide 43 text
Part2: Introduction to MLKit
Daiki Matsudate / @d_date
2018/05/28 FJUG #4
Slide 44
Slide 44 text
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 45
Slide 45 text
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 46
Slide 46 text
DEMO
Slide 47
Slide 47 text
ML Kit
• iOS / Android SDK
• Base API and Custom Model Support
• On device / On Cloud
• Deeply integrated into Firebase
Slide 48
Slide 48 text
ML Kit - iOS / Android SDK
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 49
Slide 49 text
ML Kit - Custom Model Support
• Dynamic model downloads
• No need to bundle every custom model in your ipa / apk
• A/B testing via Firebase Remote Config
• Model conversion and compression (coming soon)
Slide 50
Slide 50 text
ML Kit - Custom Local Model
guard let localModelFilePath = Bundle.main.path(
forResource: Constants.quantizedModelFilename,
ofType: DetectorConstants.modelExtension
),
let labelsFilePath = Bundle.main.path(
forResource: Constants.quantizedLabelsFilename,
ofType: DetectorConstants.labelsExtension
)
else {
resultsTextView.text = "Failed to get the paths to the local model and labels files."
return
}
let localModelSource = LocalModelSource(
modelName: Constants.localModelName,
path: localModelFilePath
)
let modelManager = ModelManager.modelManager()
if !modelManager.register(localModelSource) {
print("Model source was already registered with name: \(localModelSource.modelName).")
}
let options = ModelOptions(cloudModelName: nil, localModelName: Constants.localModelName)
detectorService.loadModel(options: options, labelsPath: labelsFilePath)
Slide 51
Slide 51 text
ML Kit - Custom Local Model
guard let localModelFilePath = Bundle.main.path(
forResource: Constants.quantizedModelFilename,
ofType: DetectorConstants.modelExtension
),
let labelsFilePath = Bundle.main.path(
forResource: Constants.quantizedLabelsFilename,
ofType: DetectorConstants.labelsExtension
)
else {
resultsTextView.text = "Failed to get the paths to the local model and labels files."
return
}
let localModelSource = LocalModelSource(
modelName: Constants.localModelName,
path: localModelFilePath
)
let modelManager = ModelManager.modelManager()
if !modelManager.register(localModelSource) {
print("Model source was already registered with name: \(localModelSource.modelName).")
}
let options = ModelOptions(cloudModelName: nil, localModelName: Constants.localModelName)
detectorService.loadModel(options: options, labelsPath: labelsFilePath)
1. Get model path from bundle
2. Register to ModelManager
3. Load model
Slide 52
Slide 52 text
ML Kit - Custom Cloud Model
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 53
Slide 53 text
ML Kit - Custom Cloud Model
let conditions = ModelDownloadConditions(wiFiRequired: false, idleRequired: false)
let cloudModelSource = CloudModelSource(
modelName: cloudModelName,
enableModelUpdates: true,
initialConditions: conditions,
updateConditions: conditions
)
let modelManager = ModelManager.modelManager()
if !modelManager.register(cloudModelSource) {
print("Model source was already registered with name: \(cloudModelSource.modelName).")
}
let options = ModelOptions(cloudModelName: cloudModelName, localModelName: nil)
detectorService.loadModel(options: options, labelsPath: labelsFilePath)
Slide 54
Slide 54 text
ML Kit - Custom Cloud Model
let conditions = ModelDownloadConditions(wiFiRequired: false, idleRequired: false)
let cloudModelSource = CloudModelSource(
modelName: cloudModelName,
enableModelUpdates: true,
initialConditions: conditions,
updateConditions: conditions
)
let modelManager = ModelManager.modelManager()
if !modelManager.register(cloudModelSource) {
print("Model source was already registered with name: \(cloudModelSource.modelName).")
}
let options = ModelOptions(cloudModelName: cloudModelName, localModelName: nil)
detectorService.loadModel(options: options, labelsPath: labelsFilePath)
1.specifiy model registed on Firebase Console
2. Register to ModelManager
3. Load model
Slide 55
Slide 55 text
ML Kit with Firebase Remote Config
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 56
Slide 56 text
ML Kit with Firebase Remote Config
let conditions = ModelDownloadConditions(wiFiRequired: false, idleRequired: false)
let cloudModelSource = CloudModelSource(
modelName: remoteConfig[cloudModelConfigKey].stringValue,
enableModelUpdates: true,
initialConditions: conditions,
updateConditions: conditions
)
let modelManager = ModelManager.modelManager()
if !modelManager.register(cloudModelSource) {
print("Model source was already registered with name: \(cloudModelSource.modelName).")
}
let options = ModelOptions(cloudModelName: cloudModelName, localModelName: nil)
detectorService.loadModel(options: options, labelsPath: labelsFilePath)
Specify cloud model name key in Remote Config
Slide 57
Slide 57 text
ML Kit - On device / On Cloud
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 58
Slide 58 text
ML Kit - On device / On Cloud
https://firebase.google.com/docs/ml-kit/
Slide 59
Slide 59 text
ML Kit - Coming Soon Features
• High density face contour
• smart reply API
• Custom model compression and conversion
Slide 60
Slide 60 text
ML Kit - High density face contour
• Detect about 100 points in
face contour
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 61
Slide 61 text
ML Kit - Smart Reply API
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 62
Slide 62 text
ML Kit - Model Compression and Conversion
• Convert Tensorflow model to Tensorflow Lite model
https://www.youtube.com/watch?v=Z-dqGRSsaBs
Slide 63
Slide 63 text
ML Kit - Coming Soon Features
https://g.co/firebase/SignUp
Slide 64
Slide 64 text
ML Kit - Resources
• Session:
• ML Kit: Machine Learning SDK for mobile developers https://
www.youtube.com/watch?v=Z-dqGRSsaBs
• TensorFlow Lite for mobile developers https://
www.youtube.com/watch?v=ByJnpbDd-zc
• Document
• https://firebase.google.com/docs/ml-kit/
Slide 65
Slide 65 text
Summary - MLKit vs CoreML
• Can be used Cloud computing resource
• CoreML only support on-device model
• CoreML supports NLP (smart reply API?)
Slide 66
Slide 66 text
Part3: ARCore on ARKit with Firebase
Daiki Matsudate / @d_date
2018/05/28 FJUG #4
Slide 67
Slide 67 text
No content
Slide 68
Slide 68 text
- Argumented Images
• Respond to 2D images, such
as posters or product
packaging
• Only support android
Slide 69
Slide 69 text
- Cloud Anchors
• Create AR experience with multiplayer
• Support iOS and Android
Slide 70
Slide 70 text
- Cloud Anchors
• Create AR experience with multiplayer
• Support iOS and Android
• iOS ARCore needs ARKit
Slide 71
Slide 71 text
DEMO
Slide 72
Slide 72 text
- Cloud Anchors
Slide 73
Slide 73 text
- Cloud Anchors
Slide 74
Slide 74 text
- Cloud Anchors
Host Cloud Anchor ID
Slide 75
Slide 75 text
- Cloud Anchors
Resolve with Cloud
Anchor ID
Slide 76
Slide 76 text
- Cloud Anchors
Anchor information
Slide 77
Slide 77 text
- Cloud Anchors
Cloud Anchor ID
Slide 78
Slide 78 text
ua-44417245235b466b84220d681b6da6a2
- Cloud Anchors
Slide 79
Slide 79 text
ua-44417245235b466b84220d681b6da6a2
"
- Cloud Anchors
Slide 80
Slide 80 text
Host ID
- Cloud Anchors
Host ID
Room ID: 1
Room ID: 1
Slide 81
Slide 81 text
Host ID
- Cloud Anchors
Host ID
Room ID: 1
Room ID: 1
Slide 82
Slide 82 text
Host ID
- Cloud Anchors
Host ID
Room ID: 1
Room ID: 1
Room ID: 1
Slide 83
Slide 83 text
- Host Cloud Anchor
func addAnchor(transform: matrix_float4x4) {
let arAnchor = ARAnchor(transform: transform)
self.arAnchor = arAnchor
sceneView.session.add(anchor: arAnchor)
do {
garAnchor = try gSession.hostCloudAnchor(arAnchor)
enter(state: .hosting)
} catch {
print(error)
}
}
Slide 84
Slide 84 text
- Host Cloud Anchor
func addAnchor(transform: matrix_float4x4) {
let arAnchor = ARAnchor(transform: transform)
self.arAnchor = arAnchor
sceneView.session.add(anchor: arAnchor)
do {
garAnchor = try gSession.hostCloudAnchor(arAnchor)
enter(state: .hosting)
} catch {
print(error)
}
}
Add anchor on AR Space
Host to Cloud by ARCore
Slide 85
Slide 85 text
func session(_ session: GARSession, didHostAnchor anchor: GARAnchor) {
guard state == .hosting, anchor == garAnchor else {
return
}
garAnchor = anchor
enter(state: .hostingFinished)
firebaseReference.child("hotspot_list").child(roomCode).child("hosted_anchor_id").setValu
e(anchor.cloudIdentifier)
let timestampInteger = NSDate().timeIntervalSince1970 * 1000
let timestamp = NSNumber(value: timestampInteger)
firebaseReference.child("hotspot_list").child(roomCode).child("updated_at_timestamp").set
Value(timestamp)
}
- After host anchor to cloud
Slide 86
Slide 86 text
- After host anchor to cloud
func session(_ session: GARSession, didHostAnchor anchor: GARAnchor) {
guard state == .hosting, anchor == garAnchor else {
return
}
garAnchor = anchor
enter(state: .hostingFinished)
firebaseReference.child("hotspot_list").child(roomCode).child("hosted_anchor_id").setValu
e(anchor.cloudIdentifier)
let timestampInteger = NSDate().timeIntervalSince1970 * 1000
let timestamp = NSNumber(value: timestampInteger)
firebaseReference.child("hotspot_list").child(roomCode).child("updated_at_timestamp").set
Value(timestamp)
}
Save hosted_anchor_id to Realtime Database
Save timestamp
Slide 87
Slide 87 text
func resolveAnchor(roomCode: String) {
self.roomCode = roomCode
enter(state: .resolving)
firebaseReference.child("hotspot_list").child(roomCode).observe(.value) { (snapshot)
in
DispatchQueue.main.async { [weak self] in
guard let strongSelf = self, strongSelf.state == .resolving, strongSelf.roomCode
== roomCode else { return }
if let anchorId: String = {
if let value = snapshot.value as? [String: Any] {
return value["hosted_anchor_id"] as? String
}
return nil
}() {
strongSelf.firebaseReference.child("hotspot_list").child(roomCode).removeAllObservers()
strongSelf.resolveAnchor(identifier: anchorId)
}
}
}
}
- Get Anchor ID from Realtime Database
Slide 88
Slide 88 text
- Get Anchor ID from Realtime Database
func resolveAnchor(roomCode: String) {
self.roomCode = roomCode
enter(state: .resolving)
firebaseReference.child("hotspot_list").child(roomCode).observe(.value) { (snapshot)
in
DispatchQueue.main.async { [weak self] in
guard let strongSelf = self, strongSelf.state == .resolving, strongSelf.roomCode
== roomCode else { return }
if let anchorId: String = {
if let value = snapshot.value as? [String: Any] {
return value["hosted_anchor_id"] as? String
}
return nil
}() {
strongSelf.firebaseReference.child("hotspot_list").child(roomCode).removeAllObservers()
strongSelf.resolveAnchor(identifier: anchorId)
}
}
}
}
Type room code stored in Realtime Database
Get host_anchor_id from Realtime Database
Resolve
Slide 89
Slide 89 text
func resolveAnchor(identifier: String) {
do {
garAnchor = try gSession?.resolveCloudAnchor(withIdentifier: identifier)
} catch {
print(error)
}
}
- Resolve with anchor ID
func session(_ session: GARSession, didResolve anchor: GARAnchor) {
guard state == .resolving, anchor == garAnchor else {
return
}
self.garAnchor = anchor
let arAnchor = ARAnchor(transform: anchor.transform)
self.arAnchor = ARAnchor(transform: anchor.transform)
sceneView.session.add(anchor: arAnchor)
enter(state: .resolvingFinished)
}
Slide 90
Slide 90 text
func resolveAnchor(identifier: String) {
do {
garAnchor = try gSession?.resolveCloudAnchor(withIdentifier: identifier)
} catch {
print(error)
}
}
- Resolve with anchor ID
Resolve with anchor id
func session(_ session: GARSession, didResolve anchor: GARAnchor) {
guard state == .resolving, anchor == garAnchor else {
return
}
self.garAnchor = anchor
let arAnchor = ARAnchor(transform: anchor.transform)
self.arAnchor = ARAnchor(transform: anchor.transform)
sceneView.session.add(anchor: arAnchor)
enter(state: .resolvingFinished)
}
Add anchor to AR Space
Slide 91
Slide 91 text
- Cloud Anchors with Firebase
• Good use case for Realtime Database / Firestore
• Realtime
• Store host_id in only active room
• Can wait until creating new room by observing
• Easy to build database with such features
• Scalable
Slide 92
Slide 92 text
- Summary
• Argumented Image
• Only for Android now
• Cloud Anchors
• Available on both iOS and Android
• More useful to use with Firebase Database
Slide 93
Slide 93 text
- Resources
• Swift Sample
• https://github.com/d-date/arcore-ios-sdk/tree/swift-sample
• Session
• https://www.youtube.com/watch?v=MeZcQguH124
• Document
• https://developers.google.com/ar
Slide 94
Slide 94 text
Summary
• Automatic screen trace in performance monitoring → NEW!
• Phone auth testing with whitelist phone number → NEW!
• Test Lab for iOS → NEW!
• MLKit is useful to use cloud model than CoreML
• ARCore can be used with ARKit to share anchor
Slide 95
Slide 95 text
7/2 ٕज़ಉਓࢽ࠶ൢNight˒
Slide 96
Slide 96 text
7/2 ٕज़ಉਓࢽ࠶ൢNight˒
ຊ࣋ͬͯΔਓ͖࣋ͬͯͯʙʂ
چόʔδϣϯ࣋ࢀͰɺ৽͍͠ͷʹަʂ
Slide 97
Slide 97 text
Thank you!