▸ kagemiku (Akira Fukunaga) ▸ GitHub: @kagemiku ▸ Twitter: @kagemiku_en ▸ iOS Engineer (19’s new grad) at Mercari JP ▸ First time participation in WWDC!!!
SESSIONS ▸ 209: What’s New in Machine Learning ▸ 704: Core ML 3 Framework ▸ 406: Create ML for Object Detection and Sound Classification ▸ 222: Understanding Images in Vision Framework ▸ 228: Creating Great Apps Using Core ML and ARKit ▸ 407: Create ML for Activity, Text, and Recommendations ▸ 232: Advances in Natural Language Framework ▸ 234: Text Recognition in Vision Framework ▸ 420: Drawing Classification and One-Shot Object Detection in Turi Create ▸ 803: Designing Great ML Experiences ▸ 614: Metal for Machine Learning
CREATE ML WHAT’S CREATE ML? ▸ Framework for creating ML model with Swift, appeared in Xcode 10 let data = try! MLDataTable(contentsOf: URL(fileURLWithPath: "/path/to/dataset.json")) let (trainingData, testingData) = data.randomSplit(by: 0.8, seed: 5) let sentimentClassifier = try! MLTextClassifier(trainingData: trainingData, textColumn: "text", labelColumn: "label")
CREATE ML ▸ 9 templates were described in session ▸ Image Classifier ▸ Sound Classifier ▸ Activity Classifier ▸ Tabular Classifier ▸ and so on… ▸ But now, there are only 2 templates at the first seed
CREATE ML ▸ 9 templates were described in session ▸ Image Classifier ▸ Sound Classifier ▸ Activity Classifier ▸ Tabular Classifier ▸ and so on… ▸ But now, there are only 2 templates at the first seed
CREATE ML ▸ Dataset notes ▸ Balanced quantity ▸ : 10 s, 100 s, 1000 s ▸ : 100 s, 100 s, 100 s ▸ At least 10 data for each label ▸ At least 299 × 299 px
DOMAIN APIS WHAT’S DOMAIN API ▸ Useful ML models prepared by Apple ▸ We don’t have to collect data and build model ▸ Main frameworks ▸ Computer Vision (Vision Framework) ▸ Natural Language Processing (NaturalLanguage Framework)
DOMAIN APIS NEW DOMAIN APIS ▸ Many new apis appear ▸ Image Saliency ▸ Image Similarity ▸ Sentiment Analysis ▸ Text Recognition ▸ NL Transfer Learning ▸ and so on…
DOMAIN APIS NEW DOMAIN APIS ▸ Many new apis appear ▸ Image Saliency ▸ Image Similarity ▸ Sentiment Analysis ▸ Text Recognition ▸ NL Transfer Learning ▸ and so on…
DOMAIN APIS IMAGE SALIENCY ▸ Saliency: the salient points or features of something are the most important or most noticeable parts of it ▸ There are 2 types of saliency ▸ Attention based ▸ Objectness based
DOMAIN APIS IMAGE SALIENCY ▸ Attention based ▸ Training data: Human eye movement ▸ App example: image cropping ▸ Objectness based ▸ Training data: distinguished foreground object from background ▸ App example: object tracking
DOMAIN APIS ▸ code // 1. prepare request and handler let request: VNRequest = VNGenerateAttentionBasedSaliencyImageRequest() let requestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .up, options: [:]) // 2. perform requests and get results try? requestHandler.perform([request]) let observation = request.results?.first as? VNSaliencyImageObservation // 3. do something using results if let salientObjects = observation?.salientObjects { for object in salientObjects { let boundingBox = object.boundingBox // do something } } IMAGE SALIENCY
DOMAIN APIS NEW DOMAIN APIS ▸ Many new apis appear ▸ Image Saliency ▸ Image Similarity ▸ Sentiment Analysis ▸ Text Recognition ▸ NL Transfer Learning ▸ and so on…
DOMAIN APIS ▸ code ▸ Support 7 languages at the moment ▸ English/French/Italian/German/Spanish/Portuguese/Simplified Chinese ▸ Japanese is not be included now // 1. Prepare NLTagger with .sentimentScore scheme let tagger = NLTagger(tagSchemes: [.sentimentScore]) // 2. Set text you want to analyze tagger.string = text // 3. Get result let (sentiment, _) = tagger.tag(at: text.startIndex, unit: .paragraph, scheme: .sentimentScore) print(sentiment!.rawValue) SENTIMENT ANALYSIS
CORE ML WHAT’S CORE ML ▸ Multi platform framework for ML ▸ Optimized for on-device performance ▸ low memory footprint ▸ low power consumption ▸ Protect security and privacy
CORE ML MODEL PERSONALIZATION ▸ Application example: ▸ User can train the existing model with user hand-written for drawing sticker automatically train
kagemiku
nttcom
hirosatogamo
wadayusuke
ykanoh
sumiren
kentosuzuki
oracle4engineer
eliasnogueira
hirokiotsuka
chiroruxx
t88
hirokiigeta
eitanlees
stephaniewalter
yeseniaperezcruz
brettharned
mza
pauljervisheath
erikaheidi
quramy
smashingmag
jakevdp
jlugia
jeffersonlam