ML Kit: 모바일_개발자를_위한_머신러닝

Transcript

1. 4BOHLZPPO
   /BN
 $IJFG
   "SDIJUFDU
   BU
   OBNTBOHLZPPO!OCUDPN
   ML Kit: 모바일 개발자를 위한 머신러닝
5. Mobile first to AI first
6. Solving problems with 
 AI for everyone
11. On-device ML
 ೙ਃࢿ 낮은 지연 시간, 서버 호출 없음 오프라인에서 동작 가능 데이터가 디바이스 측에 보관 저전력 고효율 디바이스의 모든 센서에 접근 허용
12. On-device ML
 য۰਑ 메모리 사용 제약 배터리 소모를 낮추기 위해 
 에너지 절감 작은 컴퓨팅 파워
18. ML Kit Core ML AR Core ARKit vs.
20. Android iOS
21. 기본 API 커스텀 모델
22. 텍스트 인식 (OCR) 얼굴 감지 바코드 스캔 이미지 라벨 지정 랜드마크 인식 스마트 회신 (제공 예정) 기본 API
23. VISION VISION
25. 
    IUUQTHJUIVCDPNGJSFCBTFRVJDLTUBSUBOESPJEUSFFNBTUFSNMLJU
    
26. 텍스트 인식
27. 얼굴 감지
28. 바코드 스캔
29. 이미지 라벨 지정
30. 동적 모델 다운로드 파이어베이스 원격 설정을
 통한 A/B 테스트 모델 변환과 압축
 (제공 예정) 커스텀 모델
31. 
    IUUQTDPEFMBCTEFWFMPQFSTHPPHMFDPNDPEFMBCT
    
32. 
    IUUQTDPEFMBCTEFWFMPQFSTHPPHMFDPNDPEFMBCTNMLJUBOESPJE
    기본 API
38. On-device Cloud
39. 기기별 클라우드 기반
44. <build.gradle>
 dependencies { // ... implementation 'com.google.firebase:firebase-ml-vision:15.0.0' } (ӂ੢) On-device੄ ҃਋ জ ࢸ஖ റ ݽ؛ਸ ੗ز ׮਍۽٘ೞب۾ ݺद <meta-data android:name="com.google.firebase.ml.vision.DEPENDENCIES" android:value="text" /> <!-- To use multiple models: android:value="text,model2,model3" -->
45. 이미지 로드
46. 1. Bitmap ё୓ীࢲ FirebaseVisionImage ё୓ܳ ٜ݅۰ݶ FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap); 2. ӝӝ ஠ݫۄীࢲ ੉޷૑ܳ ஭୊ೡ ٸ৬ э੉ FirebaseVisionImage ё୓ܳ media.Image ё୓ীࢲ ٜ݅۰ݶ private static final SparseIntArray ORIENTATIONS = new SparseIntArray(); static { ORIENTATIONS.append(Surface.ROTATION_0, 90); ORIENTATIONS.append(Surface.ROTATION_90, 0); ORIENTATIONS.append(Surface.ROTATION_180, 270); ORIENTATIONS.append(Surface.ROTATION_270, 180); } FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
47. 3. ByteBuffer ژח ߄੉౟ ߓৌীࢲ FirebaseVisionImage ё୓ܳ ٜ݅۰ݶ FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder() .setWidth(1280) .setHeight(720) .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21) .setRotation(rotation) .build(); FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata); // Or FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata); 4. ౵ੌীࢲ FirebaseVisionImage ё୓ܳ ٜ݅۰ݶ FirebaseVisionImage image; try { image = FirebaseVisionImage.fromFilePath(context, uri); } catch (IOException e) { e.printStackTrace(); }
48. On-device 인식
49. FirebaseVisionTextDetector detector = FirebaseVision.getInstance() .getVisionTextDetector(); Task<FirebaseVisionText> result = detector.detectInImage(image) .addOnSuccessListener(new OnSuccessListener<FirebaseVisionText>() { @Override public void onSuccess(FirebaseVisionText firebaseVisionText) { // Task completed successfully // ... } }) .addOnFailureListener( new OnFailureListener() { @Override public void onFailure(@NonNull Exception e) { // Task failed with an exception // ... } });
50. for (FirebaseVisionText.Block block: firebaseVisionText.getBlocks()) { Rect boundingBox = block.getBoundingBox(); Point[] cornerPoints = block.getCornerPoints(); String text = block.getText(); for (FirebaseVisionText.Line line: block.getLines()) { // ... for (FirebaseVisionText.Element element: line.getElements()) { // ... } } }
51. Cloud 인식
52. FirebaseVisionCloudDetectorOptions options = new FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build(); FirebaseVisionCloudTextDetector detector = FirebaseVision.getInstance() .getVisionCloudTextDetector();
 // Or, to change the default settings: FirebaseVisionLabelDetector detector = FirebaseVision.getInstance() .getVisionLabelDetector(options);
53. Task<FirebaseVisionCloudText> result = detector.detectInImage(image) .addOnSuccessListener(new OnSuccessListener<FirebaseVisionCloudText>() { @Override public void onSuccess(FirebaseVisionCloudText firebaseVisionCloudText) { // Task completed successfully // ... } }) .addOnFailureListener(new OnFailureListener() { @Override public void onFailure(@NonNull Exception e) { // Task failed with an exception // ... } });
54. String recognizedText = firebaseVisionCloudText.getText(); for (FirebaseVisionCloudText.Page page: firebaseVisionCloudText.getPages()) { List<FirebaseVisionCloudText.DetectedLanguage> languages = page.getTextProperty().getDetectedLanguages(); int height = page.getHeight(); int width = page.getWidth(); float confidence = page.getConfidence(); for (FirebaseVisionCloudText.Block block: page.getBlocks()) { Rect boundingBox = block.getBoundingBox(); List<FirebaseVisionCloudText.DetectedLanguage> blockLanguages = block.getTextProperty().getDetectedLanguages(); float blockConfidence = block.getConfidence(); // And so on: Paragraph, Word, Symbol } }
58. 
    IUUQTDPEFMBCTEFWFMPQFSTHPPHMFDPNDPEFMBCTNMLJUBOESPJEDVTUPNNPEFM
    
    커스텀 모델
59. <build.gradle>
 dependencies { // ... implementation ‘com.google.firebase:firebase-ml-model-interpreter:15.0.0’ } apply plugin: ‘com.google.gms.google-services’
65. /** * An instance of the driver class to run model inference with Firebase. */ private FirebaseModelInterpreter mInterpreter; /** * Data configuration of input & output data of model. */ private FirebaseModelInputOutputOptions mDataOptions;
66. mDataOptions = new FirebaseModelInputOutputOptions.Builder() .setInputFormat(0, FirebaseModelDataType.BYTE, inputDims) .setOutputFormat(0, FirebaseModelDataType.BYTE, outputDims) .build(); FirebaseModelDownloadConditions conditions = new FirebaseModelDownloadConditions .Builder() .requireWifi() .build(); FirebaseLocalModelSource localModelSource = new FirebaseLocalModelSource.Builder("asset") .setAssetFilePath(LOCAL_MODEL_ASSET).build(); FirebaseCloudModelSource cloudSource = new FirebaseCloudModelSource.Builder (HOSTED_MODEL_NAME) .enableModelUpdates(true) .setInitialDownloadConditions(conditions) .setUpdatesDownloadConditions(conditions) .build(); FirebaseModelManager manager = FirebaseModelManager.getInstance(); manager.registerLocalModelSource(localModelSource); manager.registerCloudModelSource(cloudSource); FirebaseModelOptions modelOptions = new FirebaseModelOptions.Builder() .setCloudModelName(HOSTED_MODEL_NAME) .setLocalModelName("asset") .build(); mInterpreter = FirebaseModelInterpreter.getInstance(modelOptions);
67. Log.e(TAG, "Image classifier has not been initialized; Skipped."); return; } // Create input data. ByteBuffer imgData = convertBitmapToByteBuffer(mSelectedImage, mSelectedImage.getWidth(), mSelectedImage.getHeight()); try { FirebaseModelInputs inputs = new FirebaseModelInputs.Builder().add(imgData).build(); // Here's where the magic happens!! mInterpreter .run(inputs, mDataOptions) .continueWith( new Continuation<FirebaseModelOutputs, List<String>>() { @Override public List<String> then(Task<FirebaseModelOutputs> task) { byte[][] labelProbArray = task.getResult() .<byte[][]>getOutput(0); List<String> topLabels = getTopLabels(labelProbArray); mGraphicOverlay.clear(); GraphicOverlay.Graphic labelGraphic = new LabelGraphic (mGraphicOverlay, topLabels); mGraphicOverlay.add(labelGraphic); return topLabels; } }); } catch (FirebaseMLException e) { e.printStackTrace(); showToast("Error running model inference");
71. Firebase A/B Testing Cloud Firestore Firebase Performance Google Analytics
72. = 변곡점!
73. 감사합니다