ARKitを触りながら振り返る

4BOTBOגࣜձٕࣾज़ຊ෦.PCJMF"QQMJDBUJPOάϧʔϓ ฏࢁஐݾ "3,JUΛ৮Γͳ͕ΒৼΓฦΔ 88%$ J04%$ J04%$+BQBO"GUFS5BML

ࣗݾ঺հ w !FMV w 4BOTBOגࣜձࣾ৽ଔೖࣾ w J04ΤϯδχΞ! w ޷͖ήʔϜ" ෩ܠ#

͓͠ͳ͕͖ "3,JUτϐοΫ 0CKFDU$BQUVSF .PUJPO$BQUVSF ͓·͚

"3,JU 88%$

w ೥ʹ"3,JU͔Βຖ೥Ξοϓσʔτ͕ଓ͘ w ೥ͷJ1BE1SPʹഎ໘5SVF%FQUIΧϝϥ͕౥ࡌ w ೥͔Β1SPγϦʔζʹ-J%"3εΩϟφ౥ࡌ w νοϓͷੑೳ͕޲্͠ෳࡶͳ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ

w ,ʹରԠ w ΧϝϥػೳڧԽ w 1MBOFBODIPSͷਫ਼౓޲্ w ϞʔγϣϯΩϟϓνϟͷੑೳ޲্ w -PDBUJPOBODIPSͷ஍Ҭ֦େ
τϐοΫ %JTDPWFS"3,JU

w ,ʹରԠ$ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ%"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্%શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ
w -PDBUJPOBODIPSͷ஍Ҭ֦େ%஍Ҭ֦େͰ౦ژ௥Ճ

,ʹରԠ w ैདྷը૾Λॖখ͔ͯ͠ΒॲཧΛߦͬͯϨϯμϦϯά͍ͯͨ͠ w ,ͷը૾Λॲཧ͢ΔϝϞϦϦιʔε΍ܭࢉೳྗෆ଍ނ w ࠷৽ͷϋʔυ΢ΣΞύϫʔ & Ͱ,ॲཧ͕Ͱ͖ΔΑ͏ʹͳͬͨ

,ʹରԠ w ,ͰࡱӨ w ͰฏۉԽ͠ॖখ w ໿NTຖʹॲཧ 㲈GQT
w ϨϯμϦϯά #FGPSF IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED

,ʹରԠ w ,ͰࡱӨ w ͰฏۉԽ͠ॖখ w ໿NTຖʹॲཧ 㲈GQT
w ϨϯμϦϯά "GUFS ,NPEF IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED &

w ୯७ͳಈըͰ΋,GQT্͕ݶ w ЋͳॲཧΛ͢ΔͷͰGQT͸ݶք

w ,ʹରԠ%ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ$"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্%શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ

w ,ʹରԠ%ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ%"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্$ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্%શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ

1MBOFBODIPSͷਫ਼౓޲্ w 1MBOFBODIPSʹઃఆͨ͠ΦϒδΣΫτ w ֯౓ɼ෯ͱߴ͞ΛೝࣝՄೳʹ IUUQTEFWFMPQFSBQQMFDPN

w ,ʹରԠ%ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ%"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্$શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ

w ,ʹରԠ%ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ%"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্%શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ
w -PDBUJPOBODIPSͷ஍Ҭ֦େ$஍Ҭ֦େͰ౦ژ௥Ճ

-PDBUJPOBODIPSͷ஍Ҭ֦େ w 714ͱ͍͏Ґஔਪఆٕज़ w Ґஔਪఆʹ͸ۭؒͷεΩϟϯσʔλ͕ඞཁ w (14 ηϯαʔ ө૾ʹΑΓ୺຤Ґஔ޲͖ͷਪఆ w
౦ژؚΊෳ਺ͷ౎ࢢ͕௥Ճ͞Εͨ IUUQTEFWFMPQFSBQQMFDPN

w ,ʹରԠ%ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ%"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্%શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ

w ,ʹରԠ$ϚγϯύϫʔͰॲཧͷޮ཰Խ w ΧϝϥػೳڧԽ$"34FTTJPOதʹ੩ࢭըΛΩϟϓνϟՄೳʹ )%3 w 1MBOFBODIPSͷਫ਼౓޲্%ฏ໘ೝࣝ΍ํ޲౳ͷਫ਼౓͕޲্ w ϞʔγϣϯΩϟϓνϟੑೳ޲্$શମతͳਫ਼౓޲্ ࣖ΋ݕग़Մೳ

͓࿳ͼ

XBSOJOHMJCPCKD"EZMJCJTCFJOHSFBEGSPNQSPDFTTNFNPSZ 5IJTJOEJDBUFTUIBU--%#DPVMEOPUGJOEUIFPOEJTLTIBSFE DBDIFGPSUIJTEFWJDF5IJTXJMMMJLFMZSFEVDFEFCVHHJOH QFSGPSNBODF

࣮ػσόοάΛ͠Α͏ͱ͢Δͱىಈʹd෼͔͔ͬͨΓ "34FTTPO్͕தͰམͪɼσϞΛ༻ҙͰ͖·ͤΜͰͨ͠

w SNSd-JCSBSZ%FWFMPQFS9DPEFJ04a%FWJDF4VQQPSU w J04΋CFUBͰ͸ͳ͘ϦϦʔε൛΁෮ݩ w 9DPEFCFUBΛ࡟আ ϦϦʔε൛ͷΈଘࡏ w 'Ͱͳ͔ͥ໰୊ղܾ

0CKFDU$BQUVSF 88%$

0CKFDU$BQUVSF w 88%$Ͱ3FBMJUZLJUʹ௥Ճ w ϞϊΛෳ਺໘͔ΒࡱӨ w ߹੒ͯ̏͠%ΦϒδΣΫτΛ࡞੒ w ཁ݅-J%"3౥ࡌ J04
"Ҏ߱ w ..BDPS".%(16 IUUQTEFWFMPQFSBQQMFDPN

0CKFDU$BQUVSF "3,JUͱ3FBMJUZ,JU w "3,JUϕʔεͱͳΔϑϨʔϜϫʔΫ w εΩϟϯ΍ΦϒδΣΫτදࣔ w 3FBMJUZ,JUԾ૝ͱݱ࣮ͷ౷߹ w ΦϒδΣΫτͷૢ࡞
w ॲཧɾܭࢉ w %ϨϯμϦϯά IUUQTEFWFMPQFSBQQMFDPN

0CKFDU$BQUVSF w "QQMFͷެࣜαϯϓϧ w ಺༰͸ʜࡱӨॲཧΛݟͯΈΔ

/// - Tag: WillBeginCapture func photoOutput(_ output: AVCapturePhotoOutput, willBeginCaptureFor resolvedSettings:
AVCaptureResolvedPhotoSettings) { maxPhotoProcessingTime = resolvedSettings.photoProcessingTimeRange.start + resolvedSettings.photoProcessingTimeRange.duration } /// - Tag: WillCapturePhoto func photoOutput(_ output: AVCapturePhotoOutput, willCapturePhotoFor resolvedSettings: AVCaptureResolvedPhotoSettings) { willCapturePhotoAnimation() // Retrieve the gravity vector at capture time. if motionManager.isDeviceMotionActive { gravity = motionManager.deviceMotion?.gravity logger.log("Captured gravity vector: \(String(describing: self.gravity))") } guard let maxPhotoProcessingTime = maxPhotoProcessingTime else { return } // Show a spinner if processing time exceeds one second. let oneSecond = CMTime(seconds: 1, preferredTimescale: 1) if maxPhotoProcessingTime > oneSecond { photoProcessingHandler(true) } } ॏྗՃ଎౓Λอଘ

/// - Tag: DidFinishProcessingPhoto func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo:
AVCapturePhoto, error: Error?) { photoProcessingHandler(false) if let error = error { print("Error capturing photo: \(error)") photoData = nil } else { // Cache the HEIF representation of the data. photoData = photo } // Cache the depth data, if it exists, as a disparity map. logger.log("DidFinishProcessingPhoto: photo=\(String(describing: photo))") if let depthData = photo.depthData?.converting(toDepthDataType: kCVPixelFormatType_DisparityFloat32), let colorSpace = CGColorSpace(name: CGColorSpace.linearGray) { let depthImage = CIImage( cvImageBuffer: depthData.depthDataMap, options: [ .auxiliaryDisparity: true ] ) depthMapData = context.tiffRepresentation(of: depthImage, format: .Lf, colorSpace: colorSpace, options: [.disparityImage: depthImage]) } else { logger.error("colorSpace .linearGray not available... can't save depth data!") depthMapData = nil } } ਂ౓ϚοϓΛอଘ

if let photoOutputConnection = self.photoOutput.connection(with: .video) { photoOutputConnection.videoOrientation = videoPreviewLayerOrientation
} var photoSettings = AVCapturePhotoSettings() // Request HEIF photos if supported and enable high-resolution photos. if self.photoOutput.availablePhotoCodecTypes.contains(.hevc) { photoSettings = AVCapturePhotoSettings( format: [AVVideoCodecKey: AVVideoCodecType.hevc]) } // Turn off the flash. The app relies on ambient lighting to avoid specular highlights. if self.videoDeviceInput!.device.isFlashAvailable { photoSettings.flashMode = .off } // Turn on high-resolution, depth data, and quality prioritzation mode. photoSettings.isHighResolutionPhotoEnabled = true photoSettings.isDepthDataDeliveryEnabled = self.photoOutput.isDepthDataDeliveryEnabled photoSettings.photoQualityPrioritization = self.photoQualityPrioritizationMode // Request that the camera embed a depth map into the HEIC output file. photoSettings.embedsDepthDataInPhoto = true // Specify a preview image. if !photoSettings.__availablePreviewPhotoPixelFormatTypes.isEmpty { photoSettings.previewPhotoFormat = [kCVPixelBufferPixelFormatTypeKey: photoSettings.__availablePreviewPhotoPixelFormatTypes.first!, kCVPixelBufferWidthKey: self.previewWidth, kCVPixelBufferHeightKey: self.previewHeight] as [String: Any] logger.log("Found available previewPhotoFormat: \(String(describing: photoSettings.previewPhotoFormat))") } else { logger.warning("Can't find preview photo formats! Not setting...") } // Tell the camera to embed a preview image in the output file. photoSettings.embeddedThumbnailPhotoFormat = [ AVVideoCodecKey: AVVideoCodecType.jpeg, AVVideoWidthKey: self.thumbnailWidth, AVVideoHeightKey: self.thumbnailHeight ] DispatchQueue.main.async { self.isHighQualityMode = photoSettings.isHighResolutionPhotoEnabled && photoSettings.photoQualityPrioritization == .quality } Χϝϥͱը૾ͷઃఆ

ࡱӨը૾ ਂ౓Ϛοϓ ॏྗՃ଎౓ 9 : ;

let inputFolderUrl = URL(fileURLWithPath: inputFolder, isDirectory: true) let configuration =
makeConfigurationFromArguments() logger.log("Using configuration: \(String(describing: configuration))") // Try to create the session, or else exit. var maybeSession: PhotogrammetrySession? = nil do { maybeSession = try PhotogrammetrySession(input: inputFolderUrl, configuration: configuration) logger.log("Successfully created session.") } catch { logger.error("Error creating session: \(String(describing: error))") Foundation.exit(1) } ϑΥϧμΛ౉͚ͩ͢

let entity = try! ModelEntity.load(named: "test.usdz") let anchorEntity = AnchorEntity(plane:
.any) anchorEntity.addChild(entity) arView.scene.addAnchor(anchorEntity) &OUJUZͷ௥Ճ

w ෳ਺ͷࣸਅσʔλ͔Β%ΦϒδΣΫτΛ࡞੒ w ղ૾౓΍৘ใ͕ଟ͍΄Ͳਫ਼౓͕ߴ͘ͳΔ w "3,JUͰ,ࡱӨ͕ରԠ͠Ͳ͏มΘͬͨʁ

let config = ARWorldTrackingConfiguration() // ୺຤͕ѻ͑Δ࠷ߴը࣭ͰͷϑΥʔϚοτΛฦ͢ if let hiResCaptureVideoFormat =
ARWorldTrackingConfiguration.recommendedVideoFormatForHighResolutionFrameCapturing { config.videoFormat = hiResCaptureVideoFormat } // 4kը࣭͕ѻ͑Δ৔߹ͷΈϑΥʔϚοτΛฦ͢ if let res4kCaptureVideoFormat = ARWorldTrackingConfiguration.recommendedVideoFormatFor4KResolution { config.videoFormat = res4kCaptureVideoFormat } "34FTTJPOͰͷө૾ϑΥʔϚοτ

arView.session.captureHighResolutionFrame { arFrame, error in guard let imageBuffer = arFrame?.capturedImage
else { return } let ciImage = CIImage(cvPixelBuffer: imageBuffer) let w = CGFloat(CVPixelBufferGetWidth(imageBuffer)) let h = CGFloat(CVPixelBufferGetHeight(imageBuffer)) let rect:CGRect = CGRect.init(x: 0, y: 0, width: w, height: h) let context = CIContext.init() guard let cgImage = context.createCGImage(ciImage, from: rect) else { return } let uiimage = UIImage(cgImage: cgImage).rotated(by: 90.0 * CGFloat.pi / 180) UIImageWriteToSavedPhotosAlbum(uiimage, self, nil, nil) } "34FTTJPOதͷϑϨʔϜऔಘ "34FTTJPO͕ઃఆΛΑ͠ͳʹͯ͘͠Εͯ "3,JUͷϝιου͸Θ͔Γ΍͍͢

.PUJPO$BQUVSF 88%$

.PUJPO$BQUVSF w 88%$"3,JUͰൃද͞Εͨػೳ w ࣍ݩը૾ͱ-J%"3εΩϟφΛ༻͍ͨਂ౓৘ใ͔Βͷ࢟੎ਪఆ w ਪఆ͞Εͨؔઅ౳ͷ৘ใΛ"3"ODIPS͔ΒऔಘͰ͖Δɽ

"3"ODIPS w Ծ૝ۭ͔ؒΒݟͨݱ࣮ੈքʹ͓͚ΔΦϒδΣΫτͷҐஔ΍޲͖ w ϫʔϧυ࠲ඪܥʹ͓͚Δ࠲ඪϙΠϯτʹͳΔ

"3"ODIPSͷछྨ w "31MBOF"ODIPSݱ࣮ۭ͔ؒΒݕग़͞Εͨฏ໘ͷҐஔ΍ܗঢ়৘ใ w "3*NBHF"ODIPS"3ϚʔΧʔ౳༧Ί༻ҙ͞Εͨը૾ͷ৘ใ w "30CKFDU"ODIPSϙϦΰϯϝογϡ͔Βݕग़ͨ͠෺ཧతͳ৘ใ w "3#PEZ"ODIPSਓͷؔઅͷ৘ใ w
"3'BDF"ODIPSਓͷإͷύʔπ΍޲͖౳ͷ৘ใ w "3.FTI"ODIPS "3(FP"ODIPS "3"QQ$MJQ$PEF"ODIPSʜ

let entity = try! ModelEntity.load(named: "test.usdz") let anchorEntity = AnchorEntity(plane:
.any) anchorEntity.addChild(entity) arView.scene.addAnchor(anchorEntity) &OUJUZͷ௥Ճ

"34$/7JFX "3,JU 4DFOF,JU "34FTTJPJO "3'SBNF "3"ODIPS "37JFX 3FBMJUZ,JU 4DFOF "ODIPS&OUJUZ
.PEFM&OUJUZ "ODIPS&OUJUZ &OUJUZ &OUJUZ $POpHVSBUJPO "3'SBNF "3"ODIPS

"3,JU 3FBMJUZ,JU "37JFX 4DFOF "ODIPS&OUJUZ .PEFM&OUJUZ "ODIPS&OUJUZ &OUJUZ &OUJUZ "34FTTJPJO
$POpHVSBUJPO

.PUJPO$BQUVSF w Ͳ͏࣮ߦ͢Δͷ͔ w ͜Ε΋ެࣜͰ΋༗Γl$BQUVSJOH#PEZ.PUJPOJO%z w ద౰ͳϞσϧͷ༻ҙ w "3#PEZ5SBDLJOH$POpHVSBUJPOͰ"34FTTJPOΛىಈ w
Ϟσϧϩʔυ

Ϟσϧͷ༻ҙ

let configuration = ARBodyTrackingConfiguration() arView.session.run(configuration) arView.scene.addAnchor(characterAnchor) if let entity =
try? Entity.loadBodyTracked(named: "character/robot") { self.character = entity } Ϟσϧͷϩʔυ 4DFOFʹϞσϧͷ"ODIPSΛઃஔ Ϟσϧͷ#PEZ5SBDLFE&OUJUZͷ༻ҙ

func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) { for anchor
in anchors { guard let bodyAnchor = anchor as? ARBodyAnchor else { continue } let bodyPosition = simd_make_float3(bodyAnchor.transform.columns.3) characterAnchor.position = bodyPosition + characterOffset characterAnchor.orientation = Transform(matrix: bodyAnchor.transform).rotation if let character = character, character.parent == nil { characterAnchor.addChild(character) } } } Ϟσϧͱ"3#PEZ"ODIPSͷ઀ଓ ϞσϧଆͰKPJOU/BNF΋߹Θ͍ͤͯΔͷͰɼ"ODIPSʹॏͶΔ͚ͩ

͓·͚

w ϐʔϓϧΦΫϧʔδϣϯ w 3PPN1MBO

ϐʔϓϧΦΫϧʔδϣϯ w ਂ౓৘ใ͔Βਓ෺ͷલޙؔ܎Λೝࣝ w "38PSME5SBDLJOH$POpHVSBUJPO w DPOpHGSBNF4FNBOUJDT<QFSTPO4FHNFOUBUJPO8JUI%FQUI>

let cameraAnchor = AnchorEntity(.camera) self.scene.addAnchor(cameraAnchor) cameraAnchor.addChild(background) background.position = [0,0,-5] എܠΛۭؒͷNޙΖʹઃஔ
ਓ෺ͷલޙҐஔΛೝ͍ࣝͯ͠ΔͷͰɼ ਓ͕̑NޙΖʹ͕͞Δͱਓ͕ӅΕΔ

w ;PPNͷഎܠ͸كʹޙΖͷՈ۩͕ࣸΓࠐΜͩΓ͢Δ w ਓ෺ͷલޙҐஔΛೝࣝͰ͖Δͨͩը૾͔Βͷਓ෺நग़Ͱ͸ͳ͍ w ࠓճͷ88%$ͰJ1IPOFΛ΢ΣϒΧϝϥʹ͢ΔτϐοΫ͋Γ w Ξοϓσʔτʹظ଴

3PPN1MBO w "3,JUΛར༻ͯ͠Χϝϥͱ-J%"3Λར༻ͨ͠ϑϩΞϚοϐϯά͕Մೳ w ڪΖ͘͠؆୯ͳίʔυͰ෦԰ͷ%؃औΓਤ͕࡞੒Մೳ w "QQMFͷαϯϓϧ͕ར༻Մೳ w l$SFBUFB%NPEFMPGBOJOUFSJPSSPPNCZHVJEJOHUIFVTFSUISPVHI BO"3FYQFSJFODFz

let roomCaptureView = RoomCaptureView(frame: view.frame) view.addSubview(roomCaptureView) roomCaptureView.captureSession.run(configuration: .init()) roomCaptureView.captureSession.stop() %FMFHBUFͰ׬ྃ࣌ʹ෦԰ͷΩϟϓνϟʔσʔλ͕ड͚औΕΔ

"3,JU·ͱΊ w ػೳ͕ॆ࣮͠؆୯ͳࣄͰ͋Ε͹༻ҙʹࢼͤΔ؀ڥʹͳ͖ͬͯͨ w 75VCFS౳J1IPOFΛ࢖ͬͯΩϟϥΫλʔϞσϧΛಈ͔ͤΔ෯͕޿͕ͬͨ w ͨͩࢼ͚ͩ͢Ͱ΋ਫ਼౓͕ߴ͍ͷͰ݁ߏΫΦϦςΟߴ͘ײָ͍ͯ͡͠ ͥͻ"3,JUΛ৮ͬͨ͜ͱແ͍ํ͸৮ΕͯΈ͍ͯͩ͘͞ʂ

5IBOLT

ARKitを触りながら振り返る

ARKitを触りながら振り返る

More Decks by Sansan

Other Decks in Technology

Featured

Transcript