Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ARKit Maniacs

satoshi0212
September 01, 2018

ARKit Maniacs

satoshi0212

September 01, 2018
Tweet

More Decks by satoshi0212

Other Decks in Technology

Transcript

  1. "3,JU.BOJBDT
    גࣜձࣾ"CFNB57෰෦ஐ
    TBUPTIJ!TINEFWFMPQ
    J04%$
    J04%$

    View full-size slide

  2. "3,JU.BOJBDT
    גࣜձࣾ"CFNB57෰෦ஐ
    TBUPTIJ!TINEFWFMPQ
    J04%$
    J04%$

    View full-size slide

  3. ஫εϥΠυ಺༰͸CFUB൛ͷެ։৘ใΛݩʹ͍ͯ͠·͢ɻ

    View full-size slide

  4. 4BWJOHBOEMPBEJOHNBQT
    8PSMEUSBDLJOHFOIBODFNFOUT
    &OWJSPONFOUUFYUVSJOH
    *NBHFUSBDLJOH
    0CKFDUEFUFDUJPO
    (B[FBOEUPOHVF
    ARKit 2 - Summary

    View full-size slide

  5. TPVSDF8IBU`T/FXJO"3,JU IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED

    View full-size slide

  6. 8PSME.BQ1FSTJTUFODFͷ࣮ݧ
    &OWJSPONFOU5FYUVSJOHͷ࣮ݧ
    ฏ໘Λݟ͚͔ͭͯΒͷ޻෉

    View full-size slide

  7. 8PSME.BQ1FSTJTUFODFͷ࣮ݧ
    &OWJSPONFOU5FYUVSJOHͷ࣮ݧ
    ฏ໘Λݟ͚͔ͭͯΒͷ޻෉

    View full-size slide

  8. 4BWJOHBOEMPBEJOHNBQT
    8PSMEUSBDLJOHFOIBODFNFOUT
    &OWJSPONFOUUFYUVSJOH
    *NBHFUSBDLJOH
    0CKFDUEFUFDUJPO
    (B[FBOEUPOHVF
    ARKit 2 - Summary

    View full-size slide

  9. "38PSME4IBSJOHBOE1FSTJTUFODF

    View full-size slide

  10. ͲΕ͚ͩͷػೳΛ͍࣋ͬͯΔͷ͔

    View full-size slide

  11. ੈք͸γΣΞͰ͖Δͷ͔

    View full-size slide

  12. ௚ײʮͰ͖ͳͦ͞͏ʯ

    View full-size slide

  13. ͦ΋ͦ΋ԿΛอଘ͍ͯ͠Δͷ͔

    View full-size slide

  14. TPVSDF8IBU`T/FXJO"3,JU IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED

    View full-size slide

  15. import ARKit
    @available(iOS 12.0, *)
    open class ARWorldMap : NSObject, NSCopying, NSSecureCoding {
    /**
    The position of the center of the mapped world in meters.
    */
    open var center: simd_float3 { get }
    /**
    The extent of the mapped world in meters.
    */
    open var extent: simd_float3 { get }
    /**
    A list of anchors in the map.
    */
    open var anchors: [ARAnchor]
    /**
    The feature points in the map.
    */
    open var rawFeaturePoints: ARPointCloud { get }
    }

    View full-size slide

  16. import ARKit
    @available(iOS 11.0, *)
    open class ARAnchor : NSObject, ARAnchorCopying, NSSecureCoding {
    /**
    Unique identifier of the anchor.
    */
    open var identifier: UUID { get }
    /**
    An optional name used to associate with the anchor.
    */
    @available(iOS 12.0, *)
    open var name: String? { get }
    /**
    The transformation matrix that defines the anchor’s rotation, translation and scale in world coordinates.
    */
    open var transform: simd_float4x4 { get }
    /**
    Initializes a new anchor object.
    @param transform The transformation matrix that defines the anchor’s rotation, translation and scale in world coordinates.
    */
    public init(transform: simd_float4x4)
    /**
    Initializes a new anchor object with the provided identifier and name.
    @param name A name to associate with the anchor.
    @param transform The transformation matrix that defines the anchor’s rotation, translation and scale in world coordinates.
    */
    @available(iOS 12.0, *)
    public init(name: String, transform: simd_float4x4)
    }

    View full-size slide

  17. import ARKit
    @available(iOS 11.0, *)
    open class ARPointCloud : NSObject, NSSecureCoding {
    /**
    The number of points in the point cloud.
    */
    open var __count: Int { get }
    /**
    The 3D points comprising the point cloud.
    */
    open var __points: UnsafePointer { get }
    /**
    The 3D point identifiers comprising the point cloud.
    */
    open var __identifiers: UnsafePointer { get }
    }
    @available(iOS 11.0, *)
    extension ARPointCloud {
    /**
    The 3D points comprising the point cloud.
    */
    @nonobjc public var points: [vector_float3] { get }
    /**
    The 3D point identifiers comprising the point cloud.
    */
    @nonobjc public var identifiers: [UInt64] { get }
    }

    View full-size slide

  18. sceneView.session.getCurrentWorldMap { worldMap, error in
    guard let map = worldMap else { return }
    do {
    let data = try NSKeyedArchiver.archivedData(withRootObject: map,
    requiringSecureCoding: true)
    try data.write(to: self.mapSaveURL, options: [.atomic])
    } catch {
    fatalError("Can't save map: \(error.localizedDescription)")
    }
    }

    View full-size slide

  19. do {
    guard let worldMap = try NSKeyedUnarchiver.unarchivedObject(ofClass: ARWorldMap.self,
    from: data)
    else { fatalError("No ARWorldMap in archive.") }
    return worldMap
    } catch {
    fatalError("Can't unarchive ARWorldMap from file data: \(error)")
    }

    View full-size slide

  20. let configuration = self.defaultConfiguration // this app's standard settings
    configuration.initialWorldMap = worldMap
    sceneView.session.run(configuration, options: [.resetTracking, .removeExistingAnchors])

    View full-size slide

  21. ੈք͸γΣΞͰ͖Δͷ͔
    ͸ର৅͕େ͖͍ͷͰ

    View full-size slide

  22. ݐ෺͸γΣΞͰ͖Δͷ͔ʁ

    View full-size slide

  23. ֊ݐͯϏϧ಺ͷ͍͔ͭ͘ͷΤϦΞͰ࣮ଌ

    View full-size slide

  24. γϯϓϧͳ෦԰
    GFBUVSFT BODIPST ,#
    ࡶଟͳ෦԰

    GFBUVSFT BODIPST .#
    ϥ΢ϯδ

    GFBUVSFT BODIPST .#
    ݐ෺ͷݰ͔ؔΒ࿓Լͱ֊ஈΛ௨Γ'·Ͱ

    GFBUVSFT BODIPST ,#

    View full-size slide

  25. ࡶଟͳ෦԰ͷೖޱΛੈքͷத৺ͱͯ͠Έͨ
    ίʔώʔΧοϓ഑ஔ

    View full-size slide

  26. ݐ෺ͰγΣΞ੒ޭ

    View full-size slide

  27. ߴ௿ࠩ͋ΔൣғͰӬଓԽ෮ݩ
    σʔλαΠζͷ໨ॲ
    ਺.#ఔ౓Ͱ͸4FTTJPO։࢝଎౓ʹେࠩͳ͠
    த৺఺Ͱ͸ͳͯ͘΋։࢝Մೳ
    อଘ͢Δ৘ใ͸ׂͱࣗ༝

    View full-size slide

  28. ߴ௿ࠩ͋ΔൣғͰӬଓԽ෮ݩ
    ˢ'΁ͷҠಈͰ΋େ͖ͳζϨͳ͠
    σʔλαΠζͷ໨ॲ
    ˢ'΁ͷҠಈέʔεͰ͸,#
    ਺.#ఔ౓Ͱ͸4FTTJPOͷ։࢝଎౓ʹେࠩͳ͠
    ˢ4FTTJPO։࢝͸Tɻͨͩ͠SFMPDBMJ[F·Ͱʹ͸਺ඵɻ
    த৺఺Ͱ͸ͳͯ͘΋SFMPDBMJ[F։࢝Մೳ
    ˢ'Λத৺ͱͨ͠ੈքͰݰؔͰSFMPDBMJ[FՄ
    อଘ͢Δ৘ใ͸ׂͱࣗ༝
    ˢ4OBQ4IPU"ODIPSͳͲ

    View full-size slide

  29. class SnapshotAnchor: ARAnchor {
    let imageData: Data
    convenience init?(capturing view: ARSCNView) {
    guard let frame = view.session.currentFrame
    else { return nil }
    let image = CIImage(cvPixelBuffer: frame.capturedImage)
    let orientation = CGImagePropertyOrientation(cameraOrientation:
    UIDevice.current.orientation)
    let context = CIContext(options: [.useSoftwareRenderer: false])
    guard let data = context.jpegRepresentation(of: image.oriented(orientation),
    colorSpace:
    CGColorSpaceCreateDeviceRGB(),
    options:
    [kCGImageDestinationLossyCompressionQuality as CIImageRepresentationOption: 0.7])
    else { return nil }
    self.init(imageData: data, transform: frame.camera.transform)
    }
    ...

    View full-size slide

  30. ੈքγΣΞ࣮ݱͷલʹौ୩γΣΞ͔
    ࣍ͷน͸σʔλαΠζ

    View full-size slide

  31. ଓใ͓଴ͪԼ͍͞

    View full-size slide

  32. 8PSME.BQ1FSTJTUFODFͷ࣮ݧ
    &OWJSPONFOU5FYUVSJOHͷ࣮ݧ
    ฏ໘Λݟ͚͔ͭͯΒͷ޻෉

    View full-size slide

  33. 4BWJOHBOEMPBEJOHNBQT
    8PSMEUSBDLJOHFOIBODFNFOUT
    &OWJSPONFOUUFYUVSJOH
    *NBHFUSBDLJOH
    0CKFDUEFUFDUJPO
    (B[FBOEUPOHVF
    ARKit 2 - Summary

    View full-size slide

  34. "EEJOH3FBMJTUJD3FqFDUJPOTUPBO"3&YQFSJFODF

    View full-size slide

  35. TPVSDF"EEJOH3FBMJTUJD3FqFDUJPOTUPBO"3&YQFSJFODF IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBEEJOH@SFBMJTUJD@SFqFDUJPOT@UP@BO@BS@FYQFSJFODF

    View full-size slide

  36. let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = .horizontal
    configuration.environmentTexturing = .automatic
    sceneView.session.run(configuration)

    View full-size slide

  37. let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = .horizontal
    configuration.environmentTexturing = .automatic
    sceneView.session.run(configuration)

    View full-size slide

  38. "3,JUHFOFSBUFTFOWJSPONFOUUFYUVSFTCZDPMMFDUJOHDBNFSBJNBHFSZ
    EVSJOHUIF"3TFTTJPO#FDBVTF"3,JUDBOOPUTFFUIFTDFOFJOBMM
    EJSFDUJPOT JUVTFT
    NBDIJOFMFBSOJOHUPFYUSBQPMBUFB
    SFBMJTUJDFOWJSPONFOUGSPNBWBJMBCMFJNBHFSZ
    TPVSDF"EEJOH3FBMJTUJD3FqFDUJPOTUPBO"3&YQFSJFODF IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBEEJOH@SFBMJTUJD@SFqFDUJPOT@UP@BO@BS@FYQFSJFODF

    View full-size slide

  39. let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = .horizontal
    configuration.environmentTexturing = .manual
    sceneView.session.run(configuration)

    View full-size slide

  40. let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = .horizontal
    configuration.environmentTexturing = .manual
    sceneView.session.run(configuration)

    View full-size slide

  41. TPVSDF"EEJOH3FBMJTUJD3FqFDUJPOTUPBO"3&YQFSJFODF IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBEEJOH@SFBMJTUJD@SFqFDUJPOT@UP@BO@BS@FYQFSJFODF

    View full-size slide

  42. func updateEnvironmentProbe(atTime time: TimeInterval) {
    guard let object = virtualObject else { return }
    ...
    // Make sure the probe encompasses the object and provides some surrounding area to
    appear in reflections.
    var extent = object.extents * object.simdScale
    extent.x *= 3 // Reflect an area 3x the width of the object.
    extent.z *= 3 // Reflect an area 3x the depth of the object.
    // Also include some vertical area around the object, but keep the bottom of the probe
    at the
    // bottom of the object so that it captures the real-world surface underneath.
    let verticalOffset = float3(0, extent.y, 0)
    let transform = float4x4(translation: object.simdPosition + verticalOffset)
    extent.y *= 2
    // Create the new environment probe anchor and add it to the session.
    let probeAnchor = AREnvironmentProbeAnchor(transform: transform, extent: extent)
    sceneView.session.add(anchor: probeAnchor)
    ...
    }

    View full-size slide

  43. func updateEnvironmentProbe(atTime time: TimeInterval) {
    guard let object = virtualObject else { return }
    ...
    // Make sure the probe encompasses the object and provides some surrounding area to
    appear in reflections.
    var extent = object.extents * object.simdScale
    extent.x *= 3 // Reflect an area 3x the width of the object.
    extent.z *= 3 // Reflect an area 3x the depth of the object.
    // Also include some vertical area around the object, but keep the bottom of the probe
    at the
    // bottom of the object so that it captures the real-world surface underneath.
    let verticalOffset = float3(0, extent.y, 0)
    let transform = float4x4(translation: object.simdPosition + verticalOffset)
    extent.y *= 2
    // Create the new environment probe anchor and add it to the session.
    let probeAnchor = AREnvironmentProbeAnchor(transform: transform, extent: extent)
    sceneView.session.add(anchor: probeAnchor)
    ...
    }

    View full-size slide

  44. // MARK: - ARSCNViewDelegate
    func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
    updateEnvironmentProbe(atTime: time)
    }

    View full-size slide

  45. ͳΜͱػցֶशͰݟ͑ͳ͍ྖҬΛࣗಈੜ੒ͯ͘͠Ε͍ͯΔ
    NBOVBMͰ͸ൣғͷϕετϓϥΫςΟε͕ࣔ͞Ε͍ͯΔ
    ಛఆ༻్ͳ͚Ε͹BVUPNBUJDͰྑ͍ͷͰ͸
    ͦͷଞʹ΋5JQT͋Γ

    View full-size slide

  46. "WPJEWJSUVBMDPOUFOUUIBUSFRVJSFTBDDVSBUFSFqFDUJPOT TVDI
    BTNJSSPSpOJTITVSGBDFT
    )BOEMFNPWJOHPCKFDUT
    %PO`UHFOFSBUFFOWJSPONFOUUFYUVSFTUPPPGUFO
    "WPJEBCSVQUUSBOTJUJPOTCFUXFFOEJ⒎FSFOUFOWJSPONFOU
    UFYUVSFT
    Use Environment Texturing Wisely

    View full-size slide

  47. 8PSME.BQ1FSTJTUFODFͷ࣮ݧ
    &OWJSPONFOU5FYUVSJOHͷ࣮ݧ
    ฏ໘Λݟ͚͔ͭͯΒͷ޻෉

    View full-size slide

  48. ฏ໘ൃݟޙ
    λοϓ͢Δͱݕ஌͸ࢭ·Γ
    αΠζͱ޲͖ͷௐ੔Ϟʔυ΁

    View full-size slide

  49. %FTJHO"3FYQFSJFODFTGPSQSFEJDUBCMFMJHIUJOHDPOEJUJPOT
    6TFUSBDLJOHRVBMJUZJOGPSNBUJPOUPQSPWJEFVTFS
    GFFECBDL
    "MMPXUJNFGPSQMBOFEFUFDUJPOUPQSPEVDFDMFBSSFTVMUT
    BOEEJTBCMFQMBOFEFUFDUJPOXIFOZPVIBWFUIFSFTVMUTZPV
    OFFE
    Best Practices and Limitations

    View full-size slide

  50. 8PSME.BQ1FSTJTUFODFͷ࣮ݧ
    ˠσʔλαΠζͷ՝୊
    &OWJSPONFOU5FYUVSJOHͷ࣮ݧ
    ˠಛఆ༻్ͳ͚Ε͹BVUPNBUJDͰྑ͍ͷͰ͸
    ฏ໘Λݟ͚͔ͭͯΒͷ޻෉
    ˠετϨεݮΒ͢࢓૊ΈͷҰྫ

    View full-size slide

  51. 6OEFSTUBOEJOH8PSME5SBDLJOHJO"3,JU
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUVOEFSTUBOEJOH@XPSME@USBDLJOH@JO@BSLJU
    4XJGU4IPU$SFBUJOHB(BNFGPS"VHNFOUFE3FBMJUZ
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUTXJGUTIPU@DSFBUJOH@B@HBNF@GPS@BVHNFOUFE@SFBMJUZ
    $SFBUJOHB1FSTJTUFOU"3&YQFSJFODF
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUDSFBUJOH@B@QFSTJTUFOU@BS@FYQFSJFODF
    "EEJOH3FBMJTUJD3FqFDUJPOTUPBO"3&YQFSJFODF
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBEEJOH@SFBMJTUJD@SFqFDUJPOT@UP@BO@BS@FYQFSJFODF
    "SDIJWJOH8PSME.BQ%BUBGPS1FSTJTUFODFPS4IBSJOH
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBSXPSMENBQBSDIJWJOH@XPSME@NBQ@EBUB@GPS@QFSTJTUFODF@PS@TIBSJOH
    8IBU`T/FXJO"3,JU
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED
    6OEFSTUBOEJOH"3,JU5SBDLJOHBOE%FUFDUJPO
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED
    *OTJEF4XJGU4IPU$SFBUJOHBO"3(BNF
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED
    $SFBUJOH(SFBU"3&YQFSJFODFT
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXED
    ࢀߟ

    View full-size slide