CocoaHeads SKG #4 - Getting Dirty With Realm.io

Transcript

1. Getting Dirty With Realm.io
   Getting Dirty With Realm.io
   @attheodo
   @attheodo
   

   View Slide

2. What is Realm?
   What is Realm?
   Realm is a
   Realm is a truly
   truly mobile database.
   mobile database.
   Realm is a
   Realm is a replacement
   replacement for SQLite, Core
   for SQLite, Core
   Data, JSON/Property List files.
   Data, JSON/Property List files.
   Realm is easy to use,
   Realm is easy to use, fast
   fast, cross-platform
   , cross-platform
   and well supported.
   and well supported.
   1
   1
   

   View Slide

3. Realm is a Mobile Database
   Realm is a Mobile Database
   Models!
   Models!
   Relationships (one-to-one, one-to-many)
   Relationships (one-to-one, one-to-many)
   CRUD!
   CRUD!
   Create
   Create
   Read
   Read
   Update
   Update
   Delete
   Delete
   Queries!
   Queries!
   Migrations!
   Migrations!
   ACID Transactions
   ACID Transactions
   

   View Slide

4. Realm is a Replacement for
   Realm is a Replacement for
   SQLite
   FMDB is cool...
   ... said someone in 2011.
   Core Data
   Learning curve anyone?
   ORM on top of SQLite
   Complex API
   Offline JSON/Plist files
   Ok for small and simple transactions
   Limited Flexibility
   Too much I/O
   Thread-safety? ACID?
   

   View Slide

5. Realm is easy to use
   Realm is easy to use
   Realm is not an ORM on top of
   SQLite/CoreData/x
   Custom C++ core with bit-packing, caching,
   vectorization and zero-copy architecture.
   It has it's own conventions and data-structures
   that are easy to grasp and use.
   Basically just 3 classes (Object, Arrays and
   Realms)
   You can use Realm models convention to write
   all your app models if you want.
   More later...
   

   View Slide

6. Realm is Fast
   Realm is Fast
   

   View Slide

7. Realm is Fast
   Realm is Fast
   

   View Slide

8. Realm is Fast
   Realm is Fast
   

   View Slide

9. Realm is Cross-Platform
   Realm is Cross-Platform
   Objective-C, Swift
   iOS, watchOS, tvOS
   React-Native
   Android
   

   View Slide

10. Realm Internals
    Realm Internals
    Realm is truly innovative
    Realm is truly innovative
    MVCC
    MVCC
    Native Links
    Native Links
    Crash Safety
    Crash Safety
    Zero Copy
    Zero Copy
    True Lazy-Loading
    True Lazy-Loading
    Built from scratch with mobile platforms and
    Built from scratch with mobile platforms and
    limitations in mind
    limitations in mind
    ​
    ​
    Why aim to be massively distributed?
    Why aim to be massively distributed?
    Why aim to be shardable across instances?
    Why aim to be shardable across instances?
    Why be low latency over connection sockets?
    Why be low latency over connection sockets?
    

    View Slide

11. Realm Internals
    Realm Internals
    Realm is objects all the way down.
    Realm is objects all the way down.
    Most existing solutions are ORMs on top of
    Most existing solutions are ORMs on top of
    something.
    something.
    ORMs abstract what's going underneath
    ORMs abstract what's going underneath
    Objects become records/rows in tables with
    Objects become records/rows in tables with
    foreign keys and primary keys.
    foreign keys and primary keys.
    The abstraction kind of falls apart because
    The abstraction kind of falls apart because
    you need to start traversing relationships
    you need to start traversing relationships
    using expensive operations.
    using expensive operations.
    

    View Slide

12. Realm Internals
    Realm Internals
    When you init a Realm, you're already "connected"
    When you init a Realm, you're already "connected"
    to the database
    to the database
    Actually a fraction of the database is being
    Actually a fraction of the database is being
    memory mapped, there's no socket whatsoever.
    memory mapped, there's no socket whatsoever.
    As soon as you add an object (record) in Realm, it
    As soon as you add an object (record) in Realm, it
    becomes an accessor.
    becomes an accessor.
    Once you start reading properties from it, you are
    Once you start reading properties from it, you are
    not accessing your iVars!
    not accessing your iVars!
    You access raw database values avoiding lots of
    You access raw database values avoiding lots of
    memory copying.
    memory copying.
    

    View Slide

13. Realm Internals (B+ Trees, Native Links)
    Realm Internals (B+ Trees, Native Links)
    Writer
    Reader
    HEAD
    HEAD~1
    The file structure is just a tree of
    The file structure is just a tree of
    links.
    links.
    Raw pointers to objects.
    Raw pointers to objects.
    Same if you're querying for an Int,
    Same if you're querying for an Int,
    String or relationship!
    String or relationship!
    

    View Slide

14. Realm Internals (Crash Safety)
    Realm Internals (Crash Safety)
    HEAD
    HEAD~1
    As you make changes, you're forking the tree.
    The top level pointer "HEAD" is always pointing to the
    latest non-corrupt tree.
    When Realm verifies the newest write transaction
    (fsync()) it move's the top level pointer and get a new
    "official version"
    

    View Slide

15. Realm Internals (Zero Copy)
    Realm Internals (Zero Copy)
    How most ORMs deal with data
    Data sits on disk.
    Data sits on disk.
    You access a CoreData model's property:
    You access a CoreData model's property:
    Translate request into a set of SQL statements.
    Translate request into a set of SQL statements.
    Create database connection.
    Create database connection.
    Perform the SQL query.
    Perform the SQL query.
    Read all the data from the rows that match the
    Read all the data from the rows that match the
    query and bring it to memory.
    query and bring it to memory.
    Deserialize from the disk format to something you
    Deserialize from the disk format to something you
    can represent in memory (align bits).
    can represent in memory (align bits).
    Convert to the language type.
    Convert to the language type.
    

    View Slide

16. Realm Internals (Zero Copy)
    Realm Internals (Zero Copy)
    How Realm does it
    The data is still on disk but it's memory mapped.
    The data is still on disk but it's memory mapped.
    You can access any offset in the file as if it was in
    You can access any offset in the file as if it was in
    memory, although it's not. It's in virtual memory.
    memory, although it's not. It's in virtual memory.
    Objects all the way down, everything is word-aligned
    Objects all the way down, everything is word-aligned
    (limitations)
    (limitations)
    The core file format is readable in memory without
    The core file format is readable in memory without
    requiring deseriliazation (core Realm principle)
    requiring deseriliazation (core Realm principle)
    No copy to memory!
    No copy to memory!
    All you need is calculate the offset of the data to read
    All you need is calculate the offset of the data to read
    in the mmap, read the value from offset up to offset +
    in the mmap, read the value from offset up to offset +
    propery.length and return the raw value from the
    propery.length and return the raw value from the
    property access!
    property access!
    

    View Slide

17. Realm Internals (True Lazy-Loading)
    Realm Internals (True Lazy-Loading)
    Hardware limitations don't allow to read just 1 bit form
    Hardware limitations don't allow to read just 1 bit form
    disk
    disk
    Even if you want a Boolean property, you need to
    Even if you want a Boolean property, you need to
    load the disk's page size, nothing smaller than that.
    load the disk's page size, nothing smaller than that.
    Most databases store things on a horizontal, linear
    Most databases store things on a horizontal, linear
    level.
    level.
    When you want a property from SQLite, you still
    When you want a property from SQLite, you still
    have to load the entire row because it's stored
    have to load the entire row because it's stored
    contiguously in the file.
    contiguously in the file.
    Realm stores properties contiguously linked at the
    Realm stores properties contiguously linked at the
    vertical level in vectors (columns vs rows)
    vertical level in vectors (columns vs rows)
    An object's property is never in an iVar unless you
    An object's property is never in an iVar unless you
    actually try to read it or modify it.
    actually try to read it or modify it.
    

    View Slide

18. Installation
    Installation
    You guessed it. It's a Cocoapod.
    You guessed it. It's a Cocoapod.
    pod 'RealmSwift'
    

    View Slide

19. Realm Models
    Realm Models
    Supported Property Types
    Bool
    Int8, Int16, Int32, Int64
    Storing optional numbers is done using
    RealmOptional
    Double, Float
    String (can be optional)
    NSDate (truncated to the second, can be
    optional)
    NSData (can be optional)
    

    View Slide

20. Realm Models
    Realm Models
    import RealmSwift
    class User: Object {
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    }
    Auto-incrementing id?
    Auto-incrementing id?
    NOPE.
    NOPE.
    

    View Slide

21. Realm Models
    Realm Models
    import RealmSwift
    class User: Object {
    dynamic var id = ""
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    override static func primaryKey() -> String? {
    return "id"
    }
    }
    Or just map the id to your server
    Or just map the id to your server
    database's id.
    database's id.
    

    View Slide

22. Realm Models
    Realm Models
    import RealmSwift
    class Post: Object {
    dynamic var id = NSUUID().UUIString
    dynamic var postDescription = ""
    dynamic var postPhotoURL = ""
    dynamic var numOfLikes = 0
    dynamic var createdAt = NSDate(timeIntervalSince1970: 1)
    override static func primaryKey() -> String? {
    return "id"
    }
    }
    

    View Slide

23. Realm Models -
    Realm Models -
    Relationships
    Relationships
    import RealmSwift
    class User: Object {
    dynamic var id = ""
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    let posts = List()
    override static func primaryKey() -> String? {
    return "id"
    }
    }
    One-To-Many Relationship
    One-To-Many Relationship
    

    View Slide

24. Realm Models
    Realm Models
    import RealmSwift
    class Post: Object {
    dynamic var id = ""
    dynamic var postDescription = ""
    dynamic var postPhotoURL = ""
    dynamic var numOfLikes = 0
    dynamic var createdAt = NSDate(timeIntervalSince1970: 1)
    var postOwners: [User] {
    return linkingObject(User.self, forProperty: "posts")
    }
    override static func primaryKey() -> String? {
    return "id"
    }
    }
    Inverse Relationship
    Inverse Relationship
    

    View Slide

25. Realm Models - Ignored Properties
    Realm Models - Ignored Properties
    import RealmSwift
    class User: Object {
    dynamic var id = ""
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    let posts = List()
    override static func primaryKey() -> String? {
    return "id"
    }
    override static func ignoredProperties() -> [String] {
    return ["nickname"]
    }
    }
    

    View Slide

26. Realm Models - Indexes
    Realm Models - Indexes
    import RealmSwift
    class User: Object {
    dynamic var id = ""
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    let posts = List()
    override static func primaryKey() -> String? {
    return "id"
    }
    override static func indexedProperties() -> [String] {
    return ["id", "firstName"]
    }
    }
    Greatly speeds up queries at
    Greatly speeds up queries at
    the cost of slower insertions.
    the cost of slower insertions.
    

    View Slide

27. Creating Realm Objects
    Creating Realm Objects
    Way 1: Object Composition
    Way 1: Object Composition
    import RealmSwift
    var user = User()
    user.id = NSUUID().UUIString
    user.firstname = "Thanos"
    user.lastname = "Theodoridis"
    user.email = "at@atworks.gr"
    

    View Slide

28. Creating Realm Objects
    Creating Realm Objects
    Way 2: Init from Dictionary
    Way 2: Init from Dictionary
    import RealmSwift
    var user = User(value: [
    "id": NSUUID().UUIString,
    "firstname": "Thanos",
    "lastname": "Theodoridis",
    "email": "at@atworks.gr"
    ])
    

    View Slide

29. Creating Realm Objects
    Creating Realm Objects
    Way 3: Init from Array
    Way 3: Init from Array
    import RealmSwift
    var user = User(value: [
    "Thanos",
    "Theodoridis",
    "at@atworks.gr"
    ])
    

    View Slide

30. Creating (Nested) Realm Objects
    Creating (Nested) Realm Objects
    var post1 = Post(value: [
    "postDescription": "Look, my dog! #dog #igers",
    "postURL": "http://lala.com/dog.jpg",
    ])
    var post2 = Post(value: [
    "postDescription": "Look, my cat! #cute",
    "postURL": "http://lala.com/cat.jpg",
    ])
    var user = User(value: [
    "id": NSUUID().UUIString,
    "firstname": "Thanos",
    "lastname": "Theodoridis",
    "email": "at@atworks.gr",
    "posts": [post1, post2]
    ])
    

    View Slide

31. Persisting Realm Objects
    Persisting Realm Objects
    let realm = try! Realm()
    var user = User(value: [
    "firstname": "Thanos",
    "lastname": "Theodoridis",
    "email": "at@atworks.gr",
    "posts": [post1, post2]
    ])
    try! realm.write {
    realm.add(user)
    }
    

    View Slide

32. Updating Realm Objects
    Updating Realm Objects
    let realm = try! Realm()
    user.lastname = "Papadopoulos"
    try! realm.write {
    realm.add(user, update: true)
    }
    This will work only if the User object has a primary key.
    If not, you need to enclose `user.lastname` in a
    `write()` block.
    If you omit `update: true` and an object with that key
    already exists, you get an exception (if unhandled, you
    crash)
    

    View Slide

33. Queries
    Queries
    All queries return a `Results` instance with a collection of
    Objects.
    All queries are lazy.
    Data is only read when the properties are accessed.
    Results are not copies.
    Modifying a Results object within a write transaction
    will persist to disk.
    Query execution is deferred until the results are used.
    Once a query has been executed, the Results object is up to
    date with any other changes in Realm.
    

    View Slide

34. Queries
    Queries
    The equivalent of `SELECT * FROM 'USERS'`
    let realm = try! Realm()
    let users = realm.objects(User)
    Complex Queries?
    Complex Queries? NSPredicate()
    

    View Slide

35. Queries for Relationships
    Queries for Relationships
    let realm = try! Realm()
    // Forward relationship
    let user = realm.objects(User).first
    let posts = user.posts
    // Inverse (backward) relationship
    let post = realm.objects(Post).first
    let postUser = post.postOwners.first
    

    View Slide

36. Queries using NSPredicate()
    Queries using NSPredicate()
    let realm = try! Realm()
    // Get a user with a matching id
    let predicate = NSPredicate(format: "id == %@", id)
    let user1 = realm.objects(User).filter(predicate)
    // Get users with GMail accounts
    predicate = NSPredicate(format: "email ENDSWITH %@", "gmail.com")
    let users = realm.objects(User).filter(predicate)
    // Get top active users
    predicate = NSPredicate(format: "posts.@count > 100")
    let activeUsers = realm.objects(User).filter(predicate)
    // Get users registered before Christmas
    predicate = NSPredicate(format: "date <= %@", christmasDate)
    

    View Slide

37. The Realm Database
    The Realm Database
    Realm() gets you the default Realm database.
    A file named `default.realm` in /Documents
    var config = Realm.Configuration()
    config.path = NSURL.fileURLWithPath(config.path!)
    .URLByDeletingLastPathComponent?
    .URLByAppendingPathComponent("NewName.realm")
    .path
    // set tihs as the config for the default realm
    Realm.Configuration.defaultConfiguration = config
    

    View Slide

38. The Realm Database
    The Realm Database
    But you can use as many different Realms as you want!
    let config = Realm.Configuration(
    path: myPathToBackupRealm.path
    readOnly: false
    )
    // Default Realm (default.realm)
    let realm = try! Realm()
    // Backup Realm (backup.realm)
    let otherRealm = try! Realm(configuration: config)
    

    View Slide

39. The Realm Database
    The Realm Database
    You can also have in-memory Realms
    Cool for caching layers
    let memoryCache = try! Realm(configuration:
    Realm.Configuration(inMemoryIdentifier: "com.me.cache")
    )
    Beware of in-memory instances going out of scope with no
    references.
    Garbage collector will wipe your data off
    Keep a strong reference to your in-memory Realms.
    

    View Slide

40. The Realm Database
    The Realm Database
    You can also bundle prepopulated Realms with your app.
    In the code where you generate the prepopulated Realm,
    make sure to "compact" it:
    Realm().writeCopyToPath(_:encryptionKey:)
    Drag and drop and add it in "Copy Bundle Resources" of your build
    phase.
    If you need to mutate this Realm, your first need to copy it to your
    application's Documents directory. Bundles are read-only.
    

    View Slide

41. Thread Safety
    Thread Safety
    One golden rule:
    let realm = try! Realm() for each thread
    Realm state for each thread will be based off the most
    recent confirmed transaction commit and will remain
    there until that version is refreshed.
    States are automatically refreshed at the start of every
    NSRunLoop iteration.
    Background threads that do not have run loops must
    explicitly call Realm.refresh()
    

    View Slide

42. Notifications
    Notifications
    Two options for monitoring updates on Realm objects:
    Collection Notifications
    KVO
    let userModelNotificationToken: NotificationToken!
    userModelNotificationToken = realm.objects(User).addNotificationBlock {
    notification, realm in
    usersTabelView.reloadData()
    }
    // When no longer needed
    userModelNotificationToken.stop()
    

    View Slide

43. Migrations In Realm
    Migrations In Realm
    Data models change...
    Data models change...
    Realm models are parsed and validated
    Realm models are parsed and validated
    during compile-time.
    during compile-time.
    But, if Realm sees a mismatch in models and
    But, if Realm sees a mismatch in models and
    the database during run-time,
    the database during run-time, it throws an
    it throws an
    exception and doesn't let you read or write to
    exception and doesn't let you read or write to
    the db.
    the db.
    ... Unless you run a migration!
    ... Unless you run a migration!
    

    View Slide

44. Migrations In Realm
    Migrations In Realm
    // v1 Model
    class User: Object {
    dynamic var id = ""
    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    }
    // v2 Model
    class User: Object {
    dynamic var id = ""
    dynamic var fullname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)
    }
    

    View Slide

45. Migrations In Realm
    Migrations In Realm
    // application(application:didFinishLaunchingWithOptions:)
    Realm.Configuration.defaultConfiguration = Realm.Configuration(
    // Increment the db schema version
    schemaVersion: 1,
    // Perform the migration
    migrationBlock: { migration, oldSchemaVersion in
    if (oldSchemaVersion < 1) {
    migration.enumerate(User.className()) { oldObject, newObject in
    let firstname = oldObject!["firstname"] as! String
    let lastname = oldObject!["lastname"] as! String
    newObject!["fullname"] = "\(fistname) \(lastname)"
    }
    }
    }
    )
    

    View Slide

46. Encrypted Realms
    Encrypted Realms
    Realm supports encrypting the db file on disk
    Realm supports encrypting the db file on disk
    with AES-256+SHA2 and 64byte keys
    with AES-256+SHA2 and 64byte keys
    let key = NSMutableData(length: 64)!
    // Generate random key
    SecRandomCopyBytes(
    kSecRandomDefault,
    key.length,
    UnsafeMutablePointer(key.mutableBytes)
    )
    // Save it securely in keychain
    // Always pass it in default Realm configuration
    // to be able to use encrypted Realm file
    Realm.Configuration.defaultConfiguration.encryptionKey = key
    

    View Slide

47. Realm Browser
    Realm Browser
    print(Realm.Configuration.defaultConfiguration.path!)
    // or
    (lldb) po Realm.defaultPath
    

    View Slide

48. Questions?
    Questions?
    @attheodo
    @attheodo http:/
    /attheo.do
    http:/
    /attheo.do
    

    View Slide