High Performance Core Data

High Performance Core Data

http://highperformancecoredata.com/

Learn how to analyze, debug, and squeeze every last bit of performance out of Core Data. The standard Core Data implementation is very powerful and flexible but lacks performance. In this advanced session we will cover various performance analysis tools, model optimization, and various high performance concurrency models. This is an advanced discussion that assumes you have used Core Data before.

2345435ebf5920b1d600b7ee7d2221a1?s=128

Matthew Morey

November 15, 2013
Tweet

Transcript

  1. 2.
  2. 3.

    Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  3. 7.
  4. 8.
  5. 9.
  6. 10.
  7. 11.

    #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed

    Time: %f", -[startTime timeIntervalSinceNow]) ... TICK; [self methodYouWantToMeasure]; TOCK;
  8. 17.
  9. 18.

    $ sqlite3 UFO.sqlite sqlite> select * from sqlite_master; table|ZUFOSIGHTING|ZUFOSIGHTING|3|CREATE TABLE

    ZUFOSIGHTING ( Z_PK...VARCHAR ) table|Z_PRIMARYKEY|Z_PRIMARYKEY|4|CREATE TABLE Z_PRIMARYKEY (Z_ENT...INTEGER) table|Z_METADATA|Z_METADATA|5|CREATE TABLE Z_METADATA (Z_VERSION...BLOB)
  10. 19.

    sqlite> SELECT t0.ZSHAPE, COUNT( t0.ZSHAPE ) FROM ZUFOSIGHTING t0 GROUP

    BY t0.ZSHAPE; changed|1 changing|1546 chevron|760 cigar|1782 circle|5271 cone|265 ... teardrop|595 triangle|6082 unknown|4490
  11. 21.
  12. 22.
  13. 24.
  14. 25.
  15. 27.
  16. 28.
  17. 29.

    NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSArray *UFOSightingsArray = [self.managedObjectContext executeFetchRequest:fetchRequest

    error:&error]; NSMutableDictionary *uniqueShapesDictionary = [NSMutableDictionary dictionary]; for (UFOSighting *sighting in UFOSightingsArray) { ... // Count unique shape items ... } NSManagedObject
  18. 31.

    NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape

    == %@", @"sphere"]]; NSUInteger sphereCount = [self.managedObjectContext countForFetchRequest:fetchRequest error:&error]; ... // Repeat for each unique shape ... countForFetchRequest:error:
  19. 32.

    NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc]

    init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType]; NSExpression
  20. 33.

    NSExpression NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription

    alloc] init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType];
  21. 34.
  22. 38.

    [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  23. 39.

    [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  24. 40.

    [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  25. 41.

    [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType;

    NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
  26. 42.

    NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:”MyEntity”]; NSAsynchronousFetchRequest *async = [[NSAsynchronousFetchRequest alloc]

    initWithFetchRequest:request completionBlock^(id result) { if (result.finalResult) { ... } }]; [context performBlock: ^() { NSError *error = nil; asyncResult = [moc executeRequest:asyncRequest error:&blockError]; }]; iOS 8 | OS X Yosemite
  27. 43.

    Fetching •Don’t fetch more than you need •Set a batch

    size •Let SQLite do the calculations •Prefetch required relationships •Fetch in the background
  28. 47.

    [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration >

    %i", @"sphere", 30]]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"duration > %i AND shape == %@", 30, @"sphere"]];
  29. 52.
  30. 57.
  31. 58.
  32. 61.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens
  33. 62.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens URIRepresentation
  34. 68.
  35. 69.
  36. 70.
  37. 71.
  38. 72.

    Data Model •Don’t overnormalize •Use external storage for large attributes

    •Large blobs as separate entity •Less data takes less time to fetch
  39. 75.

    NSBatchUpdateRequest *batchRequest = [NSBatchUpdateRequest batchUpdateRequestWithEntityName: [RSSItem entityName]]; batchRequest.propertiesToUpdate = @{NSStringFromSelector(@selector(read)):

    [NSNumber numberWithBool:YES]}; batchRequest.resultType = NSStatusOnlyResultType; batchRequest.predicate = [NSPredicate predicateWithFormat:@"..."]; NSError *requestError; NSBatchUpdateResult *result = (NSBatchUpdateResult *)[self.managedObjectContext executeRequest:batchRequest error:&requestError]; if (result == nil) { NSLog(@"Error: %@", [requestError localizedDescription]); } else { // Batch update succeeded } iOS 8 | OS X Yosemite
  40. 77.
  41. 79.

    Main Queue vs Private Queue NSManagedObjectContext *privateManagedObjectContext = [[NSManagedObjectContext alloc]

    initWithConcurrencyType:NSPrivateQueueConcurrencyType]; [privateManagedObjectContext setPersistentStoreCoordinator: self.managedObjectContext.persistentStoreCoordinator]; [privateManagedObjectContext performBlockAndWait:^{ [self import]; }];
  42. 80.
  43. 81.

    Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing

    NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
  44. 82.

    Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing

    NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
  45. 83.

    Efficient Import Algorithm 1) Sort import objects by ID 2)

    Execute a single fetch request for all matching IDs 3) Iterate through both import and existing objects 4) Update or create
  46. 85.
  47. 86.
  48. 87.
  49. 88.
  50. 89.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data Update
  51. 90.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data Update
  52. 91.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data Insert
  53. 92.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data Insert
  54. 93.

    2 1 2 3 4 ... New Data 1 4

    ... Existing Data
  55. 94.

    2 1 2 3 4 ... New Data 1 4

    ... Existing Data
  56. 95.
  57. 96.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 Insert
  58. 97.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 Insert
  59. 98.

    3 1 2 3 4 ... New Data 1 4

    ... Existing Data 2
  60. 99.

    3 1 2 3 4 ... New Data 1 4

    ... Existing Data 2
  61. 100.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 3 Update
  62. 101.

    1 2 3 4 ... New Data 1 4 ...

    Existing Data 2 3 Update
  63. 102.
  64. 103.

    Efficient Import Algorithm With Batching 1) Sort import objects by

    ID 2) Execute multiple fetch request for matching IDs - Iterate through both import and existing objects - Update or create 3) Save Batch
  65. 104.

    // Grab sorted persisted managed objects, based on the batch

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
  66. 105.

    // Grab sorted persisted managed objects, based on the batch

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
  67. 108.

    Importing Data •Do import work on private queues •Use efficient

    find-or-create algorithm •Work in batches to keep memory low •Use pre-populated SQLite file
  68. 109.
  69. 113.

    Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  70. 114.

    Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  71. 115.

    Managed Object Context Managed Object Managed Object Persistent Store Coordinator

    Persistent Store (SQLite, XML, ...) Model Main Queue
  72. 116.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  73. 117.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  74. 118.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue
  75. 119.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
  76. 120.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
  77. 121.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  78. 122.
  79. 123.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work
  80. 124.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  81. 125.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  82. 126.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  83. 127.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  84. 128.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Managed Object Context Private Queue UI Work Persisting to Disk
  85. 129.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue
  86. 130.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates
  87. 131.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work
  88. 132.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  89. 133.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  90. 134.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
  91. 135.

    Persistent Store Coordinator Persistent Store Model Main Queue Managed Object

    Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk __block NSManagedObjectContext *temporaryContext = self.workerObjectContext; __block NSManagedObjectContext *managedObjectContext = self.managedObjectContext; __block NSManagedObjectContext *writerObjectContext = self.writerManagedObjectContext; [temporaryContext performBlock:^{ NSError *error = nil; if (![temporaryContext save:&error]) { // TODO: Handle error } [managedObjectContext performBlock:^{ NSError *error = nil; if (![managedObjectContext save:&error]) { // TODO: Handle error } [writerObjectContext performBlock:^{ NSError *error = nil; if (![writerObjectContext save:&error]) { // TODO: Handle error } // Success!!! }]; // writerObjectContext }]; // managedObjectContext }]; // temporaryContext
  92. 136.
  93. 137.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  94. 138.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  95. 139.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  96. 140.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
  97. 141.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification
  98. 142.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
  99. 143.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
  100. 144.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload
  101. 145.
  102. 146.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload •Cannot pass objects between persistent store coordinators •Cannot pass objectIDs between persistent store coordinators •You can use URIRepresentation •You can use mergeChangesFromContextDidSaveNotification
  103. 147.
  104. 148.

    Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML,

    ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
  105. 149.

    Concurrency Models •Use the simplest model that meets your needs

    •Update main context after large imports have completed
  106. 151.

    In general... •Use the tools, and measure, measure, measure •Don’t

    load more than you need to •Don’t use [cd] •Be smart with your data model •Import on a private queue in batches •Pick the correct (but simplest) concurrency model
  107. 152.

    Questions? HighPerformanceCoreData.com MatthewMorey.com | @xzolian BuoyExplorer.com - Marine Conditions App

    WristPresenter.com - Presentation App ChaiOne.com - Hiring remote iOS engineers