High Performance Core Data

http://highperformancecoredata.com/

Learn how to analyze, debug, and squeeze every last bit of performance out of Core Data. The standard Core Data implementation is very powerful and flexible but lacks performance. In this advanced session we will cover various performance analysis tools, model optimization, and various high performance concurrency models. This is an advanced discussion that assumes you have used Core Data before.

Transcript

1. High Performance Core Data HighPerformanceCoreData.com MatthewMorey.com | @xzolian
2. Agenda
3. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
4. Memory Speed Less More Slow Fast
5. Memory Speed Less More Slow Fast
6. Memory Speed Less More Slow Fast
9. Tools
10. #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed Time: %f", -[startTime timeIntervalSinceNow])
11. #define TICK NSDate *startTime = [NSDate date] #define TOCK NSLog(@"Elapsed Time: %f", -[startTime timeIntervalSinceNow]) ... TICK; [self methodYouWantToMeasure]; TOCK;
12. -com.apple.CoreData.SQLDebug [1,2,3]
13. -com.apple.CoreData.SQLDebug [1,2,3]
14. -com.apple.CoreData.SQLDebug [1,2,3]
15. -com.apple.CoreData.SQLDebug [1,2,3]
16. -com.apple.CoreData.SQLDebug [1,2,3] -com.apple.CoreData.SyntaxColoredLogging 1 -com.apple.CoreData.MigrationDebug 1 -com.apple.CoreData.SQLiteDebugSynchronous [0,1,2] -com.apple.CoreData.SQLiteIntegrityCheck [1] -com.apple.CoreData.ConcurrencyDebug [1,2,3]
17. $ sqlite3
18. $ sqlite3 UFO.sqlite sqlite> select * from sqlite_master; table|ZUFOSIGHTING|ZUFOSIGHTING|3|CREATE TABLE ZUFOSIGHTING ( Z_PK...VARCHAR ) table|Z_PRIMARYKEY|Z_PRIMARYKEY|4|CREATE TABLE Z_PRIMARYKEY (Z_ENT...INTEGER) table|Z_METADATA|Z_METADATA|5|CREATE TABLE Z_METADATA (Z_VERSION...BLOB)
19. sqlite> SELECT t0.ZSHAPE, COUNT( t0.ZSHAPE ) FROM ZUFOSIGHTING t0 GROUP BY t0.ZSHAPE; changed|1 changing|1546 chevron|760 cigar|1782 circle|5271 cone|265 ... teardrop|595 triangle|6082 unknown|4490
20. SQLite Professional Pony Debugger
23. Tools •Macros •Launch Arguments •SQL •Instruments •Measure, Measure, Measure
24. Fetching
26. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setFetchBatchSize:20];
29. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSArray *UFOSightingsArray = [self.managedObjectContext executeFetchRequest:fetchRequest error:&error]; NSMutableDictionary *uniqueShapesDictionary = [NSMutableDictionary dictionary]; for (UFOSighting *sighting in UFOSightingsArray) { ... // Count unique shape items ... } NSManagedObject
30. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setResultType:NSDictionaryResultType]; [fetchRequest setPropertiesToFetch:@"shape"]; ... // Count unique shape items ... NSDictionaryResultType
31. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@", @"sphere"]]; NSUInteger sphereCount = [self.managedObjectContext countForFetchRequest:fetchRequest error:&error]; ... // Repeat for each unique shape ... countForFetchRequest:error:
32. NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc] init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType]; NSExpression
33. NSExpression NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc] init]; [expressionDescription setName:@"count"]; [expressionDescription setExpression: [NSExpression expressionForFunction:@"count:" arguments: @[[NSExpression expressionForKeyPath:@"shape"]]]]; [fetchRequest setPropertiesToFetch:@[@"shape", expressionDescription]]; [fetchRequest setPropertiesToGroupBy:@[@"shape"]]; [fetchRequest setResultType:NSDictionaryResultType];
35. Prefetch Required Relationships
36. [fetchRequest setRelationshipKeyPathsForPrefetching: @[@"photo"]];
37. Fetch in the background
38. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType; NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
39. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType; NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
40. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType; NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
41. [backgroundContext performBlock:^{ NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"UFOSighting"]; fetchRequest.resultType = NSManagedObjectIDResultType; NSArray *managedObjectIDs = [backgroundContext executeFetchRequest:fetchRequest error:nil]; [mainQueueContext performBlock:^{ for (NSManagedObjectID *managedObjectID in managedObjectIDs) { UFOSighting *UFOSighting = [mainQueueContext objectWithID:managedObjectID]; // // Update UI on main queue // } }]; }];
42. NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:”MyEntity”]; NSAsynchronousFetchRequest *async = [[NSAsynchronousFetchRequest alloc] initWithFetchRequest:request completionBlock^(id result) { if (result.finalResult) { ... } }]; [context performBlock: ^() { NSError *error = nil; asyncResult = [moc executeRequest:asyncRequest error:&blockError]; }]; iOS 8 | OS X Yosemite
43. Fetching •Don’t fetch more than you need •Set a batch size •Let SQLite do the calculations •Prefetch required relationships •Fetch in the background
44. Predicates
45. Light Comparisons First
46. [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration > %i", @"sphere", 30]];
47. [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"shape == %@ AND duration > %i", @"sphere", 30]]; [fetchRequest setPredicate: [NSPredicate predicateWithFormat: @"duration > %i AND shape == %@", 30, @"sphere"]];
48. Be cautious with strings
49. Predicate Costs Less More Beginswith Endswith Equality == Contains Matches
50. Predicate Costs Less More Beginswith Endswith Equality == Contains Matches [cd] cost you even more
51. Predicates •Do light (numerical) comparisons first •Beginswith/Endswith instead of Contains/Matches •Don’t use [cd]
52. Searching
53. Canonicalize String Properties
54. Canonicalize String Properties Description canonicalizedDesc Green Mën green men BEAM beam Prόbë probe ABDUCTION abduction
55. CFStringNormalize((CFMutableStringRef)result, kCFStringNormalizationFormD); CFStringFold((CFMutableStringRef)result, kCFCompareCaseInsensitive | kCFCompareDiacriticInsensitive | kCFCompareWidthInsensitive, NULL);
56. Canonicalized Tokens
59. componentsSeperatedByCharactersInSet
60. Separate Stack
61. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens
62. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Primary Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Search Tokens URIRepresentation
63. Hash of Tokens
64. Searching •Canonicalize text properties •Use tokens •Separate persistence stack •Hash of tokens
65. Data Model
66. Don’t Overnormalize
67. Use External Storage
72. Data Model •Don’t overnormalize •Use external storage for large attributes •Large blobs as separate entity •Less data takes less time to fetch
73. App Launch
74. “Lots of Things” Problem
75. NSBatchUpdateRequest *batchRequest = [NSBatchUpdateRequest batchUpdateRequestWithEntityName: [RSSItem entityName]]; batchRequest.propertiesToUpdate = @{NSStringFromSelector(@selector(read)): [NSNumber numberWithBool:YES]}; batchRequest.resultType = NSStatusOnlyResultType; batchRequest.predicate = [NSPredicate predicateWithFormat:@"..."]; NSError *requestError; NSBatchUpdateResult *result = (NSBatchUpdateResult *)[self.managedObjectContext executeRequest:batchRequest error:&requestError]; if (result == nil) { NSLog(@"Error: %@", [requestError localizedDescription]); } else { // Batch update succeeded } iOS 8 | OS X Yosemite
76. Importing Data
78. Main Queue vs Private Queue self.managedObjectContext = self.persistenceStack.managedObjectContext; [self.managedObjectContext performBlockAndWait:^{ [self import]; }];
79. Main Queue vs Private Queue NSManagedObjectContext *privateManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType]; [privateManagedObjectContext setPersistentStoreCoordinator: self.managedObjectContext.persistentStoreCoordinator]; [privateManagedObjectContext performBlockAndWait:^{ [self import]; }];
81. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
82. Typical Import Algorithm 1) JSON to NSDictionary 2) Find existing NSManagedObject 3) Optionally, create new NSManagedObject 4) Set attributes of new/updated NSManagedObject 5) Repeat for every JSON item
83. Efficient Import Algorithm 1) Sort import objects by ID 2) Execute a single fetch request for all matching IDs 3) Iterate through both import and existing objects 4) Update or create
84. 1 4 2 3 ... New Data
85. 1 4 2 3 ... New Data 4 1 ... Existing Data
86. 1 4 2 3 ... New Data 4 1 ... Existing Data
87. 1 4 2 3 ... New Data 4 1 ... Existing Data
88. 1 4 2 3 ... New Data 4 1 ... Existing Data
89. 1 2 3 4 ... New Data 1 4 ... Existing Data Update
90. 1 2 3 4 ... New Data 1 4 ... Existing Data Update
91. 1 2 3 4 ... New Data 1 4 ... Existing Data Insert
92. 1 2 3 4 ... New Data 1 4 ... Existing Data Insert
93. 2 1 2 3 4 ... New Data 1 4 ... Existing Data
94. 2 1 2 3 4 ... New Data 1 4 ... Existing Data
95. 1 2 3 4 ... New Data 1 4 ... Existing Data 2
96. 1 2 3 4 ... New Data 1 4 ... Existing Data 2 Insert
97. 1 2 3 4 ... New Data 1 4 ... Existing Data 2 Insert
98. 3 1 2 3 4 ... New Data 1 4 ... Existing Data 2
99. 3 1 2 3 4 ... New Data 1 4 ... Existing Data 2
100. 1 2 3 4 ... New Data 1 4 ... Existing Data 2 3 Update
101. 1 2 3 4 ... New Data 1 4 ... Existing Data 2 3 Update
103. Efficient Import Algorithm With Batching 1) Sort import objects by ID 2) Execute multiple fetch request for matching IDs - Iterate through both import and existing objects - Update or create 3) Save Batch
104. // Grab sorted persisted managed objects, based on the batch NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
105. // Grab sorted persisted managed objects, based on the batch NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] initWithEntityName:@"UFOSighting"]; NSPredicate *fetchPredicate = [NSPredicate predicateWithFormat: @"%K IN %@", @"GUID", jsonBatchGUIDArray];
106. Pre-Populated SQLite File in App Bundle
107. Download Pre-Populated SQLite File
108. Importing Data •Do import work on private queues •Use efficient find-or-create algorithm •Work in batches to keep memory low •Use pre-populated SQLite file
109. Memory
110. [context refreshObject:UFOSightingManagedObject mergeChanges:NO];
111. [context reset];
112. Concurrency Models
113. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
114. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
115. Managed Object Context Managed Object Managed Object Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Main Queue
116. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue
117. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue
118. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue
119. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
120. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Merge Changes From Notification
121. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
122. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue
123. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work
124. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
125. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
126. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
127. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
128. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Managed Object Context Private Queue UI Work Persisting to Disk
129. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue
130. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates
131. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work
132. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
133. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
134. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk
135. Persistent Store Coordinator Persistent Store Model Main Queue Managed Object Context Private Queue Managed Object Context Managed Object Context Private Queue Background Updates UI Work Persisting to Disk __block NSManagedObjectContext *temporaryContext = self.workerObjectContext; __block NSManagedObjectContext *managedObjectContext = self.managedObjectContext; __block NSManagedObjectContext *writerObjectContext = self.writerManagedObjectContext; [temporaryContext performBlock:^{ NSError *error = nil; if (![temporaryContext save:&error]) { // TODO: Handle error } [managedObjectContext performBlock:^{ NSError *error = nil; if (![managedObjectContext save:&error]) { // TODO: Handle error } [writerObjectContext performBlock:^{ NSError *error = nil; if (![writerObjectContext save:&error]) { // TODO: Handle error } // Success!!! }]; // writerObjectContext }]; // managedObjectContext }]; // temporaryContext
137. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
138. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
139. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
140. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model
141. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification
142. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
143. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Merge Changes From Notification
144. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload
146. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Persistent Store Coordinator Model Context Did Save Notification Refetch and Reload •Cannot pass objects between persistent store coordinators •Cannot pass objectIDs between persistent store coordinators •You can use URIRepresentation •You can use mergeChangesFromContextDidSaveNotification
147. TLDR
148. Managed Object Context Persistent Store Coordinator Persistent Store (SQLite, XML, ...) Model Managed Object Context Main Queue Private Queue Context Did Save Notification Refetch and Reload
149. Concurrency Models •Use the simplest model that meets your needs •Update main context after large imports have completed
150. Bang Head Here
151. In general... •Use the tools, and measure, measure, measure •Don’t load more than you need to •Don’t use [cd] •Be smart with your data model •Import on a private queue in batches •Pick the correct (but simplest) concurrency model
152. Questions? HighPerformanceCoreData.com MatthewMorey.com | @xzolian BuoyExplorer.com - Marine Conditions App WristPresenter.com - Presentation App ChaiOne.com - Hiring remote iOS engineers