CoreData strikes back

Cocoaheads Dnepr #2

1. CoreData наносит ответный удар с вертушки
2. CoreData • The Core Data framework provides generalized and automated solutions to common tasks associated with object life-cycle and object graph management, including persistence. • Не база данных
3. CoreData performance
4. Откуда ждать проблем? • Игнорироание рекомендаций Apple • Многопоточность • Неоптимизированные запросы • CoreData cache misses • Неожиданные выборки • Неожиданное поведение
5. Импорт данных
6. Есть JSON @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] };
7. @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; Есть JSON
8. @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; Есть JSON @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; … @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] };
9. @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; Есть JSON @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; … @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; @{ /*Person*/ @"id": @123456789, @"name": @"Johny Ive", @"email": @"real@email.com", @"phones": @[ @{ @"id": @12345, @"number": @"+3801234567", @"timestamp": @"12-11-2014" }, ] }; Всего 1500 уникальных объектов • 250 Person • 1250 Phones
10. Есть MagicalRecord • https://github.com/magicalpanda/MagicalRecord/ • 4523 start • 666 commits • 52 contributors • 782 forks
11. Есть MagicalRecord • https://github.com/magicalpanda/MagicalRecord/ • 4523 start • 666 commits • 52 contributors • 782 forks
12. Также имеется в наличии • Xcode 5.1.1 • iPhone 5s • Схема Release • 20 итераций для каждого теста • Time Profiler
13. Модель Person ————————— int64 personID (PK, no index); NSString *name; NSString *email; ————————— Phones Phone ————————— int64 phoneID (PK, no index) NSString *ddi; NSString *ddd; NSString *number; NSString *timestamp; 1 *
14. Время импорта Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 4.375 4.861
15. – Time Profile “Ну и тормозит”
16. Код импорта (псевдо) + (id)importObject:(NSDictionary *)JSON { id object = [self goToDatabaseAndFindObjectForPrimaryKey:JSON[@"primaryKey"]]; if (object == nil) { object = [self createObject]; } [object importAttributes:JSON]; [object importRelationships:JSON]; return object; } ! + (void)importObjects:(NSArray *)objects { for (id object in objects) { [self importObject:object]; } }
17. Time profiler #1
18. Time profiler #2
19. Improvement #1 • Проиндексируем Primary keys
20. Improvement #1 • Проиндексируем Primary keys
21. Время импорта PK Indexed Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 4.375 4.294
22. Код импорта даты NSDate * dateFromString(NSString *value, NSString *format) { NSDateFormatter *formatter = [[NSDateFormatter alloc] init]; [formatter setTimeZone:[NSTimeZone localTimeZone]]; [formatter setLocale:[NSLocale currentLocale]]; [formatter setDateFormat:format]; NSDate *parsedDate = [formatter dateFromString:value]; return parsedDate; }
23. подправим..
24. подправим.. следующий код: Потоконебезопасный Не освобождает память Это только для примера ! пожалуйста, не используйте его
25. подправим.. NSDate * dateFromString(NSString *value, NSString *format) { static NSMutableDictionary *map = nil; static dispatch_once_t token; dispatch_once(&token, ^{ map = [NSMutableDictionary new]; }); NSDateFormatter *formatter = [map objectForKey:format]; if (!formatter) { formatter = [[NSDateFormatter alloc] init]; [formatter setTimeZone:[NSTimeZone localTimeZone]]; [formatter setLocale:[NSLocale currentLocale]]; [formatter setDateFormat:format]; [map setObject:formatter forKey:format]; } NSDate *parsedDate = [formatter dateFromString:value]; return parsedDate; }
26. Время импорта PK Index, NSDateFormatter cache Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 3.356 3.254
27. Взгляд со стороны CoreData Instruments
28. CoreData Instruments #1 Fetches CPU Usage
29. CoreData Instruments #1 Fetches CPU Usage
30. Есть MagicalRecord fork • https://github.com/seepel/MagicalRecord Fetches CPU Usage fork
31. Есть MagicalRecord fork • https://github.com/seepel/MagicalRecord Fetches CPU Usage fork Fetches CPU Usage original
32. Время импорта PK Index, NSDateFormatter cache, batch fetch Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 1.337 1.236
33. Когда уже будет что-то интересное..
34. Insert Person *object = // find or create [object setPersonID:JSON[@"personIdentifier"]]; // . . . ! [object.managedObjectContext save:&error];
35. Insert // Find sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZEMAIL, t0.ZGENDER, t0.ZNAME, t0.ZPERSONID, t0.ZTIMESTAMP FROM ZPERSON t0 WHERE t0.ZPERSONID IN (?) ! // Save sql: INSERT INTO ZPERSON(Z_PK, Z_ENT, Z_OPT, ZEMAIL, ZGENDER, ZNAME, ZPERSONID, ZTIMESTAMP) VALUES(?, ?, ?, ?, ?, ?, ?, ?) Person *object = // find or create [object setPersonID:JSON[@"personIdentifier"]]; // . . . ! [object.managedObjectContext save:&error];
36. Update // Find sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZEMAIL, t0.ZGENDER, t0.ZNAME, t0.ZPERSONID, t0.ZTIMESTAMP FROM ZPERSON t0 WHERE t0.ZPERSONID IN (?) ! // Save sql: UPDATE ZPERSON SET Z_OPT = ? WHERE Z_PK = ? AND Z_OPT = ? Person *object = // find [object setPersonID:equalPersonID]; // . . . ! [object.managedObjectContext save:&error];
37. Проверка на равенство - (void)MR_setValueIfDifferent:(id)value forKey:(NSString *)key { id currentValue = [self valueForKey:key]; if(currentValue == nil && value == nil) { return; } if((currentValue == nil && value != nil) || (currentValue != nil && value == nil)) { [self setValue:value forKey:key]; return; } if(![currentValue isEqual:value]) { [self setValue:value forKey:key]; } }
38. Update // Find sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZEMAIL, t0.ZGENDER, t0.ZNAME, t0.ZPERSONID, t0.ZTIMESTAMP FROM ZPERSON t0 WHERE t0.ZPERSONID IN (?) ! // Save Person *object = // fetch from database [object setPersonID:equalPersonID]; // . . . ! [object.managedObjectContext save:&error]; incl. equality check
39. Время импорта PK Index, NSDateFormatter cache, batch fetch, equality check Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 1.339 0.756
40. Что у нас есть Fetches CPU Usage MagicalRecord fork
41. Что у нас есть Fetches CPU Usage MagicalRecord fork
42. Почему бы не сделать так? Fetches CPU Usage
43. Как работает MR fork: @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
44. Как работает MR fork: @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
45. Как работает MR fork: @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
46. Как работает MR fork: @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
47. Как работает MR fork: @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
48. Как можно ускорить @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
49. Как можно ускорить @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
50. Как можно ускорить @[ @{ @"id": @123456, /* ... */ @"phones": @[ @{@"id": @123, /* ... */ }, @{@"id": @1234, /* ... */ }, @{@"id": @12345, /* ... */ }, ], }, @{ @"id": @1300156, /* ... */ @"phones": @[ @{@"id": @5664523, /* ... */ }, @{@"id": @777412, /* ... */ }, @{@"id": @33345, /* ... */ }, ], }, /* ... */ ];
51. Время импорта PK Index, NSDateFormatter cache, improved batch fetch, equality check Insert Update t, sec 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 0.173 0.081
52. FastMappingKit • https://github.com/Yalantis/FastEasyMapping • pod "FastEasyMapping", :git => "https://github.com/Yalantis/FastEasyMapping.git"
53. FastMappingKit • https://github.com/Yalantis/FastEasyMapping • pod "FastEasyMapping", :git => "https://github.com/Yalantis/FastEasyMapping.git" + (EMKManagedObjectMapping *)personWithPhonesMapping { return [EMKManagedObjectMapping mappingForEntityName:@"Person" configuration:^(EMKManagedObjectMapping *mapping) { [mapping setPrimaryKey:@"personID"]; [mapping addAttributeMappingDictionary:@{@"personID": @"id"}]; [mapping addAttributeMappingFromArray:@[@"name", @"email"]]; [mapping addAttributeMapping:[EMKAttributeMapping mappingOfProperty:@"timestamp" keyPath:@"timestamp" dateFormat:@"dd-MM-yyyy"]]; ! [mapping addToManyRelationshipMapping:[self phoneMapping] forProperty:@"phones" keyPath:@"phones"]; }]; }
54. Неожиданные запросы
55. CoreData faults NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:@"Phone"]; NSArray *phones = [context executeFetchRequest:request error:NULL]; ! for (Phone *phone in phones) { phone.person; } Faults Fetches
56. CoreData faults NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:@"Phone"]; NSArray *phones = [context executeFetchRequest:request error:NULL]; [request setRelationshipKeyPathsForPrefetching:@[@"person"]]; ! for (Phone *phone in phones) { phone.person; } Faults Fetches
57. Время No prefetch Prefetch t, ms 0.000 1.000 2.000 3.000 4.000 5.000 6.000 5.831 0.352
58. sql NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:@"Phone"]; ! sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZDDD, t0.ZDDI, t0.ZNUMBER, t0.ZPHONEID, t0.ZTIMESTAMP, t0.ZPERSON FROM ZPHONE t0 NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:@"Phone"]; [request setRelationshipKeyPathsForPrefetching:@[@"person"]]; ! sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZDDD, t0.ZDDI, t0.ZNUMBER, t0.ZPHONEID, t0.ZTIMESTAMP, t0.ZPERSON FROM ZPHONE t0 . . . sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZEMAIL, t0.ZGENDER, t0.ZNAME, t0.ZPERSONID, t0.ZTIMESTAMP FROM ZPERSON t0 WHERE t0.Z_PK IN (SELECT * FROM _Z_intarray0)
59. CoreData stack и производительность
60. CoreData stacks Persistent Store Coordinator Root Context ! (Private queue concurrency type) Main Context! (Main queue concurrency type) Background Context ! (Private queue concurrency type) stack #1
61. CoreData stacks Persistent Store Coordinator Root Context ! (Private queue concurrency type) Main Context! (Main queue concurrency type) Background Context ! (Private queue concurrency type) stack #1 Persistent Store Coordinator Main Context ! (Main queue concurrency type) Background Context ! (Private queue concurrency type) stack #2
62. Time Profiler Main thread Background thread stack #1 Main thread Background thread stack #2
63. Time Profiler
64. Тут надо что-то умное сказать
65. Ссылки • Подопытные: • MagicalRecord • MagicalRecord fork • FastEasyMapping ! • Почитать: • CoreData performance • CD Stack performance shootout • Backstage with nested managed object contexts
66. Спасибо