How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

When people talk about Doctrine (or any ORM for that matter), the performance issue always comes up fairly quickly. Besides the fact that Doctrine will help you develop faster, so a little overhead doesn't really matter, there are numerous options to increase the performance of the application.

By understanding how the system works in the first place, a lot of issues can be avoided right away.

When you have done everything to avoid these pitfalls, you can bring in the big guns: caching. Doctrine has different caching mechanism and since Doctrine 2.5 "Second Level Cache" was added to our toolbox. After this talk, you should know what the impact is of every cache and how to use it.

Transcript

1. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj
2. Jachim Coudenys Jachim Coudenys
5. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION
6. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION If you're using Doctrine ORM. If you're using Doctrine ORM.
7. WHAT IS DOCTRINE
8. OBJECT-RELATIONAL MAPPER ... is a programming technique for converting data between incompatible type systems using object-oriented programming languages. — https://en.wikipedia.org/wiki/Object-relational_mapping
9. TERMINOLOGY Entity Mapping Repository EntityManager
10. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
11. SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS
13. ENTITYMANAGER
15. UNIT OF WORK Maintains a list of objects affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems. — https://martinfowler.com/eaaCatalog/unitOfWork.html
17. TRANSACTIONAL WRITE- BEHIND
19. IDENTITY MAP Ensures that each object gets loaded only once by keeping every loaded object in a map. Looks up objects using the map when referring to them. — https://martinfowler.com/eaaCatalog/identityMap.html
21. ENTITY STATES NEW MANAGED DETACHED REMOVED
23. HYDRATION
25. PROXIES (LAZY LOADING) An object that doesn't contain all of the data you need but knows how to get it. — https://martinfowler.com/eaaCatalog/lazyLoad.html
27. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
28. TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES
29. CREATING Create new object (NEW) Persist the object (MANAGED) Flush the EM UoW calculates changes => INSERT SQL Transaction
30. GETTING Request object Identity Map lookup Database query Object hydration Save hydrated object to IM (MANAGED)
31. UPDATING Get the object (MANAGED) Change a property Persist the object (no-op) Flush the EM UoW calculates changes => UPDATE SQL Transaction
32. DELETING Get the object (MANAGED) Ask for removal (REMOVED) Flush the EM UoW calculates changes => DELETE SQL Transaction
33. CHANGE TRACKING POLICIES
34. DEFERRED IMPLICIT Default Property-by-property comparison Persistence by reachability Slowest
35. DEFERRED EXPLICIT Property-by-property comparison EntityManager#persist(entity) No "dirty" checking
36. DEFERRED EXPLICIT http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit /** * @Entity * @ChangeTrackingPolicy("DEFERRED_EXPLICIT") */ class User { // ... }
37. NOTIFY Notify interested listeners of changes NotifyPropertyChanged
38. NOTIFY http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify use Doctrine\Common\NotifyPropertyChanged, Doctrine\Common\PropertyChangedListener; /** * @Entity * @ChangeTrackingPolicy("NOTIFY") */ class MyEntity implements NotifyPropertyChanged { private $listeners = array(); public function addPropertyChangedListener(PropertyChangedListener $listener) { $this->listeners[] = $listener; } }
39. NOTIFY http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify class MyEntity implements NotifyPropertyChanged { protected function onPropertyChanged($propName, $oldValue, $newValue) { if ($this->listeners) { foreach ($this->listeners as $listener) { $listener->propertyChanged($this, $propName, $oldValue, $newValue); } } } public function setData($data) { if ($data != $this->data) { $this->onPropertyChanged('data', $this->data, $data); $this->data = $data; } } }
40. FURTHER IMPROVEMENTS
41. ALTERNATIVE QUERY RESULT FORMATS Read only data Scalar values Nested array graph
42. ALTERNATIVE QUERY RESULT FORMATS $dql = "SELECT b, e, r, p FROM Bug b JOIN b.engineer e ". "JOIN b.reporter r JOIN b.products p ORDER BY b.created DESC"; $query = $entityManager->createQuery($dql); $bugs = $query->getArrayResult(); foreach ($bugs as $bug) { echo $bug['description'] . " - " . $bug['created']->format('d.m.Y')."\n"; echo " Reported by: ".$bug['reporter']['name']."\n"; echo " Assigned to: ".$bug['engineer']['name']."\n"; foreach ($bug['products'] as $product) { echo " Platform: ".$product['name']."\n"; } echo "\n"; }
43. READ ONLY ENTITIES /** * @Entity(readOnly=true) * @Table(name="products") */ class Product { }
44. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
45. EXTRA-LAZY COLLECTIONS LAZY (default) EAGER (join) EXTRA_LAZY
46. EXTRA-LAZY COLLECTIONS http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations namespace Doctrine\Tests\Models\CMS; use Doctrine\ORM\Annotation as ORM; /** * @ORM\Entity */ class CmsGroup { /** * @ORM\ManyToMany( * targetEntity="CmsUser", mappedBy="groups", fetch="EXTRA_LAZY" * ) */ public $users; }
47. DOCTRINE\CACHE
48. DOCTRINE\COMMON\ CACHE\CACHE INTERFACE fetch($id) contains($id) save($id, $data, $lifeTime = false) delete($id)
49. CACHE DRIVERS APCu Memcached Xcache Redis File Array* ...
50. CACHING IN DOCTRINE
51. METADATA Removes parsing overhead (Annotations, YML, XML)
52. QUERY DQL => SQL
53. RESULT Store SQL data result to cache Still a need to hydrate objects Joins will be stored as-is
54. CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING
55. HYDRATED RESULT
56. REPOSITORY DECORATOR
57. GENERALLY A BAD IDEA Serializing entities? Serializing collections? Detached entities https://github.com/doctrine/doctrine2/pull/172/files
59. https://cowburn.info/public/files/php7_logo.svg https://cowburn.info/public/files/php7_logo.svg
60. SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL CACHING CACHING CACHING CACHING CACHING CACHING CACHING CACHING
62. SECOND LEVEL CACHING No entity instances Only entity identifier and values Best suited for read-only data
63. TYPES Entity data: id + values Collection data: ownerId id + list of ids Query data: list of ids
64. ENTITY CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition /** * @Entity * @Cache() */ class Country { // ... }
65. ASSOCIATION CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition /** * @Entity * @Cache() */ class State { //... /** * @Cache() * @ManyToOne(targetEntity="Country") * @JoinColumn(name="country_id", referencedColumnName="id") */ protected $country; }
66. REGIONS
67. MODES READ_ONLY NONSTRICT_READ_WRITE READ_WRITE
68. QUERY CACHE
69. QUERY CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache $result1 = $em->createQuery('SELECT c FROM Country c ORDER BY c.name') ->setCacheable(true) ->getResult();
70. QUERY CACHE: MODES Cache::MODE_GET Cache::MODE_PUT Cache::MODE_NORMAL (default) Cache::MODE_REFRESH
71. QUERY CACHE: DELETE / UPDATE QUERIES
72. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute and invalidate $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->setHint(Query::HINT_CACHE_EVICT, true) ->execute();
73. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->execute(); // Invoke Cache API $em->getCache()->evictEntityRegion('Entity\Country');
74. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->execute(); // Invoke Cache API $em->getCache()->evictEntity('Entity\Country', 1);
75. LIMITATIONS Single application Single Primary key column
76. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
77. CONCLUSION Keep Identity Map / internals in mind Using the basic caching (query + mapping) + opcode caching is a must Cache heavy queries with result cache Give Second Level Caching a try (even if only for entities) Query Cache !== Second Level Query Cache
78. http://doctrine-project.org/ https://www.flickr.com/photos/spacex/40126461851 https://github.com/coudenysj?tab=repositories&q=doctrine @coudenysj jachim.coudenys@combell.group THANK YOU joind.in/talk/2ce9e joind.in/talk/2ce9e