How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

When people talk about Doctrine (or any ORM for that matter), the performance issue always comes up fairly quickly. Besides the fact that Doctrine will help you develop faster, so a little overhead doesn't really matter, there are numerous options to increase the performance of the application.

By understanding how the system works in the first place, a lot of issues can be avoided right away.

When you have done everything to avoid these pitfalls, you can bring in the big guns: caching. Doctrine has different caching mechanism and since Doctrine 2.5 "Second Level Cache" was added to our toolbox. After this talk, you should know what the impact is of every cache and how to use it.

Transcript

1. HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   Jachim Coudenys
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   jachim.coudenys@combellgroup.com
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   @coudenysj
   

   View Slide

2. Jachim Coudenys
   Jachim Coudenys
   

   View Slide

3. View Slide

4. View Slide

5. HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   

   View Slide

6. HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   HOW DOCTRINE
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   CACHING CAN
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   SKYROCKET YOUR
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   APPLICATION
   If you're using Doctrine ORM.
   If you're using Doctrine ORM.
   

   View Slide

7. WHAT IS DOCTRINE
   

   View Slide

8. OBJECT-RELATIONAL MAPPER
   ... is a programming technique for
   converting data between incompatible
   type systems using object-oriented
   programming languages.
   — https://en.wikipedia.org/wiki/Object-relational_mapping
   

   View Slide

9. TERMINOLOGY
   Entity
   Mapping
   Repository
   EntityManager
   

   View Slide

10. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL
    

    View Slide

11. SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    

    View Slide

12. View Slide

13. ENTITYMANAGER
    

    View Slide

14. View Slide

15. UNIT OF WORK
    Maintains a list of objects affected by a
    business transaction and coordinates
    the writing out of changes and the
    resolution of concurrency problems.
    — https://martinfowler.com/eaaCatalog/unitOfWork.html
    

    View Slide

16. View Slide

17. TRANSACTIONAL WRITE-
    BEHIND
    

    View Slide

18. View Slide

19. IDENTITY MAP
    Ensures that each object gets loaded
    only once by keeping every loaded
    object in a map. Looks up objects using
    the map when referring to them.
    — https://martinfowler.com/eaaCatalog/identityMap.html
    

    View Slide

20. View Slide

21. ENTITY STATES
    NEW
    MANAGED
    DETACHED
    REMOVED
    

    View Slide

22. View Slide

23. HYDRATION
    

    View Slide

24. View Slide

25. PROXIES (LAZY LOADING)
    An object that doesn't contain all of
    the data you need but knows how to
    get it.
    — https://martinfowler.com/eaaCatalog/lazyLoad.html
    

    View Slide

26. View Slide

27. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL
    

    View Slide

28. TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    

    View Slide

29. CREATING
    Create new object (NEW)
    Persist the object (MANAGED)
    Flush the EM
    UoW calculates changes => INSERT SQL
    Transaction
    

    View Slide

30. GETTING
    Request object
    Identity Map lookup
    Database query
    Object hydration
    Save hydrated object to IM (MANAGED)
    

    View Slide

31. UPDATING
    Get the object (MANAGED)
    Change a property
    Persist the object (no-op)
    Flush the EM
    UoW calculates changes => UPDATE SQL
    Transaction
    

    View Slide

32. DELETING
    Get the object (MANAGED)
    Ask for removal (REMOVED)
    Flush the EM
    UoW calculates changes => DELETE SQL
    Transaction
    

    View Slide

33. CHANGE TRACKING POLICIES
    

    View Slide

34. DEFERRED IMPLICIT
    Default
    Property-by-property comparison
    Persistence by reachability
    Slowest
    

    View Slide

35. DEFERRED EXPLICIT
    Property-by-property comparison
    EntityManager#persist(entity)
    No "dirty" checking
    

    View Slide

36. DEFERRED EXPLICIT
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    /**
    * @Entity
    * @ChangeTrackingPolicy("DEFERRED_EXPLICIT")
    */
    class User
    {
    // ...
    }
    

    View Slide

37. NOTIFY
    Notify interested listeners of changes
    NotifyPropertyChanged
    

    View Slide

38. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    use Doctrine\Common\NotifyPropertyChanged,
    Doctrine\Common\PropertyChangedListener;
    /**
    * @Entity
    * @ChangeTrackingPolicy("NOTIFY")
    */
    class MyEntity implements NotifyPropertyChanged
    {
    private $listeners = array();
    public function addPropertyChangedListener(PropertyChangedListener $listener)
    {
    $this->listeners[] = $listener;
    }
    }
    

    View Slide

39. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    class MyEntity implements NotifyPropertyChanged
    {
    protected function onPropertyChanged($propName, $oldValue, $newValue)
    {
    if ($this->listeners) {
    foreach ($this->listeners as $listener) {
    $listener->propertyChanged($this, $propName, $oldValue, $newValue);
    }
    }
    }
    public function setData($data)
    {
    if ($data != $this->data) {
    $this->onPropertyChanged('data', $this->data, $data);
    $this->data = $data;
    }
    }
    }
    

    View Slide

40. FURTHER
    IMPROVEMENTS
    

    View Slide

41. ALTERNATIVE QUERY RESULT
    FORMATS
    Read only data
    Scalar values
    Nested array graph
    

    View Slide

42. ALTERNATIVE QUERY RESULT
    FORMATS
    $dql = "SELECT b, e, r, p FROM Bug b JOIN b.engineer e ".
    "JOIN b.reporter r JOIN b.products p ORDER BY b.created DESC";
    $query = $entityManager->createQuery($dql);
    $bugs = $query->getArrayResult();
    foreach ($bugs as $bug) {
    echo $bug['description'] . " - " . $bug['created']->format('d.m.Y')."\n";
    echo " Reported by: ".$bug['reporter']['name']."\n";
    echo " Assigned to: ".$bug['engineer']['name']."\n";
    foreach ($bug['products'] as $product) {
    echo " Platform: ".$product['name']."\n";
    }
    echo "\n";
    }
    

    View Slide

43. READ ONLY ENTITIES
    /**
    * @Entity(readOnly=true)
    * @Table(name="products")
    */
    class Product
    {
    }
    

    View Slide

44. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL
    

    View Slide

45. EXTRA-LAZY COLLECTIONS
    LAZY (default)
    EAGER (join)
    EXTRA_LAZY
    

    View Slide

46. EXTRA-LAZY COLLECTIONS
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    namespace Doctrine\Tests\Models\CMS;
    use Doctrine\ORM\Annotation as ORM;
    /**
    * @ORM\Entity
    */
    class CmsGroup
    {
    /**
    * @ORM\ManyToMany(
    * targetEntity="CmsUser", mappedBy="groups", fetch="EXTRA_LAZY"
    * )
    */
    public $users;
    }
    

    View Slide

47. DOCTRINE\CACHE
    

    View Slide

48. DOCTRINE\COMMON\
    CACHE\CACHE
    INTERFACE
    fetch($id)
    contains($id)
    save($id, $data, $lifeTime = false)
    delete($id)
    

    View Slide

49. CACHE DRIVERS
    APCu
    Memcached
    Xcache
    Redis
    File
    Array*
    ...
    

    View Slide

50. CACHING IN DOCTRINE
    

    View Slide

51. METADATA
    Removes parsing overhead (Annotations, YML,
    XML)
    

    View Slide

52. QUERY
    DQL => SQL
    

    View Slide

53. RESULT
    Store SQL data result to cache
    Still a need to hydrate objects
    Joins will be stored as-is
    

    View Slide

54. CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    

    View Slide

55. HYDRATED RESULT
    

    View Slide

56. REPOSITORY DECORATOR
    

    View Slide

57. GENERALLY A BAD IDEA
    Serializing entities?
    Serializing collections?
    Detached entities
    https://github.com/doctrine/doctrine2/pull/172/files
    

    View Slide

58. View Slide

59. https://cowburn.info/public/files/php7_logo.svg
    https://cowburn.info/public/files/php7_logo.svg
    

    View Slide

60. SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    

    View Slide

61. View Slide

62. SECOND LEVEL CACHING
    No entity instances
    Only entity identifier and values
    Best suited for read-only data
    

    View Slide

63. TYPES
    Entity data: id + values
    Collection data: ownerId id + list of ids
    Query data: list of ids
    

    View Slide

64. ENTITY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class Country
    {
    // ...
    }
    

    View Slide

65. ASSOCIATION CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class State
    {
    //...
    /**
    * @Cache()
    * @ManyToOne(targetEntity="Country")
    * @JoinColumn(name="country_id", referencedColumnName="id")
    */
    protected $country;
    }
    

    View Slide

66. REGIONS
    

    View Slide

67. MODES
    READ_ONLY
    NONSTRICT_READ_WRITE
    READ_WRITE
    

    View Slide

68. QUERY CACHE
    

    View Slide

69. QUERY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    $result1 = $em->createQuery('SELECT c FROM Country c ORDER BY c.name')
    ->setCacheable(true)
    ->getResult();
    

    View Slide

70. QUERY CACHE: MODES
    Cache::MODE_GET
    Cache::MODE_PUT
    Cache::MODE_NORMAL (default)
    Cache::MODE_REFRESH
    

    View Slide

71. QUERY CACHE: DELETE /
    UPDATE QUERIES
    

    View Slide

72. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute and invalidate
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->setHint(Query::HINT_CACHE_EVICT, true)
    ->execute();
    

    View Slide

73. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntityRegion('Entity\Country');
    

    View Slide

74. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntity('Entity\Country', 1);
    

    View Slide

75. LIMITATIONS
    Single application
    Single Primary key column
    

    View Slide

76. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL
    

    View Slide

77. CONCLUSION
    Keep Identity Map / internals in mind
    Using the basic caching (query + mapping) +
    opcode caching is a must
    Cache heavy queries with result cache
    Give Second Level Caching a try (even if only for
    entities)
    Query Cache !== Second Level Query Cache
    

    View Slide

78. http://doctrine-project.org/
    https://www.flickr.com/photos/spacex/40126461851
    https://github.com/coudenysj?tab=repositories&q=doctrine
    @coudenysj
    jachim.coudenys@combell.group
    THANK YOU
    joind.in/talk/2ce9e
    joind.in/talk/2ce9e
    

    View Slide