Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

When people talk about Doctrine (or any ORM for that matter), the performance issue always comes up fairly quickly. Besides the fact that Doctrine will help you develop faster, so a little overhead doesn't really matter, there are numerous options to increase the performance of the application.

By understanding how the system works in the first place, a lot of issues can be avoided right away.

When you have done everything to avoid these pitfalls, you can bring in the big guns: caching. Doctrine has different caching mechanism and since Doctrine 2.5 "Second Level Cache" was added to our toolbox. After this talk, you should know what the impact is of every cache and how to use it.

Jachim Coudenys

October 27, 2018
Tweet

More Decks by Jachim Coudenys

Other Decks in Technology

Transcript

  1. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj

    View Slide

  2. Jachim Coudenys
    Jachim Coudenys

    View Slide

  3. View Slide

  4. View Slide

  5. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION

    View Slide

  6. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    If you're using Doctrine ORM.
    If you're using Doctrine ORM.

    View Slide

  7. WHAT IS DOCTRINE

    View Slide

  8. OBJECT-RELATIONAL MAPPER
    ... is a programming technique for
    converting data between incompatible
    type systems using object-oriented
    programming languages.
    — https://en.wikipedia.org/wiki/Object-relational_mapping

    View Slide

  9. TERMINOLOGY
    Entity
    Mapping
    Repository
    EntityManager

    View Slide

  10. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View Slide

  11. SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS

    View Slide

  12. View Slide

  13. ENTITYMANAGER

    View Slide

  14. View Slide

  15. UNIT OF WORK
    Maintains a list of objects affected by a
    business transaction and coordinates
    the writing out of changes and the
    resolution of concurrency problems.
    — https://martinfowler.com/eaaCatalog/unitOfWork.html

    View Slide

  16. View Slide

  17. TRANSACTIONAL WRITE-
    BEHIND

    View Slide

  18. View Slide

  19. IDENTITY MAP
    Ensures that each object gets loaded
    only once by keeping every loaded
    object in a map. Looks up objects using
    the map when referring to them.
    — https://martinfowler.com/eaaCatalog/identityMap.html

    View Slide

  20. View Slide

  21. ENTITY STATES
    NEW
    MANAGED
    DETACHED
    REMOVED

    View Slide

  22. View Slide

  23. HYDRATION

    View Slide

  24. View Slide

  25. PROXIES (LAZY LOADING)
    An object that doesn't contain all of
    the data you need but knows how to
    get it.
    — https://martinfowler.com/eaaCatalog/lazyLoad.html

    View Slide

  26. View Slide

  27. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View Slide

  28. TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES

    View Slide

  29. CREATING
    Create new object (NEW)
    Persist the object (MANAGED)
    Flush the EM
    UoW calculates changes => INSERT SQL
    Transaction

    View Slide

  30. GETTING
    Request object
    Identity Map lookup
    Database query
    Object hydration
    Save hydrated object to IM (MANAGED)

    View Slide

  31. UPDATING
    Get the object (MANAGED)
    Change a property
    Persist the object (no-op)
    Flush the EM
    UoW calculates changes => UPDATE SQL
    Transaction

    View Slide

  32. DELETING
    Get the object (MANAGED)
    Ask for removal (REMOVED)
    Flush the EM
    UoW calculates changes => DELETE SQL
    Transaction

    View Slide

  33. CHANGE TRACKING POLICIES

    View Slide

  34. DEFERRED IMPLICIT
    Default
    Property-by-property comparison
    Persistence by reachability
    Slowest

    View Slide

  35. DEFERRED EXPLICIT
    Property-by-property comparison
    EntityManager#persist(entity)
    No "dirty" checking

    View Slide

  36. DEFERRED EXPLICIT
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    /**
    * @Entity
    * @ChangeTrackingPolicy("DEFERRED_EXPLICIT")
    */
    class User
    {
    // ...
    }

    View Slide

  37. NOTIFY
    Notify interested listeners of changes
    NotifyPropertyChanged

    View Slide

  38. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    use Doctrine\Common\NotifyPropertyChanged,
    Doctrine\Common\PropertyChangedListener;
    /**
    * @Entity
    * @ChangeTrackingPolicy("NOTIFY")
    */
    class MyEntity implements NotifyPropertyChanged
    {
    private $listeners = array();
    public function addPropertyChangedListener(PropertyChangedListener $listener)
    {
    $this->listeners[] = $listener;
    }
    }

    View Slide

  39. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    class MyEntity implements NotifyPropertyChanged
    {
    protected function onPropertyChanged($propName, $oldValue, $newValue)
    {
    if ($this->listeners) {
    foreach ($this->listeners as $listener) {
    $listener->propertyChanged($this, $propName, $oldValue, $newValue);
    }
    }
    }
    public function setData($data)
    {
    if ($data != $this->data) {
    $this->onPropertyChanged('data', $this->data, $data);
    $this->data = $data;
    }
    }
    }

    View Slide

  40. FURTHER
    IMPROVEMENTS

    View Slide

  41. ALTERNATIVE QUERY RESULT
    FORMATS
    Read only data
    Scalar values
    Nested array graph

    View Slide

  42. ALTERNATIVE QUERY RESULT
    FORMATS
    $dql = "SELECT b, e, r, p FROM Bug b JOIN b.engineer e ".
    "JOIN b.reporter r JOIN b.products p ORDER BY b.created DESC";
    $query = $entityManager->createQuery($dql);
    $bugs = $query->getArrayResult();
    foreach ($bugs as $bug) {
    echo $bug['description'] . " - " . $bug['created']->format('d.m.Y')."\n";
    echo " Reported by: ".$bug['reporter']['name']."\n";
    echo " Assigned to: ".$bug['engineer']['name']."\n";
    foreach ($bug['products'] as $product) {
    echo " Platform: ".$product['name']."\n";
    }
    echo "\n";
    }

    View Slide

  43. READ ONLY ENTITIES
    /**
    * @Entity(readOnly=true)
    * @Table(name="products")
    */
    class Product
    {
    }

    View Slide

  44. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View Slide

  45. EXTRA-LAZY COLLECTIONS
    LAZY (default)
    EAGER (join)
    EXTRA_LAZY

    View Slide

  46. EXTRA-LAZY COLLECTIONS
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    namespace Doctrine\Tests\Models\CMS;
    use Doctrine\ORM\Annotation as ORM;
    /**
    * @ORM\Entity
    */
    class CmsGroup
    {
    /**
    * @ORM\ManyToMany(
    * targetEntity="CmsUser", mappedBy="groups", fetch="EXTRA_LAZY"
    * )
    */
    public $users;
    }

    View Slide

  47. DOCTRINE\CACHE

    View Slide

  48. DOCTRINE\COMMON\
    CACHE\CACHE
    INTERFACE
    fetch($id)
    contains($id)
    save($id, $data, $lifeTime = false)
    delete($id)

    View Slide

  49. CACHE DRIVERS
    APCu
    Memcached
    Xcache
    Redis
    File
    Array*
    ...

    View Slide

  50. CACHING IN DOCTRINE

    View Slide

  51. METADATA
    Removes parsing overhead (Annotations, YML,
    XML)

    View Slide

  52. QUERY
    DQL => SQL

    View Slide

  53. RESULT
    Store SQL data result to cache
    Still a need to hydrate objects
    Joins will be stored as-is

    View Slide

  54. CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING

    View Slide

  55. HYDRATED RESULT

    View Slide

  56. REPOSITORY DECORATOR

    View Slide

  57. GENERALLY A BAD IDEA
    Serializing entities?
    Serializing collections?
    Detached entities
    https://github.com/doctrine/doctrine2/pull/172/files

    View Slide

  58. View Slide

  59. https://cowburn.info/public/files/php7_logo.svg
    https://cowburn.info/public/files/php7_logo.svg

    View Slide

  60. SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING

    View Slide

  61. View Slide

  62. SECOND LEVEL CACHING
    No entity instances
    Only entity identifier and values
    Best suited for read-only data

    View Slide

  63. TYPES
    Entity data: id + values
    Collection data: ownerId id + list of ids
    Query data: list of ids

    View Slide

  64. ENTITY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class Country
    {
    // ...
    }

    View Slide

  65. ASSOCIATION CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class State
    {
    //...
    /**
    * @Cache()
    * @ManyToOne(targetEntity="Country")
    * @JoinColumn(name="country_id", referencedColumnName="id")
    */
    protected $country;
    }

    View Slide

  66. REGIONS

    View Slide

  67. MODES
    READ_ONLY
    NONSTRICT_READ_WRITE
    READ_WRITE

    View Slide

  68. QUERY CACHE

    View Slide

  69. QUERY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    $result1 = $em->createQuery('SELECT c FROM Country c ORDER BY c.name')
    ->setCacheable(true)
    ->getResult();

    View Slide

  70. QUERY CACHE: MODES
    Cache::MODE_GET
    Cache::MODE_PUT
    Cache::MODE_NORMAL (default)
    Cache::MODE_REFRESH

    View Slide

  71. QUERY CACHE: DELETE /
    UPDATE QUERIES

    View Slide

  72. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute and invalidate
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->setHint(Query::HINT_CACHE_EVICT, true)
    ->execute();

    View Slide

  73. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntityRegion('Entity\Country');

    View Slide

  74. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntity('Entity\Country', 1);

    View Slide

  75. LIMITATIONS
    Single application
    Single Primary key column

    View Slide

  76. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View Slide

  77. CONCLUSION
    Keep Identity Map / internals in mind
    Using the basic caching (query + mapping) +
    opcode caching is a must
    Cache heavy queries with result cache
    Give Second Level Caching a try (even if only for
    entities)
    Query Cache !== Second Level Query Cache

    View Slide

  78. http://doctrine-project.org/
    https://www.flickr.com/photos/spacex/40126461851
    https://github.com/coudenysj?tab=repositories&q=doctrine
    @coudenysj
    [email protected]
    THANK YOU
    joind.in/talk/2ce9e
    joind.in/talk/2ce9e

    View Slide