Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

How Doctrine caching can skyrocket your application (PHPCE Conference 2018)

When people talk about Doctrine (or any ORM for that matter), the performance issue always comes up fairly quickly. Besides the fact that Doctrine will help you develop faster, so a little overhead doesn't really matter, there are numerous options to increase the performance of the application.

By understanding how the system works in the first place, a lot of issues can be avoided right away.

When you have done everything to avoid these pitfalls, you can bring in the big guns: caching. Doctrine has different caching mechanism and since Doctrine 2.5 "Second Level Cache" was added to our toolbox. After this talk, you should know what the impact is of every cache and how to use it.

Jachim Coudenys

October 27, 2018
Tweet

More Decks by Jachim Coudenys

Other Decks in Technology

Transcript

  1. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    Jachim Coudenys
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj
    @coudenysj

    View full-size slide

  2. Jachim Coudenys
    Jachim Coudenys

    View full-size slide

  3. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION

    View full-size slide

  4. HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    HOW DOCTRINE
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    CACHING CAN
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    SKYROCKET YOUR
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    APPLICATION
    If you're using Doctrine ORM.
    If you're using Doctrine ORM.

    View full-size slide

  5. WHAT IS DOCTRINE

    View full-size slide

  6. OBJECT-RELATIONAL MAPPER
    ... is a programming technique for
    converting data between incompatible
    type systems using object-oriented
    programming languages.
    — https://en.wikipedia.org/wiki/Object-relational_mapping

    View full-size slide

  7. TERMINOLOGY
    Entity
    Mapping
    Repository
    EntityManager

    View full-size slide

  8. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View full-size slide

  9. SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS
    SOME INTERNALS

    View full-size slide

  10. ENTITYMANAGER

    View full-size slide

  11. UNIT OF WORK
    Maintains a list of objects affected by a
    business transaction and coordinates
    the writing out of changes and the
    resolution of concurrency problems.
    — https://martinfowler.com/eaaCatalog/unitOfWork.html

    View full-size slide

  12. TRANSACTIONAL WRITE-
    BEHIND

    View full-size slide

  13. IDENTITY MAP
    Ensures that each object gets loaded
    only once by keeping every loaded
    object in a map. Looks up objects using
    the map when referring to them.
    — https://martinfowler.com/eaaCatalog/identityMap.html

    View full-size slide

  14. ENTITY STATES
    NEW
    MANAGED
    DETACHED
    REMOVED

    View full-size slide

  15. PROXIES (LAZY LOADING)
    An object that doesn't contain all of
    the data you need but knows how to
    get it.
    — https://martinfowler.com/eaaCatalog/lazyLoad.html

    View full-size slide

  16. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View full-size slide

  17. TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES
    TRACKING CHANGES

    View full-size slide

  18. CREATING
    Create new object (NEW)
    Persist the object (MANAGED)
    Flush the EM
    UoW calculates changes => INSERT SQL
    Transaction

    View full-size slide

  19. GETTING
    Request object
    Identity Map lookup
    Database query
    Object hydration
    Save hydrated object to IM (MANAGED)

    View full-size slide

  20. UPDATING
    Get the object (MANAGED)
    Change a property
    Persist the object (no-op)
    Flush the EM
    UoW calculates changes => UPDATE SQL
    Transaction

    View full-size slide

  21. DELETING
    Get the object (MANAGED)
    Ask for removal (REMOVED)
    Flush the EM
    UoW calculates changes => DELETE SQL
    Transaction

    View full-size slide

  22. CHANGE TRACKING POLICIES

    View full-size slide

  23. DEFERRED IMPLICIT
    Default
    Property-by-property comparison
    Persistence by reachability
    Slowest

    View full-size slide

  24. DEFERRED EXPLICIT
    Property-by-property comparison
    EntityManager#persist(entity)
    No "dirty" checking

    View full-size slide

  25. DEFERRED EXPLICIT
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit
    /**
    * @Entity
    * @ChangeTrackingPolicy("DEFERRED_EXPLICIT")
    */
    class User
    {
    // ...
    }

    View full-size slide

  26. NOTIFY
    Notify interested listeners of changes
    NotifyPropertyChanged

    View full-size slide

  27. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    use Doctrine\Common\NotifyPropertyChanged,
    Doctrine\Common\PropertyChangedListener;
    /**
    * @Entity
    * @ChangeTrackingPolicy("NOTIFY")
    */
    class MyEntity implements NotifyPropertyChanged
    {
    private $listeners = array();
    public function addPropertyChangedListener(PropertyChangedListener $listener)
    {
    $this->listeners[] = $listener;
    }
    }

    View full-size slide

  28. NOTIFY
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify
    class MyEntity implements NotifyPropertyChanged
    {
    protected function onPropertyChanged($propName, $oldValue, $newValue)
    {
    if ($this->listeners) {
    foreach ($this->listeners as $listener) {
    $listener->propertyChanged($this, $propName, $oldValue, $newValue);
    }
    }
    }
    public function setData($data)
    {
    if ($data != $this->data) {
    $this->onPropertyChanged('data', $this->data, $data);
    $this->data = $data;
    }
    }
    }

    View full-size slide

  29. FURTHER
    IMPROVEMENTS

    View full-size slide

  30. ALTERNATIVE QUERY RESULT
    FORMATS
    Read only data
    Scalar values
    Nested array graph

    View full-size slide

  31. ALTERNATIVE QUERY RESULT
    FORMATS
    $dql = "SELECT b, e, r, p FROM Bug b JOIN b.engineer e ".
    "JOIN b.reporter r JOIN b.products p ORDER BY b.created DESC";
    $query = $entityManager->createQuery($dql);
    $bugs = $query->getArrayResult();
    foreach ($bugs as $bug) {
    echo $bug['description'] . " - " . $bug['created']->format('d.m.Y')."\n";
    echo " Reported by: ".$bug['reporter']['name']."\n";
    echo " Assigned to: ".$bug['engineer']['name']."\n";
    foreach ($bug['products'] as $product) {
    echo " Platform: ".$product['name']."\n";
    }
    echo "\n";
    }

    View full-size slide

  32. READ ONLY ENTITIES
    /**
    * @Entity(readOnly=true)
    * @Table(name="products")
    */
    class Product
    {
    }

    View full-size slide

  33. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View full-size slide

  34. EXTRA-LAZY COLLECTIONS
    LAZY (default)
    EAGER (join)
    EXTRA_LAZY

    View full-size slide

  35. EXTRA-LAZY COLLECTIONS
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations
    namespace Doctrine\Tests\Models\CMS;
    use Doctrine\ORM\Annotation as ORM;
    /**
    * @ORM\Entity
    */
    class CmsGroup
    {
    /**
    * @ORM\ManyToMany(
    * targetEntity="CmsUser", mappedBy="groups", fetch="EXTRA_LAZY"
    * )
    */
    public $users;
    }

    View full-size slide

  36. DOCTRINE\CACHE

    View full-size slide

  37. DOCTRINE\COMMON\
    CACHE\CACHE
    INTERFACE
    fetch($id)
    contains($id)
    save($id, $data, $lifeTime = false)
    delete($id)

    View full-size slide

  38. CACHE DRIVERS
    APCu
    Memcached
    Xcache
    Redis
    File
    Array*
    ...

    View full-size slide

  39. CACHING IN DOCTRINE

    View full-size slide

  40. METADATA
    Removes parsing overhead (Annotations, YML,
    XML)

    View full-size slide

  41. QUERY
    DQL => SQL

    View full-size slide

  42. RESULT
    Store SQL data result to cache
    Still a need to hydrate objects
    Joins will be stored as-is

    View full-size slide

  43. CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING
    CUSTOM CACHING

    View full-size slide

  44. HYDRATED RESULT

    View full-size slide

  45. REPOSITORY DECORATOR

    View full-size slide

  46. GENERALLY A BAD IDEA
    Serializing entities?
    Serializing collections?
    Detached entities
    https://github.com/doctrine/doctrine2/pull/172/files

    View full-size slide

  47. https://cowburn.info/public/files/php7_logo.svg
    https://cowburn.info/public/files/php7_logo.svg

    View full-size slide

  48. SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    SECOND LEVEL
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING
    CACHING

    View full-size slide

  49. SECOND LEVEL CACHING
    No entity instances
    Only entity identifier and values
    Best suited for read-only data

    View full-size slide

  50. TYPES
    Entity data: id + values
    Collection data: ownerId id + list of ids
    Query data: list of ids

    View full-size slide

  51. ENTITY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class Country
    {
    // ...
    }

    View full-size slide

  52. ASSOCIATION CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition
    /**
    * @Entity
    * @Cache()
    */
    class State
    {
    //...
    /**
    * @Cache()
    * @ManyToOne(targetEntity="Country")
    * @JoinColumn(name="country_id", referencedColumnName="id")
    */
    protected $country;
    }

    View full-size slide

  53. MODES
    READ_ONLY
    NONSTRICT_READ_WRITE
    READ_WRITE

    View full-size slide

  54. QUERY CACHE
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache
    $result1 = $em->createQuery('SELECT c FROM Country c ORDER BY c.name')
    ->setCacheable(true)
    ->getResult();

    View full-size slide

  55. QUERY CACHE: MODES
    Cache::MODE_GET
    Cache::MODE_PUT
    Cache::MODE_NORMAL (default)
    Cache::MODE_REFRESH

    View full-size slide

  56. QUERY CACHE: DELETE /
    UPDATE QUERIES

    View full-size slide

  57. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute and invalidate
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->setHint(Query::HINT_CACHE_EVICT, true)
    ->execute();

    View full-size slide

  58. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntityRegion('Entity\Country');

    View full-size slide

  59. QUERY CACHE: DELETE /
    UPDATE QUERIES
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries
    // Execute
    $this->em->createQuery(
    "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1"
    )
    ->execute();
    // Invoke Cache API
    $em->getCache()->evictEntity('Entity\Country', 1);

    View full-size slide

  60. LIMITATIONS
    Single application
    Single Primary key column

    View full-size slide

  61. https://github.com/coudenysj/doctrine2-orm-tutorial
    https://github.com/coudenysj/doctrine2-orm-tutorial
    DOCTRINE2-ORM-TUTORIAL

    View full-size slide

  62. CONCLUSION
    Keep Identity Map / internals in mind
    Using the basic caching (query + mapping) +
    opcode caching is a must
    Cache heavy queries with result cache
    Give Second Level Caching a try (even if only for
    entities)
    Query Cache !== Second Level Query Cache

    View full-size slide

  63. http://doctrine-project.org/
    https://www.flickr.com/photos/spacex/40126461851
    https://github.com/coudenysj?tab=repositories&q=doctrine
    @coudenysj
    [email protected]
    THANK YOU
    joind.in/talk/2ce9e
    joind.in/talk/2ce9e

    View full-size slide