Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Document Databases with TYPO3 Flow

Using Document Databases with TYPO3 Flow

Presentation given during Inspiring Flow 2013 in Kolbermoor

Karsten Dambekalns

April 19, 2013
Tweet

More Decks by Karsten Dambekalns

Other Decks in Programming

Transcript

  1. Inspiring people to
    share
    Inspiring people to
    share
    Using Document
    Databases with
    TYPO3 Flow

    View Slide

  2. Karsten Dambekalns
    TYPO3 Neos and Flow developer
    35 years old
    lives in Lübeck, Germany
    1 wife, 3 sons
    1 espresso machine
    likes canoeing & climbing

    View Slide

  3. Persistence Basics
    in Flow

    View Slide

  4. Flow uses Doctrine 2 ORM
    MySQL PostgreSQL SQLite Oracle
    Native and PDO DB drivers
    Doctrine DBAL
    Doctrine ORM
    TYPO3 Flow

    View Slide

  5. Flow provides a centralized
    PersistenceManager
    Doctrine DBAL
    Doctrine ORM
    Doctrine\PersistenceManager
    Repository Doctrine\Repository

    View Slide

  6. Repositories encapsulate
    persistence concerns
    Doctrine\PersistenceManager
    Repository Doctrine\Repository
    Your Application Code

    View Slide

  7. Models do not know
    about persistence

    View Slide

  8. Models do not know
    (a lot)
    about persistence
    (internals)

    View Slide

  9. Properties
    /**
    * @Flow\Entity
    */
    class Organisation {
    /**
    * The name
    * @var string
    * @Flow\Validate(type="NotEmpty")
    * @Flow\Validate(type="StringLength", options={"maximum"=40})
    * @ORM\Column(length=40)
    */
    protected $name;

    View Slide

  10. Relations
    /**
    * @Flow\Entity
    */
    class Organisation {
    /**
    * @var \Doctrine\Common\Collections\Collection
    <\Acme\Demo\Domain\Model\Alias>
    * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"})
    * @ORM\OrderBy({"name" = "ASC"})
    * @Flow\Lazy
    */
    protected $aliases;

    View Slide

  11. Document Databases
    Provide some powerful advantages
    •schema-less storage
    •usually scale well
    •really good for non-relational data
    No more Object-Relational Mapping, now you have Object-
    Document Mapping
    Queries, but not as you know them
    •Most document databases do not allow queries on arbitary
    properties

    View Slide

  12. Which Database to Choose
    Many different document databases exist
    Which one is right for your project depends on many factors
    •MongoDB: For most things that you would do with MySQL or
    PostgreSQL, but having predefined columns really holds you back.
    •CouchDB: For accumulating, occasionally changing data, on which
    pre-defined queries are to be run. Places where versioning is
    important.
    A nice overview with much more can be found on
    http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

    View Slide

  13. CouchDB
    Is one of the more widespread products
    Stores documents in JSON
    Provides a REST interface
    Provides master-master replication and MVCC

    View Slide

  14. Futon on CouchDB

    View Slide

  15. CouchDB REST API
    Create a database
    curl -X PUT http://127.0.0.1:5984/demo
    {"ok":true}
    Create documents
    curl -H 'Content-Type: application/json' \
    -X POST http://127.0.0.1:5984/demo \
    -d '{"company": "Example, Inc."}'
    {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8",
    "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}

    View Slide

  16. CouchDB Basics
    Fetch a document
    curl -X GET http://127.0.0.1:5984/demo/
    8843faaf0b831d364278331bc3001bd8
    {"_id":"8843faaf0b831d364278331bc3001bd8",
    "_rev":"1-33b9fbce46930280dab37d672bbc8bb9",
    "company":"Example, Inc."}

    View Slide

  17. CouchDB Basics
    Create a database using Futon or with
    curl -X PUT http://127.0.0.1:5984/demo
    Create documents using Futon or with
    curl -H 'Content-Type: application/json' \
    -X POST http://127.0.0.1:5984/demo \
    -d '{"company": "Example, Inc."}'
    Fetch documents using Futon
    If you do manual work on the shell, try HTTPie instead of cURL
    Read guide.couchdb.org and docs.couchdb.org

    View Slide

  18. CouchDB Basics
    Read guide.couchdb.org and docs.couchdb.org
    Learn about views and map/reduce

    View Slide

  19. Hint: A better cURL
    If you do manual work on the shell, try HTTPie instead of cURL

    View Slide

  20. TYPO3.CouchDB
    Developed for Rossmann by networkteam
    Fully replaces the ORM persistence
    Model annotations stay the same
    Provides basic QOM-to-View mapping
    Note: Not working with TYPO3 Flow 2.0 right now

    View Slide

  21. Installation
    cd Packages/Application
    git clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git
    No composer package, so just clone it

    View Slide

  22. Configuration
    TYPO3:
    FLOW3:
    persistence:
    # Options for the CouchDB backend
    backendOptions:
    # database: 'database_name'
    dataSourceName: 'http://127.0.0.1:5984'
    username: ~
    password: ~
    enableCouchdbLucene: no
    driver: ~
    path: ~

    View Slide

  23. Usage – Repository
    class TestEntityRepository extends Repository {
    /**
    * @param string $name
    * @return \TYPO3\FLOW3\Persistence\QueryResultInterface
    */
    public function findByNameLike($name) {
    $query = $this->testIndex->createQuery();
    $query->matching(
    $query->like('name', $name)
    );
    return $query->execute();
    }
    }

    View Slide

  24. Usage – LuceneIndex
    class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex {
    /**
    * Configure the index
    *
    * @return void
    */
    public function configure() {
    $this->forEntity('Acme\Demo\Domain\Model\SomeEntity')
    ->indexProperty('name')
    ->indexProperty('relatedValueObject.color');
    }
    }

    View Slide

  25. Usage – Views & Design Docs
    Views can be defined by
    •implementing TYPO3\CouchDB\ViewInterface
    •configuring TYPO3\CouchDB\QueryView
    Design documents can be defined by extending
    TYPO3\CouchDB\DesignDocument

    View Slide

  26. Now for some
    bad news

    View Slide

  27. Only CouchDB is supported
    Kristina Alexanderson
    CC BY-NC-SA 2.0

    View Slide

  28. One Backend at a Time
    Doctrine DBAL
    Doctrine ORM
    Doctrine\PersistenceManager
    Repository Doctrine\Repository

    View Slide

  29. One Backend at a Time
    Doctrine DBAL
    Doctrine ORM
    Doctrine\PersistenceManager
    Repository Doctrine\Repository

    View Slide

  30. You can work around this!
    In your repositories you can basically do what you want
    •Open a direct database connection
    •Read CSV files
    •Connect to a Solr server
    •Instantiate another Doctrine EntityManager
    Do something more advanced, let Radmiraal.CouchDB inspire you
    As long as you encapsulate nicely!

    View Slide

  31. And now some
    good news

    View Slide

  32. Radmiraal.CouchDB

    View Slide

  33. Radmiraal.CouchDB
    Developed for some projects having similar needs
    Started out as an extension to TYPO3.CouchDB
    Since then refactored & now based on Doctrine 2
    CouchDB ODM
    Can be used alongside the usual ORM

    View Slide

  34. Radmiraal.CouchDB

    View Slide

  35. Radmiraal.CouchDB

    View Slide

  36. "repositories": [
    {
    "type": "git",
    "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git"
    }
    ],
    "require": {
    "radmiraal/couchdb": "dev-doctrine",
    }
    Installation

    View Slide

  37. TYPO3:
    Flow:
    persistence:
    backendOptions:
    … as usual
    Radmiraal:
    CouchDB:
    persistence:
    backendOptions:
    databaseName: 'mycouchdb'
    username: 'mycouchuser'
    password: 'mycouchpassw0rd'
    Configuration

    View Slide

  38. Usage – Models
    use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM;
    /**
    * @ODM\Document(indexed=true)
    */
    class DataSet {
    /**
    * @var string
    * @ODM\Id(type="string")
    */
    protected $id;
    /**
    * @var \Acme\Demo\Domain\Model\Organisation

    View Slide

  39. Usage – Models
    /**
    * @var \Acme\Demo\Domain\Model\Organisation
    * @ODM\Field(type="string")
    * @ODM\Index
    */
    protected $organisation;
    /**
    * @var \DateTime
    * @ODM\Field(type="datetime")
    * @ODM\Index
    */
    protected $creationDateTime;

    View Slide

  40. Usage – Repository
    class OrganisationBoundaryRepository
    extends \Radmiraal\CouchDB\Persistence\AbstractRepository {
    public function findAll() {
    return $this->createQuery(
    'organisationboundaries', 'by_organisation_id')
    ->execute();
    }
    }

    View Slide

  41. Usage – Repository
    class DataSetRepository
    extends \Radmiraal\CouchDB\Persistence\AbstractRepository {
    public function findByQuux(Quux $quux) {
    return
    $this->createQuery('datasets', 'most_recent', 'native')
    ->setStartKey(array($this->getQueryMatchValue($quux)))
    ->setGroup(TRUE)
    ->setGroupLevel(1)
    ->setLimit(1)
    ->execute()
    }

    View Slide

  42. Usage – Views & Design Docs

    View Slide

  43. And now some
    even better news

    View Slide

  44. Multiple backends in parallel
    Doctrine DBAL
    Doctrine ORM
    PersistenceManager
    Doctrine
    \Repository
    MongoDB
    \Repository
    Doctrine MongoDB
    MongoDB ODM
    Doctrine
    \Repository

    View Slide

  45. The Future
    TYPO3 Flow will support multiple persistence backends in
    parallel:
    •of the same type: 2 MySQL and 1 PostgreSQL databases
    •of different types: MySQL and MongoDB

    View Slide

  46. The Future
    Configuration extended but backwards compatible
    Consistent use independent of backend used:
    Annotations, result interfaces & queries stay consistent
    References across persistence backends? Eventually, but not
    right away…

    View Slide

  47. You can help, too…
    The work on a proper ODM integration so far has been
    done and made possible by (applause!):
    •Beech IT
    •BKWI
    •Rens Admiraal
    •myself
    More work is needed, so if you think this is the way to
    go… contact me!

    View Slide

  48. questions
    your
    now!

    View Slide

  49. Thank You!
    These slides can be found at:
    http://speakerdeck.com/kdambekalns
    http://slideshare.net/kfish
    Give me feedback: [email protected] | [email protected]
    Book me: [email protected]
    Follow me on twitter: @kdambekalns
    Support me using
    My Amazon wishlist: http://bit.ly/kdambekalns-wishlist

    View Slide

  50. View Slide