Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Document Databases with TYPO3 Flow

Using Document Databases with TYPO3 Flow

Presentation given during Inspiring Flow 2013 in Kolbermoor

Karsten Dambekalns

April 19, 2013

More Decks by Karsten Dambekalns

Other Decks in Programming


  1. Karsten Dambekalns TYPO3 Neos and Flow developer 35 years old

    lives in Lübeck, Germany 1 wife, 3 sons 1 espresso machine likes canoeing & climbing
  2. Flow uses Doctrine 2 ORM MySQL PostgreSQL SQLite Oracle Native

    and PDO DB drivers Doctrine DBAL Doctrine ORM TYPO3 Flow
  3. Properties /** * @Flow\Entity */ class Organisation { /** *

    The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;
  4. Relations /** * @Flow\Entity */ class Organisation { /** *

    @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;
  5. Document Databases Provide some powerful advantages •schema-less storage •usually scale

    well •really good for non-relational data No more Object-Relational Mapping, now you have Object- Document Mapping Queries, but not as you know them •Most document databases do not allow queries on arbitary properties
  6. Which Database to Choose Many different document databases exist Which

    one is right for your project depends on many factors •MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. •CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. A nice overview with much more can be found on http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  7. CouchDB Is one of the more widespread products Stores documents

    in JSON Provides a REST interface Provides master-master replication and MVCC
  8. CouchDB REST API Create a database curl -X PUT

    {"ok":true} Create documents curl -H 'Content-Type: application/json' \ -X POST \ -d '{"company": "Example, Inc."}' {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8", "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
  9. CouchDB Basics Fetch a document curl -X GET 8843faaf0b831d364278331bc3001bd8

    {"_id":"8843faaf0b831d364278331bc3001bd8", "_rev":"1-33b9fbce46930280dab37d672bbc8bb9", "company":"Example, Inc."}
  10. CouchDB Basics Create a database using Futon or with curl

    -X PUT Create documents using Futon or with curl -H 'Content-Type: application/json' \ -X POST \ -d '{"company": "Example, Inc."}' Fetch documents using Futon If you do manual work on the shell, try HTTPie instead of cURL Read guide.couchdb.org and docs.couchdb.org
  11. Hint: A better cURL If you do manual work on

    the shell, try HTTPie instead of cURL
  12. TYPO3.CouchDB Developed for Rossmann by networkteam Fully replaces the ORM

    persistence Model annotations stay the same Provides basic QOM-to-View mapping Note: Not working with TYPO3 Flow 2.0 right now
  13. Configuration TYPO3: FLOW3: persistence: # Options for the CouchDB backend

    backendOptions: # database: 'database_name' dataSourceName: '' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~
  14. Usage – Repository class TestEntityRepository extends Repository { /** *

    @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); } }
  15. Usage – LuceneIndex class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex { /** *

    Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); } }
  16. Usage – Views & Design Docs Views can be defined

    by •implementing TYPO3\CouchDB\ViewInterface •configuring TYPO3\CouchDB\QueryView Design documents can be defined by extending TYPO3\CouchDB\DesignDocument
  17. You can work around this! In your repositories you can

    basically do what you want •Open a direct database connection •Read CSV files •Connect to a Solr server •Instantiate another Doctrine EntityManager Do something more advanced, let Radmiraal.CouchDB inspire you As long as you encapsulate nicely!
  18. Radmiraal.CouchDB Developed for some projects having similar needs Started out

    as an extension to TYPO3.CouchDB Since then refactored & now based on Doctrine 2 CouchDB ODM Can be used alongside the usual ORM
  19. TYPO3: Flow: persistence: backendOptions: … as usual Radmiraal: CouchDB: persistence:

    backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd' Configuration
  20. Usage – Models use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM; /** * @ODM\Document(indexed=true)

    */ class DataSet { /** * @var string * @ODM\Id(type="string") */ protected $id; /** * @var \Acme\Demo\Domain\Model\Organisation
  21. Usage – Models /** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") *

    @ODM\Index */ protected $organisation; /** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;
  22. Usage – Repository class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function

    findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); } }
  23. Usage – Repository class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function

    findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute() }
  24. Multiple backends in parallel Doctrine DBAL Doctrine ORM PersistenceManager Doctrine

    \Repository MongoDB \Repository Doctrine MongoDB MongoDB ODM Doctrine \Repository
  25. The Future TYPO3 Flow will support multiple persistence backends in

    parallel: •of the same type: 2 MySQL and 1 PostgreSQL databases •of different types: MySQL and MongoDB
  26. The Future Configuration extended but backwards compatible Consistent use independent

    of backend used: Annotations, result interfaces & queries stay consistent References across persistence backends? Eventually, but not right away…
  27. You can help, too… The work on a proper ODM

    integration so far has been done and made possible by (applause!): •Beech IT •BKWI •Rens Admiraal •myself More work is needed, so if you think this is the way to go… contact me!
  28. Thank You! These slides can be found at: http://speakerdeck.com/kdambekalns http://slideshare.net/kfish

    Give me feedback: [email protected] | [email protected] Book me: [email protected] Follow me on twitter: @kdambekalns Support me using My Amazon wishlist: http://bit.ly/kdambekalns-wishlist