Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Document Databases with TYPO3 Flow

Using Document Databases with TYPO3 Flow

Presentation given during Inspiring Flow 2013 in Kolbermoor

Ecfb276920529fc9b924ad4a0877d9ce?s=128

Karsten Dambekalns

April 19, 2013
Tweet

Transcript

  1. Inspiring people to share Inspiring people to share Using Document

    Databases with TYPO3 Flow
  2. Karsten Dambekalns TYPO3 Neos and Flow developer 35 years old

    lives in Lübeck, Germany 1 wife, 3 sons 1 espresso machine likes canoeing & climbing
  3. Persistence Basics in Flow

  4. Flow uses Doctrine 2 ORM MySQL PostgreSQL SQLite Oracle Native

    and PDO DB drivers Doctrine DBAL Doctrine ORM TYPO3 Flow
  5. Flow provides a centralized PersistenceManager Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager

    Repository Doctrine\Repository
  6. Repositories encapsulate persistence concerns Doctrine\PersistenceManager Repository Doctrine\Repository Your Application Code

  7. Models do not know about persistence

  8. Models do not know (a lot) about persistence (internals)

  9. Properties /** * @Flow\Entity */ class Organisation { /** *

    The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;
  10. Relations /** * @Flow\Entity */ class Organisation { /** *

    @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;
  11. Document Databases Provide some powerful advantages •schema-less storage •usually scale

    well •really good for non-relational data No more Object-Relational Mapping, now you have Object- Document Mapping Queries, but not as you know them •Most document databases do not allow queries on arbitary properties
  12. Which Database to Choose Many different document databases exist Which

    one is right for your project depends on many factors •MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. •CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. A nice overview with much more can be found on http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  13. CouchDB Is one of the more widespread products Stores documents

    in JSON Provides a REST interface Provides master-master replication and MVCC
  14. Futon on CouchDB

  15. CouchDB REST API Create a database curl -X PUT http://127.0.0.1:5984/demo

    {"ok":true} Create documents curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8", "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
  16. CouchDB Basics Fetch a document curl -X GET http://127.0.0.1:5984/demo/ 8843faaf0b831d364278331bc3001bd8

    {"_id":"8843faaf0b831d364278331bc3001bd8", "_rev":"1-33b9fbce46930280dab37d672bbc8bb9", "company":"Example, Inc."}
  17. CouchDB Basics Create a database using Futon or with curl

    -X PUT http://127.0.0.1:5984/demo Create documents using Futon or with curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' Fetch documents using Futon If you do manual work on the shell, try HTTPie instead of cURL Read guide.couchdb.org and docs.couchdb.org
  18. CouchDB Basics Read guide.couchdb.org and docs.couchdb.org Learn about views and

    map/reduce
  19. Hint: A better cURL If you do manual work on

    the shell, try HTTPie instead of cURL
  20. TYPO3.CouchDB Developed for Rossmann by networkteam Fully replaces the ORM

    persistence Model annotations stay the same Provides basic QOM-to-View mapping Note: Not working with TYPO3 Flow 2.0 right now
  21. Installation cd Packages/Application git clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git No composer package, so

    just clone it
  22. Configuration TYPO3: FLOW3: persistence: # Options for the CouchDB backend

    backendOptions: # database: 'database_name' dataSourceName: 'http://127.0.0.1:5984' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~
  23. Usage – Repository class TestEntityRepository extends Repository { /** *

    @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); } }
  24. Usage – LuceneIndex class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex { /** *

    Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); } }
  25. Usage – Views & Design Docs Views can be defined

    by •implementing TYPO3\CouchDB\ViewInterface •configuring TYPO3\CouchDB\QueryView Design documents can be defined by extending TYPO3\CouchDB\DesignDocument
  26. Now for some bad news

  27. Only CouchDB is supported Kristina Alexanderson CC BY-NC-SA 2.0

  28. One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager

    Repository Doctrine\Repository
  29. One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager

    Repository Doctrine\Repository
  30. You can work around this! In your repositories you can

    basically do what you want •Open a direct database connection •Read CSV files •Connect to a Solr server •Instantiate another Doctrine EntityManager Do something more advanced, let Radmiraal.CouchDB inspire you As long as you encapsulate nicely!
  31. And now some good news

  32. Radmiraal.CouchDB

  33. Radmiraal.CouchDB Developed for some projects having similar needs Started out

    as an extension to TYPO3.CouchDB Since then refactored & now based on Doctrine 2 CouchDB ODM Can be used alongside the usual ORM
  34. Radmiraal.CouchDB

  35. Radmiraal.CouchDB

  36. "repositories": [ { "type": "git", "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git" } ], "require":

    { "radmiraal/couchdb": "dev-doctrine", } Installation
  37. TYPO3: Flow: persistence: backendOptions: … as usual Radmiraal: CouchDB: persistence:

    backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd' Configuration
  38. Usage – Models use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM; /** * @ODM\Document(indexed=true)

    */ class DataSet { /** * @var string * @ODM\Id(type="string") */ protected $id; /** * @var \Acme\Demo\Domain\Model\Organisation
  39. Usage – Models /** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") *

    @ODM\Index */ protected $organisation; /** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;
  40. Usage – Repository class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function

    findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); } }
  41. Usage – Repository class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function

    findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute() }
  42. Usage – Views & Design Docs

  43. And now some even better news

  44. Multiple backends in parallel Doctrine DBAL Doctrine ORM PersistenceManager Doctrine

    \Repository MongoDB \Repository Doctrine MongoDB MongoDB ODM Doctrine \Repository
  45. The Future TYPO3 Flow will support multiple persistence backends in

    parallel: •of the same type: 2 MySQL and 1 PostgreSQL databases •of different types: MySQL and MongoDB
  46. The Future Configuration extended but backwards compatible Consistent use independent

    of backend used: Annotations, result interfaces & queries stay consistent References across persistence backends? Eventually, but not right away…
  47. You can help, too… The work on a proper ODM

    integration so far has been done and made possible by (applause!): •Beech IT •BKWI •Rens Admiraal •myself More work is needed, so if you think this is the way to go… contact me!
  48. questions your now!

  49. Thank You! These slides can be found at: http://speakerdeck.com/kdambekalns http://slideshare.net/kfish

    Give me feedback: karsten@typo3.org | karsten@dambekalns.de Book me: karsten@dambekalns.de Follow me on twitter: @kdambekalns Support me using My Amazon wishlist: http://bit.ly/kdambekalns-wishlist
  50. None