Using Document Databases with TYPO3 Flow

Presentation given during Inspiring Flow 2013 in Kolbermoor

Transcript

1. Inspiring people to share Inspiring people to share Using Document Databases with TYPO3 Flow
2. Karsten Dambekalns TYPO3 Neos and Flow developer 35 years old lives in Lübeck, Germany 1 wife, 3 sons 1 espresso machine likes canoeing & climbing
3. Persistence Basics in Flow
4. Flow uses Doctrine 2 ORM MySQL PostgreSQL SQLite Oracle Native and PDO DB drivers Doctrine DBAL Doctrine ORM TYPO3 Flow
5. Flow provides a centralized PersistenceManager Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository
6. Repositories encapsulate persistence concerns Doctrine\PersistenceManager Repository Doctrine\Repository Your Application Code
7. Models do not know about persistence
8. Models do not know (a lot) about persistence (internals)
9. Properties /** * @Flow\Entity */ class Organisation { /** * The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;
10. Relations /** * @Flow\Entity */ class Organisation { /** * @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;
11. Document Databases Provide some powerful advantages •schema-less storage •usually scale well •really good for non-relational data No more Object-Relational Mapping, now you have Object- Document Mapping Queries, but not as you know them •Most document databases do not allow queries on arbitary properties
12. Which Database to Choose Many different document databases exist Which one is right for your project depends on many factors •MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. •CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. A nice overview with much more can be found on http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
13. CouchDB Is one of the more widespread products Stores documents in JSON Provides a REST interface Provides master-master replication and MVCC
14. Futon on CouchDB
15. CouchDB REST API Create a database curl -X PUT http://127.0.0.1:5984/demo {"ok":true} Create documents curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8", "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
16. CouchDB Basics Fetch a document curl -X GET http://127.0.0.1:5984/demo/ 8843faaf0b831d364278331bc3001bd8 {"_id":"8843faaf0b831d364278331bc3001bd8", "_rev":"1-33b9fbce46930280dab37d672bbc8bb9", "company":"Example, Inc."}
17. CouchDB Basics Create a database using Futon or with curl -X PUT http://127.0.0.1:5984/demo Create documents using Futon or with curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' Fetch documents using Futon If you do manual work on the shell, try HTTPie instead of cURL Read guide.couchdb.org and docs.couchdb.org
18. CouchDB Basics Read guide.couchdb.org and docs.couchdb.org Learn about views and map/reduce
19. Hint: A better cURL If you do manual work on the shell, try HTTPie instead of cURL
20. TYPO3.CouchDB Developed for Rossmann by networkteam Fully replaces the ORM persistence Model annotations stay the same Provides basic QOM-to-View mapping Note: Not working with TYPO3 Flow 2.0 right now
21. Installation cd Packages/Application git clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git No composer package, so just clone it
22. Configuration TYPO3: FLOW3: persistence: # Options for the CouchDB backend backendOptions: # database: 'database_name' dataSourceName: 'http://127.0.0.1:5984' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~
23. Usage – Repository class TestEntityRepository extends Repository { /** * @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); } }
24. Usage – LuceneIndex class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex { /** * Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); } }
25. Usage – Views & Design Docs Views can be defined by •implementing TYPO3\CouchDB\ViewInterface •configuring TYPO3\CouchDB\QueryView Design documents can be defined by extending TYPO3\CouchDB\DesignDocument
26. Now for some bad news
27. Only CouchDB is supported Kristina Alexanderson CC BY-NC-SA 2.0
28. One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository
29. One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository
30. You can work around this! In your repositories you can basically do what you want •Open a direct database connection •Read CSV files •Connect to a Solr server •Instantiate another Doctrine EntityManager Do something more advanced, let Radmiraal.CouchDB inspire you As long as you encapsulate nicely!
31. And now some good news
32. Radmiraal.CouchDB
33. Radmiraal.CouchDB Developed for some projects having similar needs Started out as an extension to TYPO3.CouchDB Since then refactored & now based on Doctrine 2 CouchDB ODM Can be used alongside the usual ORM
34. Radmiraal.CouchDB
35. Radmiraal.CouchDB
36. "repositories": [ { "type": "git", "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git" } ], "require": { "radmiraal/couchdb": "dev-doctrine", } Installation
37. TYPO3: Flow: persistence: backendOptions: … as usual Radmiraal: CouchDB: persistence: backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd' Configuration
38. Usage – Models use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM; /** * @ODM\Document(indexed=true) */ class DataSet { /** * @var string * @ODM\Id(type="string") */ protected $id; /** * @var \Acme\Demo\Domain\Model\Organisation
39. Usage – Models /** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation; /** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;
40. Usage – Repository class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); } }
41. Usage – Repository class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute() }
42. Usage – Views & Design Docs
43. And now some even better news
44. Multiple backends in parallel Doctrine DBAL Doctrine ORM PersistenceManager Doctrine \Repository MongoDB \Repository Doctrine MongoDB MongoDB ODM Doctrine \Repository
45. The Future TYPO3 Flow will support multiple persistence backends in parallel: •of the same type: 2 MySQL and 1 PostgreSQL databases •of different types: MySQL and MongoDB
46. The Future Configuration extended but backwards compatible Consistent use independent of backend used: Annotations, result interfaces & queries stay consistent References across persistence backends? Eventually, but not right away…
47. You can help, too… The work on a proper ODM integration so far has been done and made possible by (applause!): •Beech IT •BKWI •Rens Admiraal •myself More work is needed, so if you think this is the way to go… contact me!
48. questions your now!
49. Thank You! These slides can be found at: http://speakerdeck.com/kdambekalns http://slideshare.net/kfish Give me feedback: karsten@typo3.org | karsten@dambekalns.de Book me: karsten@dambekalns.de Follow me on twitter: @kdambekalns Support me using My Amazon wishlist: http://bit.ly/kdambekalns-wishlist