Using Document Databases with TYPO3 Flow

Slide 1

Slide 1 text

Inspiring people to share Inspiring people to share Using Document Databases with TYPO3 Flow

Slide 2

Slide 2 text

Karsten Dambekalns TYPO3 Neos and Flow developer 35 years old lives in Lübeck, Germany 1 wife, 3 sons 1 espresso machine likes canoeing & climbing

Slide 3

Slide 3 text

Persistence Basics in Flow

Slide 4

Slide 4 text

Flow uses Doctrine 2 ORM MySQL PostgreSQL SQLite Oracle Native and PDO DB drivers Doctrine DBAL Doctrine ORM TYPO3 Flow

Slide 5

Slide 5 text

Flow provides a centralized PersistenceManager Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository

Slide 6

Slide 6 text

Repositories encapsulate persistence concerns Doctrine\PersistenceManager Repository Doctrine\Repository Your Application Code

Slide 7

Slide 7 text

Models do not know about persistence

Slide 8

Slide 8 text

Models do not know (a lot) about persistence (internals)

Slide 9

Slide 9 text

Properties /** * @Flow\Entity */ class Organisation { /** * The name * @var string * @Flow\Validate(type="NotEmpty") * @Flow\Validate(type="StringLength", options={"maximum"=40}) * @ORM\Column(length=40) */ protected $name;

Slide 10

Slide 10 text

Relations /** * @Flow\Entity */ class Organisation { /** * @var \Doctrine\Common\Collections\Collection <\Acme\Demo\Domain\Model\Alias> * @ORM\OneToMany(mappedBy="organisation",cascade={"persist"}) * @ORM\OrderBy({"name" = "ASC"}) * @Flow\Lazy */ protected $aliases;

Slide 11

Slide 11 text

Document Databases Provide some powerful advantages •schema-less storage •usually scale well •really good for non-relational data No more Object-Relational Mapping, now you have Object- Document Mapping Queries, but not as you know them •Most document databases do not allow queries on arbitary properties

Slide 12

Slide 12 text

Which Database to Choose Many different document databases exist Which one is right for your project depends on many factors •MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predeﬁned columns really holds you back. •CouchDB: For accumulating, occasionally changing data, on which pre-deﬁned queries are to be run. Places where versioning is important. A nice overview with much more can be found on http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Slide 13

Slide 13 text

CouchDB Is one of the more widespread products Stores documents in JSON Provides a REST interface Provides master-master replication and MVCC

Slide 14

Slide 14 text

Futon on CouchDB

Slide 15

Slide 15 text

CouchDB REST API Create a database curl -X PUT http://127.0.0.1:5984/demo {"ok":true} Create documents curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8", "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}

Slide 16

Slide 16 text

CouchDB Basics Fetch a document curl -X GET http://127.0.0.1:5984/demo/ 8843faaf0b831d364278331bc3001bd8 {"_id":"8843faaf0b831d364278331bc3001bd8", "_rev":"1-33b9fbce46930280dab37d672bbc8bb9", "company":"Example, Inc."}

Slide 17

Slide 17 text

CouchDB Basics Create a database using Futon or with curl -X PUT http://127.0.0.1:5984/demo Create documents using Futon or with curl -H 'Content-Type: application/json' \ -X POST http://127.0.0.1:5984/demo \ -d '{"company": "Example, Inc."}' Fetch documents using Futon If you do manual work on the shell, try HTTPie instead of cURL Read guide.couchdb.org and docs.couchdb.org

Slide 18

Slide 18 text

CouchDB Basics Read guide.couchdb.org and docs.couchdb.org Learn about views and map/reduce

Slide 19

Slide 19 text

Hint: A better cURL If you do manual work on the shell, try HTTPie instead of cURL

Slide 20

Slide 20 text

TYPO3.CouchDB Developed for Rossmann by networkteam Fully replaces the ORM persistence Model annotations stay the same Provides basic QOM-to-View mapping Note: Not working with TYPO3 Flow 2.0 right now

Slide 21

Slide 21 text

Installation cd Packages/Application git clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git No composer package, so just clone it

Slide 22

Slide 22 text

Conﬁguration TYPO3: FLOW3: persistence: # Options for the CouchDB backend backendOptions: # database: 'database_name' dataSourceName: 'http://127.0.0.1:5984' username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~

Slide 23

Slide 23 text

Usage – Repository class TestEntityRepository extends Repository { /** * @param string $name * @return \TYPO3\FLOW3\Persistence\QueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like('name', $name) ); return $query->execute(); } }

Slide 24

Slide 24 text

Usage – LuceneIndex class MyIndex extends \TYPO3\CouchDB\Domain\Index\LuceneIndex { /** * Configure the index * * @return void */ public function configure() { $this->forEntity('Acme\Demo\Domain\Model\SomeEntity') ->indexProperty('name') ->indexProperty('relatedValueObject.color'); } }

Slide 25

Slide 25 text

Usage – Views & Design Docs Views can be defined by •implementing TYPO3\CouchDB\ViewInterface •configuring TYPO3\CouchDB\QueryView Design documents can be defined by extending TYPO3\CouchDB\DesignDocument

Slide 26

Slide 26 text

Now for some bad news

Slide 27

Slide 27 text

Only CouchDB is supported Kristina Alexanderson CC BY-NC-SA 2.0

Slide 28

Slide 28 text

One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository

Slide 29

Slide 29 text

One Backend at a Time Doctrine DBAL Doctrine ORM Doctrine\PersistenceManager Repository Doctrine\Repository

Slide 30

Slide 30 text

You can work around this! In your repositories you can basically do what you want •Open a direct database connection •Read CSV ﬁles •Connect to a Solr server •Instantiate another Doctrine EntityManager Do something more advanced, let Radmiraal.CouchDB inspire you As long as you encapsulate nicely!

Slide 31

Slide 31 text

And now some good news

Slide 32

Slide 32 text

Radmiraal.CouchDB

Slide 33

Slide 33 text

Radmiraal.CouchDB Developed for some projects having similar needs Started out as an extension to TYPO3.CouchDB Since then refactored & now based on Doctrine 2 CouchDB ODM Can be used alongside the usual ORM

Slide 34

Slide 34 text

Radmiraal.CouchDB

Slide 35

Slide 35 text

Radmiraal.CouchDB

Slide 36

Slide 36 text

"repositories": [ { "type": "git", "url": "https://github.com/radmiraal/Radmiraal.CouchDB.git" } ], "require": { "radmiraal/couchdb": "dev-doctrine", } Installation

Slide 37

Slide 37 text

TYPO3: Flow: persistence: backendOptions: … as usual Radmiraal: CouchDB: persistence: backendOptions: databaseName: 'mycouchdb' username: 'mycouchuser' password: 'mycouchpassw0rd' Conﬁguration

Slide 38

Slide 38 text

Usage – Models use Doctrine\ODM\CouchDB\Mapping\Annotations as ODM; /** * @ODM\Document(indexed=true) */ class DataSet { /** * @var string * @ODM\Id(type="string") */ protected $id; /** * @var \Acme\Demo\Domain\Model\Organisation

Slide 39

Slide 39 text

Usage – Models /** * @var \Acme\Demo\Domain\Model\Organisation * @ODM\Field(type="string") * @ODM\Index */ protected $organisation; /** * @var \DateTime * @ODM\Field(type="datetime") * @ODM\Index */ protected $creationDateTime;

Slide 40

Slide 40 text

Usage – Repository class OrganisationBoundaryRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function findAll() { return $this->createQuery( 'organisationboundaries', 'by_organisation_id') ->execute(); } }

Slide 41

Slide 41 text

Usage – Repository class DataSetRepository extends \Radmiraal\CouchDB\Persistence\AbstractRepository { public function findByQuux(Quux $quux) { return $this->createQuery('datasets', 'most_recent', 'native') ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute() }

Slide 42

Slide 42 text

Usage – Views & Design Docs

Slide 43

Slide 43 text

And now some even better news

Slide 44

Slide 44 text

Multiple backends in parallel Doctrine DBAL Doctrine ORM PersistenceManager Doctrine \Repository MongoDB \Repository Doctrine MongoDB MongoDB ODM Doctrine \Repository

Slide 45

Slide 45 text

The Future TYPO3 Flow will support multiple persistence backends in parallel: •of the same type: 2 MySQL and 1 PostgreSQL databases •of different types: MySQL and MongoDB

Slide 46

Slide 46 text

The Future Conﬁguration extended but backwards compatible Consistent use independent of backend used: Annotations, result interfaces & queries stay consistent References across persistence backends? Eventually, but not right away…

Slide 47

Slide 47 text

You can help, too… The work on a proper ODM integration so far has been done and made possible by (applause!): •Beech IT •BKWI •Rens Admiraal •myself More work is needed, so if you think this is the way to go… contact me!

Slide 48

Slide 48 text

questions your now!

Slide 49

Slide 49 text

Thank You! These slides can be found at: http://speakerdeck.com/kdambekalns http://slideshare.net/kﬁsh Give me feedback: [email protected] | [email protected] Book me: [email protected] Follow me on twitter: @kdambekalns Support me using My Amazon wishlist: http://bit.ly/kdambekalns-wishlist

Slide 50

Slide 50 text

No content