Slide 1

Slide 1 text

Schema Evolution and Data Migration { "name" : "Timmo Freudl-Gierke", "twitter" : "@timmo_gierke", "email" : "[email protected]" }

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Schema / JSON Structure { name : "Timmo Freudl-Gierke", company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } Application Domain Model Speaker name company street zipCode city

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Speaker name company street zipCode city Speaker firstName lastName Company name street zipCode city 0,1 Speaker firstName lastName company street zipCode city Version 1 Version 2 Version 3

Slide 9

Slide 9 text

Speaker name company street zipCode city Speaker firstName lastName company street zipCode city Version 1 Version 2 db.speakers.find({}).forEach( function(speaker) { var original = speaker.name var first = original.match(/[A-Za-z]+/)[0] var last = original.match(/[A-Za-z-]+$/)[0] speaker.firstName = first speaker.lastName = last db. speakers.save(speaker) })

Slide 10

Slide 10 text

Speaker firstName lastName Company name street zipCode city 0,1 Speaker firstName lastName company street zipCode city Version 2 Version 3 db.speakers.update({},{$rename:{"company":"company.name"}}) db.speakers.update({},{$rename:{"street":"company.street"}}) db.speakers.update({},{$rename:{"zipCode":"company.zipCode"}}) db.speakers.update({},{$rename:{"city":"company.city"}}) http://docs.mongodb.org/manual/reference/operator/rename/

Slide 11

Slide 11 text

Speaker firstName lastName Company name street zipCode city 0,1 Speaker firstName lastName company street zipCode city Version 2 Version 3 DBObject speaker = ... DBObject company = new BasicDBObject(); company.put("name", speaker.get("name")); company.put("street", speaker.get("street")); company.put("zipCode", speaker.get("zipCode")); company.put("city", speaker.get("city")); speaker.put("company", company); speaker.removeField("street"); speaker.removeField("zipCode"); speaker.removeField("city"); collection.save(speaker);

Slide 12

Slide 12 text

@Test public void companySeperation() { // given String json = getFileAsJson("speaker_v2.json") DBObject speaker = (DBObject) JSON.parse(json); // when converter.seperateCompany(speaker); // then assertEquals(speaker.get("company.name"), "Hypoport AG"); assertFalse(speaker.containsField("street"); . . . } { firstName : "Timmo", lastName : "Freudl-Gierke" company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } speaker_v2.json Unit Testing

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Version 1 Version 2 Version 3 MongoDB 1. serialize() 2. addModelVersion() 3. save()

Slide 15

Slide 15 text

MongoDB speakers {name: "Timmo Freudl-Gierke ", modelVersion:1, ...} {name: "Joe Morrissey", modelVersion:1, ...} {name: "Stefan Rudnitzki ", modelVersion:2, ...} {name: "Christian Kvalheim ", modelVersion:1, ...} {name: "Ross Lawley ", modelVersion:2, ...} {name: "Thomas Rueckstiess ", modelVersion:3, ...} {name: "Johannes Hoppe ", modelVersion:3, ...} . . .

Slide 16

Slide 16 text

Version 1 Version 2 Version 3 MongoDB 1. find() 2. convert() 3. deserialize()

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

{ name : "Timmo Freudl-Gierke", company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } { name : "Timmo Freudl-Gierke", company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } { name : "Timmo Freudl-Gierke", company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } Multiple Version Converter Integration Test speaker_v1.json speaker_v2.json speaker_v3.json src/test/resources/

Slide 19

Slide 19 text

Multiple Version Converter Integration Test src/test/java/ @DataProvider public Object[][] allVersions() { return new Object[][] { {"speaker_v1.json"}, {"speaker_v2.json"}, {"speaker_v3.json"}, }} @Test public void deserializationOfAllVersion(String filename) { String json = getFileAsJson(filename) DBObject speaker = (DBObject) JSON.parse(json); speakerConverter.convert(speaker); Speaker s = deserializeSpeaker(speaker); // assert structure and relevant attributes . . . }

Slide 20

Slide 20 text

speakers = collection.find(queryBuilder().put("company").is("Hypoport AG")); speakers.addAll( collection.find(queryBuilder().put("company").is("Hypoport AG"))); speakers.addAll( collection.find(queryBuilder().put("company.name").is("Hypoport AG"))); Queries

Slide 21

Slide 21 text

SpringData-Integration @Component public class SpeakerMongoEventListener extends AbstractMongoEventListener { @Inject SpeakerVersionConverter versionConverter; @Override public void onBeforeSave(Speaker source, DBObject dbo) { setModelVersion(dbo); } @Override public void onAfterLoad(DBObject dbo) { versionConverter.convert(dbo); } }

Slide 22

Slide 22 text

Testing speakers {name: "Timmo Freudl-Gierke ", modelVersion: {name: "Joe Morrissey", modelVersion:1, ...} {name: "Stefan Rudnitzki ", modelVersion:2, ...} {name: "Christian Kvalheim ", modelVersion:1, ...} {name: "Ross Lawley ", modelVersion:2, ...} {name: "Thomas Rueckstiess ", modelVersion:3, ...} {name: "Johannes Hoppe ", modelVersion:3, ...} . . . $rename { name : "Timmo Freudl-Gierke", company : "Hypoport AG" street : "Klosterstr. 71", zipCode : "10179", city : "Berlin" } Speaker name company street zipCode city Text

Slide 23

Slide 23 text

{ "name" : "Timmo Freudl-Gierke", "twitter" : "@timmo_gierke", "email" : "[email protected]" } Schema Evolution and Data Migration