App Server App Server App Server Camera Camera Camera Camera Frame & Feature Data (mostly) Inspection Metadata (mostly) • Cameras save frame images to GridFS • Inspections, measurements, etc. defined by metadata in MongoDB • Access locally + from the web Tuesday, December 4, 12
App Server App Server App Server Camera Camera Camera Camera Frame & Feature Data (mostly) Inspection Metadata (mostly) • Frames must still be saved, inspections performed • Tolerate being inaccessible from the cloud Tuesday, December 4, 12
(cloud) App Server App Server App Server App Server Camera Camera Camera Camera Frame & Feature Data (mostly) Inspection Metadata (mostly) Primary • Pros: • More conventional • Additional seconaries in the cloud • Cons: • Dependent on internet • Saving frame data up an ADSL connection??!! Tuesday, December 4, 12
(cloud) App Server App Server App Server App Server Camera Camera Camera Camera Frame & Feature Data (mostly) Inspection Metadata (mostly) Primary • Pros: • Frame data saves fast • Resilient to outages • Cons: • Backups / durability? • Seems wrong, somehow... Tuesday, December 4, 12
App Server App Server App Server Camera Camera Camera Camera Frame & Feature Data (mostly) Inspection Metadata (mostly) Primary Primary • Pros: • Frame data saves fast • Resilient to outages • Backups / durability in cloud • Cons: • But MongoDB doesn’t do multi-master replication Tuesday, December 4, 12
spec = dict(ts={'$gt': self.checkpoint}) q = self._oplog.find( spec, tailable=True, await_data=True) found=False for op in q.sort('$natural'): found = True self.checkpoint = op['ts'] for callback in self._callbacks.get( (op['ns'], op['op']), []): callback(**op) if found: sleep(0) else: sleep(1) Tuesday, December 4, 12
spec = dict(ts={'$gt': self.checkpoint}) q = self._oplog.find( spec, tailable=True, await_data=True) found=False for op in q.sort('$natural'): found = True self.checkpoint = op['ts'] for callback in self._callbacks.get( (op['ns'], op['op']), []): callback(**op) if found: sleep(0) else: sleep(1) Find ops Tuesday, December 4, 12
h, op, ns, o, o2=None, b=False): if op == 'i': if o.get(MMM_REPL_FLAG) == self.id: return o.setdefault(MMM_REPL_FLAG, src_id) collection.insert(o) elif op == 'u': upsert = b if any(k.startswith('$') for k in o): # With modifiers, check & update setters setters = o.setdefault('$set', {}) else: # Without modifiers, update doc directly setters = o if setters.get(MMM_REPL_FLAG) == self.id: return setters.setdefault(MMM_REPL_FLAG, src_id) collection.update(o2, o, upsert) elif op == 'd': justOne = b collection.remove(o) Tuesday, December 4, 12
h, op, ns, o, o2=None, b=False): if op == 'i': if o.get(MMM_REPL_FLAG) == self.id: return o.setdefault(MMM_REPL_FLAG, src_id) collection.insert(o) elif op == 'u': upsert = b if any(k.startswith('$') for k in o): # With modifiers, check & update setters setters = o.setdefault('$set', {}) else: # Without modifiers, update doc directly setters = o if setters.get(MMM_REPL_FLAG) == self.id: return setters.setdefault(MMM_REPL_FLAG, src_id) collection.update(o2, o, upsert) elif op == 'd': justOne = b collection.remove(o) Inserts Tuesday, December 4, 12
h, op, ns, o, o2=None, b=False): if op == 'i': if o.get(MMM_REPL_FLAG) == self.id: return o.setdefault(MMM_REPL_FLAG, src_id) collection.insert(o) elif op == 'u': upsert = b if any(k.startswith('$') for k in o): # With modifiers, check & update setters setters = o.setdefault('$set', {}) else: # Without modifiers, update doc directly setters = o if setters.get(MMM_REPL_FLAG) == self.id: return setters.setdefault(MMM_REPL_FLAG, src_id) collection.update(o2, o, upsert) elif op == 'd': justOne = b collection.remove(o) Inserts Updates Tuesday, December 4, 12
h, op, ns, o, o2=None, b=False): if op == 'i': if o.get(MMM_REPL_FLAG) == self.id: return o.setdefault(MMM_REPL_FLAG, src_id) collection.insert(o) elif op == 'u': upsert = b if any(k.startswith('$') for k in o): # With modifiers, check & update setters setters = o.setdefault('$set', {}) else: # Without modifiers, update doc directly setters = o if setters.get(MMM_REPL_FLAG) == self.id: return setters.setdefault(MMM_REPL_FLAG, src_id) collection.update(o2, o, upsert) elif op == 'd': justOne = b collection.remove(o) Inserts Updates Deletes Tuesday, December 4, 12