C T Modeling IT Infrastucture using The Assimilation Project #AssimProj @OSSAlanR http://assimproj.org/ http://bit.ly/AssimGC2013 Alan Robertson <[email protected]> Assimilation Systems Limited http://assimilationsystems.com
G R A P H C O N N E C T Upcoming Events GraphConnect San Francisco (today!) Open Source Monitoring Conference - Nürnberg NSA / Homeland Security Assimilation Technical Talk Large Installation System Administration Conference - DC Colorado Springs Open Source User’s Group linux.conf.au – Awesome Australian Linux Conf - Perth Details on http://assimilationsystems.com/
G R A P H C O N N E C T Assimilation Project History • Inspired by 2 million core computer (cyclops64) • Concerns for extreme scale • Topology aware monitoring • Topology discovery w/out security issues =►Discovery of everything!
G R A P H C O N N E C T Project Scope Zero-network-footprint continuous Discovery integrated with extreme-scale Monitoring • Continuous extensible discovery – systems, switches, services, dependencies – zero network footprint • Extensible exception monitoring – more than 100K systems • All data goes into central graph database
G R A P H C O N N E C T Discovery Discovering • systems you've forgotten • what you're not monitoring • whatever you'd like • without setting off security alarms
G R A P H C O N N E C T Why Discovery? • Documentation: incomplete, incorrect • Dependencies: unknown • Planning: Needs accurate data • Best Practices: Verification needs data • ITIL CMDB (Configuration Mgmt DataBase) Our Discovery: continuous, low-profile
G R A P H C O N N E C T Why Neo4j (graph db)? • Dependency & Discovery information: graph • Speed of graph traversals depends on size of subgraph, not total graph size • Root cause queries graph traversals – notoriously slow in relational databases • Visualization of relationships • Schema-less design: good for constantly changing heterogeneous environment • Graph Model === Object Model
G R A P H C O N N E C T OGM – Object Graph Mapping • Managing the Graph Nodes “disappears” • The Object Model is the Graph Model • Significant Improvement in Thinking Clarity – The “mud” is gone
G R A P H C O N N E C T OGM rules • Don't use Constructors • Relationships replace hash tables and object references and so on • Constructor parameter names match attribute names
G R A P H C O N N E C T OGM sample @RegisterGraphClass class Person(GraphNode): 'A Person - or at least an electronic representation of one' def __init__(self, firstname, lastname, dateofbirth=None): GraphNode.__init__(self, domain='global') self.firstname = firstname self.lastname = lastname if dateofbirth is not None: self.dateofbirth = dateofbirth else: self.dateofbirth='unknown' @staticmethod def __meta_keyattrs__(): 'Return our key attributes in order of significance' return ['lastname', 'firstname']
G R A P H C O N N E C T OGM sample @RegisterGraphClass class TestSystem(GraphNode): 'Some kind of semi-intelligent system' def __init__(self, designation, roles=[]): GraphNode.__init__(self, domain) self.designation = designation.lower() if roles == []: roles = [''] self.roles = roles def addroles(self, role): if self.roles[0] == '''': del self.roles[0] if isinstance(role, list): for arole in role: self.addroles(arole) elif role not in self.roles: self.roles.append(role)
G R A P H C O N N E C T OGM sample # (seven)-[:formerly]->(Annika) Annika = store.load_or_create(Person, firstname='Annika', lastname='Hansen') seven = store.load_or_create(Drone, designation='SevenOfNine', roles='Borg') store.relate(seven, 'formerly', Annika) store.commit()
G R A P H C O N N E C T OGM sample @RegisterGraphClass class TestDrone(TestSystem): def __init__(self, designation, roles=[]): TestSystem.__init__(self, designation=designation) if isinstance(roles, str): roles = [roles] roles.extend(['host', 'Drone']) System.__init__(self, designation, roles=roles)
G R A P H C O N N E C T Current State • First release was April 2013 • Great unit test infrastructure • Nanoprobe code – works well • Service monitoring works • Lacks digital signatures, encryption, compression • Reliable UDP comm code working • Several discovery methods written • CMA and database code restructuring near-complete • UI development underway • Licensed under the GPL, commercial license available
G R A P H C O N N E C T Future Plans • Production grade by end of year • Purchased support • “Real digital signatures, compression, encryption • Other security enhancements • Much more discovery • GUI • Alerting • Reporting • Add Statistical Monitoring • Best Practice Audits • Dynamic (aka cloud) specialization • Hundreds more ideas – See: https://trello.com/b/OpaED3AT
G R A P H C O N N E C T Get Involved! Powerful Ideas and Infrastucture Fun, ground-breaking project Looking for early adopters, testers!! Needs for every kind of skill • Awesome User Interfaces (UI/UX) • Evangelism, community building • Test Code (simulate 106 servers!) • Python, C, script coding • Documentation • Feedback: Testing, Ideas, Plans • Many others!
G R A P H C O N N E C T Resistance Is Futile! #AssimProj @OSSAlanR #AssimMon Project Web Site http://assimproj.org Blog techthoughts.typepad.com lists.community.tummy.com/cgi-bin/mailman/admin/assimilation
ossalanr
msyksphinz
%20(1).jpg){kind=avatar}
sakana_ai
nwiizo
duelist2020jp
bun913
akashi_sn
imtnd
masatokinugawa
_kensh
sansan_randd
ks91
yuitosato
rocio
sstephenson
mraible
stephaniewalter
shpigford
pedronauck
rasmusluckow
moore
sergeychernyshev
sonatard
62gerente
csswizardry
