LCA2014 Providing a Continuously Updated ITIL CMDB using the Assimilation Project - SysAdmin Miniconf

A 15 minute talk on how the Assimilation Project creates a CMDB and continually keeps it up to date. This was presented at the 2014 linux.conf.au SysAdmin miniconf.

Transcript

1. L
   C
   A
   2
   0
   1
   4
   A Continuously Updated CMDB
   using
   The Assimilation Project
   #AssimProj @OSSAlanR
   http://assimproj.org/
   http://bit.ly/LCA2014-SysAdmin
   Alan Robertson
   Assimilation Systems Limited
   http://assimilationsystems.com
   

   View Slide

2. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 2/18
   L
   C
   A
   2
   0
   1
   4
   Assimilation Project Scope
   Zero-network-footprint continuous Discovery
   Integrated with extreme-scale Monitoring
   => Discovery creates a graph-based CMDB
   

   View Slide

3. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 3/18
   L
   C
   A
   2
   0
   1
   4
   Using a CMDB for
   Risk Management/Mitigation
   ●
   Intrusions
   ●
   Licensed Software
   ●
   Audit Risk
   ●
   System modeling
   ●
   Outages
   

   View Slide

4. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 4/18
   L
   C
   A
   2
   0
   1
   4
   Why a Configuration
   Management Database
   (CMDB)?
   ●
   Documentation: incomplete, incorrect
   ●
   Dependencies: unknown
   ●
   Planning: Needs accurate data
   ●
   Best Practices: Verification needs
   data
   ●
   Compliance
   ●
   Our Discovery: continuous, low-profile
   

   View Slide

5. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 5/18
   L
   C
   A
   2
   0
   1
   4
   Discovery Features
   ●
   Continuous Discovery
   ●
   Zero network footprint
   ●
   Discover dependency information
   ●
   Discovery drives monitoring
   ●
   Easily extensible
   ●
   Configuration-free setup (!)
   

   View Slide

6. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 6/18
   L
   C
   A
   2
   0
   1
   4
   What do we discover?
   ●
   IP and MAC addresses (servers, etc)
   ●
   Services and service details
   ●
   Switches, switch connections and settings
   ●
   Installed services
   ●
   OS configuration
   ●
   Whatever you want ;-)
   

   View Slide

7. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 7/18
   L
   C
   A
   2
   0
   1
   4
   Architectural Elements
   ●
   Collective Management Authority
   (CMA) – one per installation
   ●
   Nanoprobes (agents) – one per system
   

   View Slide

8. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 8/18
   L
   C
   A
   2
   0
   1
   4
   How does discovery work?
   Nanoprobe scripts perform
   discovery
   ●
   Each discovers one kind of information
   ●
   Can take arguments from environment
   ●
   Output JSON
   CMA stores Discovery Information
   ●
   JSON stored in Neo4j database
   ●
   CMA discovery plugins => graph nodes and
   relationships
   

   View Slide

9. linux.conf.au
   06 January
   2014
   LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 9/18
   L
   C
   A
   2
   0
   1
   4
   OS discovery JSON Snippet
   { "nodename": "alanr-1225B",
   "operating-system": "GNU/Linux",
   "machine": "x86_64",
   "processor": "x86_64",
   "hardware-platform": "x86_64",
   "kernel-name": "Linux",
   "kernel-release": "3.8.0-31-generic",
   "kernel-version": "#46-Ubuntu SMP ...",
   "Distributor ID": "Ubuntu",
   "Description": "Ubuntu 13.04",
   "Release": "13.04",
   "Codename": "raring"
   }
   

   View Slide

10. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 10/18
    L
    C
    A
    2
    0
    1
    4
    ssh -> sshd dependency graph
    

    View Slide

11. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 11/18
    L
    C
    A
    2
    0
    1
    4
    Switch Discovery Data
    from LLDP (or CDP)
    CRM transforms LLDP (CDP) Data to JSON
    

    View Slide

12. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 12/18
    L
    C
    A
    2
    0
    1
    4
    Current Status
    ●
    First release April 2013
    ●
    Great unit tests
    ●
    Nanoprobe code works well
    ●
    Several discovery methods written
    ●
    Discovery => Automatic Monitoring (WOOT!)
    ●
    UI development underway
    ●
    Licensed under GPL: commercial options
    available
    

    View Slide

13. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 13/18
    L
    C
    A
    2
    0
    1
    4
    Get Involved!
    We need every talent!
    ●
    Early adopters (SysAdmins(!))
    ●
    Testers
    ●
    Designers
    ●
    Developers (C,Python, Shell, PowerShell, JavaScript)
    ●
    Porters (esp Windows)
    ●
    Promoters, publicists
    ●
    Packagers
    ●
    And so on...
    

    View Slide

14. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 14/18
    L
    C
    A
    2
    0
    1
    4
    Resistance Is Futile!
    Mailing List bit.ly/AssimML
    #AssimProj @OSSAlanR
    Project Web Site
    assimproj.org
    Blog
    techthoughts.typepad.com
    assimilationsystems.com
    

    View Slide

15. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 15/18
    L
    C
    A
    2
    0
    1
    4
    Why a graph database? (Neo4j)
    ●
    Humans describe systems as graphs
    ●
    Dependency & Discovery information: graph
    ●
    Speed of graph traversals depends on size
    of subgraph, not total graph size
    ●
    Root cause queries  graph traversals –
    notoriously slow in relational databases
    ●
    Visualization is Natural
    ●
    Schema-less design: good for constantly
    changing heterogeneous environment
    ●
    Graph Model === Object Model
    

    View Slide

16. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 16/18
    L
    C
    A
    2
    0
    1
    4
    ssh Client JSON Snippet
    (from netstat and /proc)
    "ssh": {
    "exe": "/usr/sbin/ssh",
    "cmdline": [ "ssh", "servidor" ],
    "uid": "alanr",
    "gid": "alanr",
    "cwd": "/home/alanr/monitor/src",
    "clientaddrs": {
    "10.10.10.5:22": {
    "proto": "tcp",
    "addr": "10.10.10.5",
    "port": 22
    }, and so on...
    

    View Slide

17. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 17/18
    L
    C
    A
    2
    0
    1
    4
    sshd Service JSON Snippet
    (from netstat and /proc)
    "sshd": {
    "exe": "/usr/sbin/sshd",
    "cmdline": [ "/usr/sbin/sshd", "-D" ],
    "uid": "root",
    "gid": "root",
    "cwd": "/",
    "listenaddrs": {
    "0.0.0.0:22": {
    "proto": "tcp",
    "addr": "0.0.0.0",
    "port": 22
    }, and so on...
    

    View Slide

18. linux.conf.au
    06 January
    2014
    LCA SysAdmin Miniconf --- © 2014 Assimilation Systems Limited 18/18
    L
    C
    A
    2
    0
    1
    4
    A multi-dimensional demo
    ●
    Demonstrate basic capabilities
    – Discovery
    – Automatic monitoring configuration
    – Monitoring – failures / successes
    ●
    No configuration was supplied
    – everything comes from discovery
    

    View Slide