Tailored SNMP monitoring: Your own SNMP MIB and sub-agent with Python and python-netsnmpagent

Tailored SNMP monitoring: Your own SNMP MIB and sub-agent with Python and python-netsnmpagent

SNMP continues to be a essential component in monitoring where the information being made available is structured in so-called Management Information (MIB) modules. The standard net-snmp distribution comes a with a variety of standard MIBs implemented by its snmpd, but sometimes there is the need to make your own information available via SNMP.

Luckily snmpd can be dynamically extended by so-called subagents implementing the AgentX protocol (RFC2747). The net-snmp API however pretty much focusses on the C programming language only, laying the entrance barrier especially for non-developers rather high.

In this talk Pieter will not only demonstrate the creation of a MIB, which, being a text file, is the easier part, but also how easy it is to implement a simple subagent in Python using his python-netsnmpagent module. python-netsnmpagent is a shim wrapper over the net-snmp C API trying to implement just enough abstraction. Licensed under the GPL it is available at https://github.com/pief/python-netsnmpagent as well as PyPI.

93a37bec61dd875257921cea9cebad4b?s=128

Pieter Hollants

November 06, 2018
Tweet

Transcript

  1. 1.

    Tailored SNMP monitoring Your own SNMP MIB and sub-agent with

    Python and python-netsnmpagent OSMC 2018 November 6th, 2018 Pieter Hollants
  2. 2.

    Pieter who? Frankfurt, Germany-based developer (mostly Python) but also... ➔

    3y IT support in Netware/Win 95 times ➔ 9y (Senior) Intern at SUSE Consulting ➔ 4y Linux Systems Engineer at German Air Traffic Control (e.g. automated installations of high availability-systems, hardware standardization) ...and freelancing since over 15y (dev & admin)
  3. 3.
  4. 5.

    MIB ➔ Management Information Base ➔ Used to monitor and

    configure devices remotely ➔ Structured as tree with nameless root ➔ Defines entities called Managed Objects ➔ Access protocol “on the wire”: SNMP (Simple Network Management Protocol)
  5. 6.

    MIB modules ➔ MIB modules (“MIBs”) define branches of the

    tree, can be enterprise-specific or generic, e.g.: ➔ RFC 2863: IF-MIB (network interfaces) ➔ RFC 4022: TCP/IP MIB ➔ CISCOSB-MIB ➔ Defined in text files using ASN.1 subset called SMI (Structure of Management Information), v2: RFC2578 ➔ Definition != Implementation Wikipedia / CC-BY-SA
  6. 7.

    Managed Objects ➔ Uniquely identified by OID (Object Identifier) ➔

    OIDs represented as ordered series of numbers (e.g. “.1.3.6.1.4.1”) or ASCII text labels (e.g. “.iso.org.dod. internet.private.enterprise”) ➔ Enterprise-specific OIDs with IANA assigned Enterprise Number ➔ https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers ➔ Two categories of data types: ➔ Scalar objects: ➔ (Un)Signed Integers ➔ Counters, Gauges, Time ticks ➔ Octet Strings/Display Strings, Bit Strings ➔ Network addresses ➔ Object identifiers ➔ Tabular objects (scalar objects grouped in tables)
  7. 8.

    Who’d write their own MIB? ➔ Application programmers wanting to

    expose internal metrics (e.g. transactions per second) ➔ OEMs (e.g. Nasuni Edge Appliance: https://cdn2.hubspot.net/ hubfs/3316942/Nasuni.com-assets/Support-Docs/Nasuni_Filer_Third- Party_Licensing_Guide.pdf) ➔ End users wishing to unify otherwise spread-out information under a common tree (e.g. vendor-neutral hardware agent used by German Air Traffic Control) ➔ InfraOps (e.g. DNSSEC monitoring: http://www.delaat.net/rp/2014-2015/p38/presentation.pdf) ➔ You? ;)
  8. 10.

    MIB deep dive (5m) ➔ Formally defines “<MIBNAME>“ as everything

    between “BEGIN” and “END” ➔ Multiple MIB modules in a text file possible but uncommon ➔ Two dashes (“--”): comment lines
  9. 11.

    MIB deep dive (10m) ➔ Imports definitions (”features”) from existing

    MIBs, similar to “from … import …” in Python ➔ RFC 2579: SNMPv2-TC, RFC 2580: SNMPv2-CONF ➔ AgentX? We’ll see later...
  10. 12.

    MIB deep dive (15m) ➔ Collects module purpose, contact information,

    description, revision information ➔ Defined as OID of subtree this MIB handles (“plugs in”) ➔ Must be first item after module header and imports Wrong OID! Why?
  11. 13.

    MIB deep dive (20m) ➔ Associates a name with subtrees

    in a particular part of the MIB module’s OID tree, similar to naming subdirs ➔ For human convenience, technically not really required ➔ len(OID subtree def) == 2 ➔ “[..] must not be included in an IMPORTS statement”
  12. 14.

    MIB deep dive (25m) ➔ Defines scalar variables in a

    particular part of the MIB module’s OID tree, similar to naming files ➔ SYNTAX: type of the variable ➔ MAX-ACCESS: read-only/read-write/not-accessible/... ➔ STATUS: current/deprecated/obsolete ➔ DESCRIPTION: for humans and humanoids
  13. 15.

    MIB deep dive (30m) ➔ Defines a “row” as a

    SEQUENCE of scalar MOs ➔ Simulation of table through N subtrees ~= columns as defined here (holes possible!) ➔ MOs referenced with name and type only ➔ Still need explicit definition...
  14. 16.

    MIB deep dive (32m) ➔ Definitions of columns in a

    row ➔ MAX-ACCESS not-accessible in practical use for row that will be used as index row (here the only one) ➔ In this example other rows will be read-only
  15. 17.

    MIB deep dive (34m) ➔ Table ~= SEQUENCE OF (SEQUENCE

    ...) ➔ Both table and row themselves not-accessible ➔ INDEX magically overwrites 1 with values in referenced index column (values = indexes)
  16. 18.

    MIB deep dive (36m) ➔ Table object not-accessible → snmpget

    = no such object ➔ Same for defined row, e.g. firstTable.1 = no such object ➔ snmpwalk reveals low-level .<table>.<column>.<row> construct ➔ snmptable knows how to interpret tables semantics ➔ firstTableRowIndex values = indexes
  17. 19.

    MIB deep dive (40m) ➔ Always check your MIBs with

    “smilint” ➔ Part of libsmi, other nice tools such as “smidiff”, “smidump”… ➔ http://www.ibr.cs.tu-bs.de/projects/libsmi/ ➔ Also validate with increased “-lX” severity levels, then learn about MIB concepts you haven’t heard about yet (e.g. conformance groups)
  18. 23.
  19. 24.

    Our GT-MIB (1/2) ...we should actually register our own Enterprise

    Number with IANA! http://pen.iana.org/pen/PenApplication.page
  20. 26.

    Implementing SNMP ➔ De-facto standard: Net-SNMP suite (formerly called UCD-SNMP),

    found in all major distros ➔ Current release: 5.8 (July 16th, 2018) ➔ Consists of ➔ Command-line applications such as snmpget, snmpset, snmpwalk, snmptable, snmptranslate... ➔ Extensible agent (software that handles SNMP requests) with built-in support for wide range of MIB modules: snmpd ➔ Daemon for receiving SNMP traps: snmptrapd ➔ Libraries for developing SNMP apps, C/Perl APIs ➔ Tools such as mib2c
  21. 27.

    Implementing MIBs ➔ snmpd: built-in support for number of MIBs,

    e.g. IF-MIB ➔ Extensible: snmpd as master agent allows for subagents implementing parts of the MIB (MIB modules) ➔ Three alternatives differing in data representation and communication mechanisms: ➔ Proxied SNMP: standard SNMP packet format, full fledged agent unaware of subagent role on non-standard port ➔ SMUX (RFC 1227): standard SNMP packet format, subagent registers with master agent, issues, historical ➔ AgentX (RFC 2741): more compact packet format, advanced functionality, state of the art, enabled with “master agentx” line in snmpd.conf
  22. 28.

    Net-SNMP mib2c (1/2) mib2c currently can’t handle MIBs with both

    tables and scalars Different approaches to implement tables possible Input: MIB Output: Skeleton C code
  23. 32.

    Python & SNMP support ➔ Net-SNMP ships with Python “netsnmp”

    module ➔ 2500 lines C code that abstract Net-SNMP C API ➔ SNMP client only: get, walk… operations ➔ C API has support for writing agents as well (as seen with mib2c) ➔ Idea: access C API directly from Python using ctypes, imitating agents written in C ➔ Existing python-agentx module on Sourceforge ➔ Design issues (= I didn’t grok it ;) ➔ Orphaned ➔ 2015: pyagentx module on Github, actually implements AgentX network protocol, meanwhile orphaned as well
  24. 33.

    Hello python-netsnmpagent! ➔ Python module I wrote back in 2013

    driven by requirements at German Air Traffic Control ➔ LGPLv3 licensed ➔ Source at https://github.com/pief/python-netsnmpagent ➔ Distributable archives on PyPI ➔ RPMs available on Open Build Service ➔ Compatible with Python 2.6, 2.7 and >=3.5 ➔ Tested with net-snmp 5.4.x (SLES11…), 5.7.x, 5.8 ➔ Two extensively commented files ➔ netsnmpapi.py (ctypes stuff for internal use), 317 LOC ➔ netsnmpagent.py (abstraction classes), 839 LOC ➔ No docs but example MIBs/agents included
  25. 34.

    Our gtmib_agent.py (1/3) Used by Net-SNMP e.g. to translate OIDs

    Default value for column Classes representing data types Only import required Column number Class factory OID this SNMP object handles
  26. 35.

    Our gtmib_agent.py (2/3) Values for index and data columns Connects

    to master agent – no more object registrations!
  27. 36.

    Our gtmib_agent.py (3/3) Signal handler triggered eg. by ^C Net-SNMP

    internal packet processing Example in this form not well-suited for real agents. Why?
  28. 38.

    The fineprint ➔ Currently no support for callback pattern, i.e.

    data structures must be periodically refreshed ➔ check_and_process() blocks, threading required to do processing and data updating in parallel ➔ Development has slowed somewhat because no active sponsor anymore → your chance? ➔ Some open issues ➔ No support for SNMP traps yet (but MR exists) ➔ Only rudimentary unit tests ➔ It’s Open Source, take your chance!