Slide 1

Slide 1 text

agri inventory

Slide 2

Slide 2 text

agri inventory nouka data collector naya data store yaoya data convertor nouka naya yaoya output push crawl

Slide 3

Slide 3 text

yaoya output example(1) ¤ $ bin/yaoya --group system_A --name command_uname {u'return_code': u'0', u'command_line': u'uname -a', u'execute_at': u'1339942622', u'group_name': u'system_A', u'visible': u'True', u'host_name': u'bb-mba.local', u'time': datetime.datetime(2012, 6, 17, 14, 17, 3), u'output': u'Darwin bb-mba.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386¥n', u'_id': ObjectId('4fdde6e0c286630ca0000009'), u'command_name': u'command_uname'} {u'return_code': u'0', u'command_line': u'uname -a', u'execute_at': u'1339943547', u'group_name': u'system_A', u'visible': u'True', u'host_name': u'bbtest-c6', u'time': datetime.datetime(2012, 6, 17, 14, 32, 28), u'output': u'Linux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64 GNU/Linux¥n', u'_id': ObjectId('4fddea7dc286630ca000003c'), u'command_name': u'command_uname'}

Slide 4

Slide 4 text

yaoya output example(2) ¤ $ bin/csv --group system_A --name command_uname "bb-mba.local","Darwin bb-mba.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu- 1504.15.3~1/RELEASE_I386 i386" "bbtest-c6","Linux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64 GNU/Linux"

Slide 5

Slide 5 text

default commands ¤ command_ifconfig = ifconfig -a ¤ command_hostname = hostname ¤ command_uname = uname -a ¤ command_df = df -akl ¤ command_iptables = /sbin/iptables-save ¤ command_rpm = rpm -qa ¤ command_chkconfig = /sbin/chkconfig --list ¤ command_ip_addr = /sbin/ip addr show ¤ command_ip_route = /sbin/ip route show ¤ command_dmidecode = /usr/sbin/dmidecode ¤ command_resolv = cat /etc/resolv.conf ¤ command_passwd = cat /etc/passwd ¤ command_group = cat /etc/group ¤ command_shadow = cat /etc/shadow ¤ command_proc_cpuinfo = cat /proc/cpuinfo ¤ command_proc_meminfo = cat /proc/meminfo ¤ command_proc_mounts = cat /proc/mounts To add command, just only define in nouka.conf!

Slide 6

Slide 6 text

components

Slide 7

Slide 7 text

nouka data collector ¤ pure python 2.4 ¤ run with python standard library ¤ POST result to naya with HTTP

Slide 8

Slide 8 text

naya data store ¤ fluentd / td-agent ¤ fluent-plugin-http-enhanced ¤ https://github.com/parolkar/fluent-plugin-http-enhanced ¤ fluent-plugin-mongo ¤ mongodb

Slide 9

Slide 9 text

data format { *** are same value in same execution ‘group_name’ : ‘‘, *** 'host_name' : '', *** 'command_name' : '', 'command_line' : '', 'output' : '', 'return_code' : '', ‘visible’ : ‘True’, *** 'execute_at' : '’, *** }

Slide 10

Slide 10 text

getting started

Slide 11

Slide 11 text

project repository ¤ nouka ¤ https://bitbucket.org/netmarkjp/nouka ¤ yaoya ¤ https://bitbucket.org/netmarkjp/yaoya

Slide 12

Slide 12 text

setup(1/2) ¤ td-agent ¤ cat </etc/yum.repos.d/td.repo [treasuredata] name=TreasureData baseurl=http://packages.treasure- data.com/redhat/¥$basearch gpgcheck=0 EOT ¤ yum -y install td-agent ¤ /usr/lib64/fluent/ruby/bin/gem install bson_ext ¤ /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-http- enhanced

Slide 13

Slide 13 text

setup(2/2) ¤ mongod ¤ cat </etc/yum.repos.d/10gen.repo [10gen] name=10gen Repository baseurl=http://downloads- distro.mongodb.org/repo/redhat/os/x86_64 gpgcheck=0 EOT ¤ yum -y install mongo-10gen.x86_64 mongo-10gen-server.x86_64 ¤ mercurial ¤ yum -y install python-devel python-setuptools gcc ¤ pip install mercurial

Slide 14

Slide 14 text

enjoy development!