之前写过一个帖子,是如何完美监控ESXi节点的硬盘的,没想到效果那么好。最近好多朋友过来问面包肿么配置。说实话我也是太懒了,所以就把这个脚本优化了一下,一不小心没有刹住车,把CPU、内存、硬盘、网卡流量什么的都监控出来了,哦,对了,还有虚拟机在节点上占用的空间(主要是因为我使用的大多是本地的空间,然后没有一个完美的曲线展示是节点上哪个虚拟机增长量过快导致的本地空间耗费,对运维是有很大隐患的)。然后本来前天v2.0版都已经写好了,上线了!但是发现了一些性能上面的问题,最后没有发布。经过两天的奋笔疾书,终于最新版的v2.0诞生了,监控脚本内容介绍:
config.ini : 配置文件
cpu_mem_info.py : 取CPU值的脚本
get_disk_info.py : 取硬盘相关信息的脚本
main.py : 主的运行文件
traffic.py : 网络流量监控文件
vm_info.py : 虚拟机在节点占用空间获取脚本
zabbix_send.py : 向zabbix send数据的脚本
OK,介绍过脚本作用之后,开始来说一下肿么配置吧!主要是修改"config.ini":
上面是固定的格式:
[VCenter01] ------> 主要是VC应用的别名,用于日志上面报错的时候好识别
host:VC的IP地址,格式:10.10.10.10
user : VC的用户名,格式:mianbao
password:VC用户的密码,格式:mianbao.cn.com
注:上面的host、user、password这些项名是不能修改的!
你有几个VC就写几个上面的结构就行,zabbix我就不再多说了,切记 '[zabbix]'是不能修改的哦!
然后就是准备环境了,所需环境 Centos 或 Window都可以,这里只说一下Centos的环境准备方法,只需安装所需要的python以来模块,安装方法如下:
unzip pysphere-0.1.8.zipcd pysphere-0.1.8python setup.py install
OK,安装过程是不是简单的有点让人恐怖,接下来验证一下,所安装的环境是否OK:
如果没有报错,OK那就是安装完成了,然后把脚本下载之后放到一个目录里面,运行如下命令就可以收集监控项了:
python main.py
But,在爽之前别忘记了在zabbix上面还需要做相关的配置,主要分为两大块:
1.模板
2.host
1.模板已为大家准备好了,请下载导入即可:
zabbix 2.0 模板:
2.host大家可以通过两种方式添加,一种是自己手动添加,另外一种就是discovery。不会配置的小朋友们要认真做功课了~
一切配置停当,并且有数值了,那么在linux的crontab中添加如下定时任务即可:
*/5 * * * * /usr/bin/python /vmware/main.py
温馨提示:
初次使用脚本的友友们,请运行两次之后再去看数据!因为zabbix第一次会根据send过去的key去建立item,所以第一次会有一半是失败的,第二次就OK了!
附件上传不上来,我直接把代码公布出来吧!
Config.ini
- [VCenter01]
- host = VC IP地址(ex:10.10.10.10)
- user = VC 用户名
- password = VC 用户密码
-
- [VCenter02]
- host = VC IP地址(ex:10.10.10.11)
- user = VC 用户名
- password = VC 用户密码
-
- [VCenter03]
- host = VC IP地址(ex:10.10.10.12)
- user = VC 用户名
- password = VC 用户密码
-
- [zabbix]
- host = Zabbix服务器地址
- port = 10051
cpu_mem_info.py
- #-*- coding:utf-8 -*-
- '''
- @Created on 2016年6月14日
-
- @author: MianBao
- @author_web: Mianbao.cn.com
- '''
- import copy
- from pysphere import VIServer,VIProperty
- from zabbix_send import zabbix_sender
-
-
-
- def getNodeinfo(s,zabbix_info):
- properties = [
- 'summary.config.name',
- 'summary.quickStats.overallCpuUsage',
- 'summary.quickStats.overallMemoryUsage',
- 'summary.hardware.cpuMhz',
- 'summary.hardware.memorySize',
- 'summary.hardware.numCpuCores',
- 'summary.hardware.numCpuPkgs',
- ]
-
- props = s._retrieve_properties_traversal(property_names=properties,obj_type="HostSystem")
- total = {}
- for item in props:
- host = dict()
- for p in item.PropSet:
- if p.Name == 'summary.config.name':
- host_name = p.Val
- if p.Name == 'summary.quickStats.overallCpuUsage':
- cpu_use = p.Val
- if p.Name == 'summary.quickStats.overallMemoryUsage':
- memroy_use = p.Val
- if p.Name == 'summary.hardware.cpuMhz':
- cpu_hz = p.Val
- if p.Name == 'summary.hardware.memorySize':
- memory_total = p.Val
- if p.Name == 'summary.hardware.numCpuCores':
- cpu_core = p.Val
- if p.Name == 'summary.hardware.numCpuPkgs':
- cpu_num = p.Val
-
- host['cpu_use'] = cpu_use
- host['memory_use'] = memroy_use
- host['cpu_hz'] = cpu_hz * cpu_core
- host['memory_total'] = memory_total
- host['cpu_core'] = cpu_core
- host['cpu_num'] = cpu_num
- total[host_name] = host
- host = dict()
- #print total
- SendDataToZabbix(total,zabbix_info)
-
- def SendDataToZabbix(total,zabbix_info):
- for x,y in total.items():
- value = dict()
- value['host'] = x
-
- zabbix=zabbix_sender(**zabbix_info)
-
- value['key'] = 'cpu.info'
- value['value'] = '{"data":[{"{#DATA}":"CPU"}]}'
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'cpu.use.hz[CPU]'
- value['value'] = y.get('cpu_use')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'cpu.physical.core[CPU]'
- value['value'] = y.get('cpu_core')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'cpu.total.hz[CPU]'
- value['value'] = y.get('cpu_hz')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'cpu.chacao.num[CPU]'
- value['value'] = y.get('cpu_num')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'cpu.vcpu.num[CPU]'
- value['value'] = y.get('cpu_num') * y.get('cpu_core')
- zabbix.adddata(**copy.deepcopy(value))
-
- response=zabbix.send()
- #print 'memory----',response
-
- zabbix_memory=zabbix_sender(**zabbix_info)
-
- value['key'] = 'memory.info'
- value['value'] = '{"data":[{"{#DATA}":"MEMORY"}]}'
- zabbix_memory.adddata(**copy.deepcopy(value))
-
- value['key'] = 'memory.use.mb[MEMORY]'
- value['value'] = y.get('memory_use') * 1024 * 1024
- zabbix_memory.adddata(**copy.deepcopy(value))
-
- value['key'] = 'memory.total.bytes[MEMORY]'
- value['value'] = y.get('memory_total')
- zabbix_memory.adddata(**copy.deepcopy(value))
-
- response=zabbix_memory.send()
- #print 'cpu----',response
- return response
get_disk_info.py:
- #-*- coding:utf-8 -*-
- '''
- @Created on 2016年6月14日
-
- @author: MianBao
- @author_web: Mianbao.cn.com
- '''
- import os
- import ssl
- import copy
- import json
- from pysphere import VIServer,VIProperty
- from zabbix_send import zabbix_sender
-
- import time
-
-
- def HostSystem(s):
- properties = [
- 'summary.config.name',
- 'datastore',
- ]
- props = s._retrieve_properties_traversal(property_names=properties,obj_type="HostSystem")
- host = dict()
- for item in props:
- for p in item.PropSet:
- if p.Name == 'datastore':
- datastore_id = p.Val.ManagedObjectReference
- if p.Name == 'summary.config.name':
- host_ip = p.Val
- for x in datastore_id:
- host[str(x)] = host_ip
- return host
-
- def DatatStore(s):
- properties = [
- 'summary.capacity',
- 'summary.freeSpace',
- 'summary.name',
- 'summary.uncommitted',
- ]
- props = s._retrieve_properties_traversal(property_names=properties,obj_type="Datastore")
- host = dict()
- for item in props:
- disk = dict()
- for p in item.PropSet:
- if p.Name == 'summary.capacity':
- total = p.Val
- if p.Name == 'summary.freeSpace':
- free = p.Val
- if p.Name == 'summary.name':
- name = p.Val
- if p.Name == 'summary.uncommitted':
- uncommitted = p.Val
- disk['total'] = total
- disk['free'] = free
- disk['name'] = name
- #disk['uncommitted'] = uncommitted
- host[name] = disk
- return host
-
- def Get_Datastore(s):
- return s.get_datastores()
-
- def MergeAndSend(s,zabbixs):
- host = HostSystem(s)
- disk = DatatStore(s)
- middle = Get_Datastore(s)
- hosts = dict()
- for M,N in host.items():
- X = middle.get(M)
- Y = disk.get(X)
- zabbix_send_content(N,Y,zabbixs)
- return hosts
-
- def zabbix_send_content(host,cont_dict,zabbixs):
- if isinstance(cont_dict, dict):
- zabbix=zabbix_sender(**zabbixs)
-
- #the discovery rules
- value = dict()
- value['host'] = host
- value['key'] = 'vmware.disk.name'
- value['value'] = '{"data":[{"{#DATA}":"%s"}]}' % cont_dict.get('name')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'vmware.disk.free[%s]' % cont_dict.get('name')
- value['value'] = '%s' % cont_dict.get('free')
- zabbix.adddata(**copy.deepcopy(value))
-
- value['key'] = 'vmware.disk.total[%s]' % cont_dict.get('name')
- value['value'] = '%s' % cont_dict.get('total')
- zabbix.adddata(**copy.deepcopy(value))
-
- response=zabbix.send()
- #print 'disk----',response
- return response
-
-
- if '__main__' == __name__:
- pass
main.py:
- #-*- coding:utf-8 -*-
- '''
- @Created on 2016.11.14
-
- @author: MianBao
- @author_web: Mianbao.cn.com
- '''
-
- import os
- import ssl
- import logging
- import ConfigParser
- import threading
- import time
-
- from pysphere import VIServer
-
- from get_disk_info import MergeAndSend
- from cpu_mem_info import getNodeinfo
- from vm_info import main
- from traffic import NetTraffic
-
-
-
- class MianBao():
-
- def __init__(self):
- self.now_path = os.path.dirname(__file__)
- self.vcinfo = None
-
- def GetConfig(self):
- config = dict()
- config_items = ['host', 'user', 'password']
- config_zabbix_items = ['host', 'port']
-
- file = os.path.join(self.now_path,'config.ini')
-
- Cfile = ConfigParser.ConfigParser()
- Cfile.read(file)
- Cfile_zone = Cfile.sections()
-
- for C in Cfile_zone:
- zone_config = dict()
- config_items = config_zabbix_items if C == 'zabbix' else ['host', 'user', 'password']
- try:
- for Y in config_items:
- zone_config[Y] = Cfile.get(C,Y)
- except Exception,e:
- self.logs('config read',e.message)
- pass
- config[C] = zone_config
- self.vcinfo=config
-
- def logs(self,where,log):
- log_name = os.path.join(self.now_path,'main.log')
- logger = logging.getLogger(where)
- logger.setLevel(logging.DEBUG)
- fh = logging.FileHandler(log_name)
- fh.setLevel(logging.DEBUG)
- formatter = logging.Formatter('%(asctime)s - %(filename)s:%(lineno)s - %(name)s - %(message)s' )
- fh.setFormatter(formatter)
- logger.addHandler(fh)
- logger.debug(log)
-
- def Connect(self,zone,vc):
- # windows need import the ssl
- try:
- ssl._create_default_https_context = ssl._create_unverified_context
- except Exception,e:
- pass
-
- self.Server = VIServer()
- try:
- self.Server.connect(**vc)
- except Exception,e:
- self.logs('%s VC 连接错误' % zone,e.message)
- return self.Server if self.Server.is_connected() else 1
-
-
- def Run(self):
- self.GetConfig()
- if self.vcinfo is not None:
- zabbix = self.vcinfo.get('zabbix',None)
- for zone,vc in self.vcinfo.items():
- if zone != 'zabbix' and zabbix != None:
- S = self.Connect(zone,vc)
- try:
- MergeAndSend(S,zabbix)
- getNodeinfo(S,zabbix)
- main(S,zabbix)
- NetTraffic(S,zabbix)
- S.disconnect()
- except Exception,e:
- self.logs('%s VC 连接错误' % zone,e.message)
- pass
-
-
-
-
- if '__main__' == __name__:
- collor = MianBao()
- collor.Run()
traffic.py:
- #-*- coding:utf-8 -*-
- '''
- @Created on 2016.11.14
-
- @author: MianBao
- @author_web: Mianbao.cn.com
- '''
- import copy
- from pysphere import VIServer,VIProperty
- from zabbix_send import zabbix_sender
- import time
-
-
- def NetTraffic(s,zabbix_info):
-
- pm = s.get_performance_manager()
- get_key = ['net.bytesTx', 'net.bytesRx']
-
- for x,y in s.get_hosts().items():
-
- network_type = dict()
-
- for M in get_key:
- network_traffic = dict()
- start = time.clock()
- nettx_out = pm.get_entity_statistic(x,M)
- end = time.clock()
- for nettxtraffic in nettx_out:
- nettxinstance = dict((name, getattr(nettxtraffic, name)) for name in dir(nettxtraffic) if not name.startswith('__'))
- if nettxinstance['instance'] != "":
- netname = nettxinstance['instance']
- nettxtraffic = nettxinstance['value']
- network_traffic[netname] = nettxtraffic
- network_type[M] = network_traffic
-
- #print "traffic use time:",end - start
- MergeNetName(network_type,y,zabbix_info)
-
-
- def MergeNetName(network_type,y,zabbix_info):
- if isinstance(network_type,dict):
- for m,n in network_type.get('net.bytesTx').items():
- network_in = network_type.get('net.bytesRx').get(m)
- UpToZabbix(y,m,n,network_in,zabbix_info)
-
- def UpToZabbix(y,net_name,out,net_in,zabbix_info):
- zabbix_traffic=zabbix_sender(**zabbix_info)
- value = dict()
- value['host'] = y
-
- value['key'] = 'traffic.info'
- value['value'] = '{"data":[{"{#DATA}":"%s"}]}' % net_name
- zabbix_traffic.adddata(**copy.deepcopy(value))
-
- value['key'] = 'traffic.net.out.kb[%s]' % net_name
- value['value'] = out
- zabbix_traffic.adddata(**copy.deepcopy(value))
-
- value['key'] = 'traffic.net.in.kb[%s]' % net_name
- value['value'] = net_in
- zabbix_traffic.adddata(**copy.deepcopy(value))
-
- response=zabbix_traffic.send()
- #print 'traffic----',response
vm_info.py:
- #-*- coding:utf-8 -*-
- '''
- @Created on 2016614
-
- @author: MianBao
- @author_web: Mianbao.cn.com
- '''
- import copy
- import ssl
- from pysphere import VIServer,VIProperty
- from zabbix_send import zabbix_sender
-
- import time
-
-
- def GetNodeInfo(Server):
- managed_object_types = 'VirtualMachine'
- properties = [
- 'summary.runtime.host',
- 'summary.storage.committed',
- 'name',
- ]
- props = Server._retrieve_properties_traversal(property_names=properties,obj_type=managed_object_types)
- vms = dict()
- for item in props:
- vm = {}
- host_id = None
- for p in item.PropSet:
- if p.Name == 'summary.runtime.host':
- if vms.get(p.Val,None):
- host_id = p.Val
- vm[p.Name] = p.Val
- if host_id:
- vms[host_id].append(vm)
- else:
- vms[vm['summary.runtime.host']] = [vm,]
- return vms
-
- def GetHosts(s):
- hosts = s.get_hosts()
- return hosts
-
- def main(s,zabbix_info):
- start = time.clock()
- hosts = GetHosts(s)
- vms = GetNodeInfo(s)
- for x,y in vms.items():
- for m in y:
- response = SendData(x,m,hosts,zabbix_info)
- end = time.clock()
- #print 'vm use time:',end - start
- return response
-
- def SendData(x,m,hosts,zabbix_info):
- zabbix_vm = zabbix_sender(**zabbix_info)
-
- value = dict()
- value['host'] = hosts.get(x,None)
- value['key'] = 'vmware.vm'
- vm_name = m.get('name')
- value['value'] = '{"data":[{"{#DATA}":"%s"}]}' % str(vm_name)
- zabbix_vm.adddata(**copy.deepcopy(value))
- for key,val in m.items():
- if key not in ['summary.runtime.host','name',]:
- value['key'] = '%s[%s]' % (key, vm_name)
- value['value'] = val
- zabbix_vm.adddata(**copy.deepcopy(value))
-
- response=zabbix_vm.send()
- return response