CentOS6.6+Puppet3.7.4分布式部署Nagios監控系統

測試框架

CentOS-6.6-x86_64(minimal)
puppet-3.7.4
nagios-4.0.8.tar.gz
nagios-plugins-2.0.3.tar.gz
nrpe-2.15.tar.gz

192.168.188.10 mirrors.redking.com
192.168.188.20 master.redking.com
192.168.188.20 nagios.redking.com 
192.168.188.31 agent1.redking.com
192.168.188.32 agent2.redking.com
192.168.188.33 agent3.redking.com

Puppet 要求所有機器有完整的域名（FQDN），如果沒有 DNS 服務器提供域名的話，可以在兩台機器上設置主機名（注意要先設置主機名再安裝 Puppet，因為安裝 Puppet 時會把主機名寫入證書，客戶端和服務端通信需要這個證書）,因為我配置了DNS，所以就不用改hosts了，如果沒有就需要改hosts文件指定。

1.關閉selinux，iptables，並設置ntp
采用CentOS-6.6-x86_64.iso進行minimal最小化安裝

關閉selinux

[root@master ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@master ~]# sed -i '/SELINUX/ s/enforcing/disabled/g' /etc/selinux/config
[root@master ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection. 
SELINUXTYPE=targeted
[root@master ~]# setenforce 0

停止iptables

[root@node1 ~]# chkconfig --list |grep tables
ip6tables 0:off 1:off 2:on 3:on 4:on 5:on 6:off
iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[root@node1 ~]# chkconfig ip6tables off
[root@node1 ~]# chkconfig iptables off
[root@node1 ~]# service ip6tables stop
ip6tables: Setting chains to policy ACCEPT: filter [ OK ]
ip6tables: Flushing firewall rules: [ OK ]
ip6tables: Unloading modules: [ OK ]
[root@node1 ~]# service iptables stop
iptables: Setting chains to policy ACCEPT: filter [ OK ] 
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ] 

設置ntp

[root@master ~]# ntpdate pool.ntp.org
[root@master ~]# chkconfig --list|grep ntp
ntpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
ntpdate 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@master ~]# chkconfig ntpd on
[root@master ~]# service ntpd start
Starting ntpd: [ OK ] 

2.安裝puppet服務
puppet不在CentOS的基本源中，需要加入 PuppetLabs 提供的官方源：

[root@master ~]# wget http://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm
[root@master ~]# rpm -ivh puppetlabs-release-6-7.noarch.rpm
[root@master ~]# yum -y update

在master上安裝和啟用 puppet 服務：

[root@master ~]# yum install -y puppet-server
[root@master ~]# chkconfig puppet on
[root@master ~]# chkconfig puppetmaster on
[root@master ~]# service puppet start
Starting puppet agent:                                     [  OK  ]
[root@master ~]# service puppetmaster start
Starting puppetmaster:                                     [  OK  ]

在clients上安裝puppet客戶端

[root@agent1 ~]# yum install -y puppet
[root@agent1 ~]# chkconfig puppet on
[root@agent1 ~]# service puppet start

3.配置puppet
對於puppet 客戶端，修改/etc/puppet/puppet.conf，指定master服務器
並重啟puppet服務

[root@agent1 ~]# service puppet restart

4.Client申請證書
服務端自動簽發證書設置
設置master自動簽發所有的證書，我們只需要在/etc/puppet目錄下創建 autosign.conf文件。（不需要修改 /etc/puppet/puppet.conf文件，因為我默認的autosign.conf 文件的位置沒有修改）

[root@master ~]# cat > /etc/puppet/autosign.conf <<EOF
> *.redking.com
> EOF

[root@master ~]# service puppetmaster restart
Stopping puppetmaster:                                     [  OK  ]
Starting puppetmaster:                                     [  OK  ]

這樣就會對所有來自fisteam2.com的機器的請求，都自動簽名。
client需要向服務器端發出請求, 讓服務器對客戶端進行管理. 這其實是一個證書簽發的過程. 第一次運行 puppet 客戶端的時候會生成一個 SSL 證書並指定發給 Puppet 服務端, 服務器端如果同意管理客戶端,就會對這個證書進行簽發，可以用這個命令來簽發證書，由於我們已經在客戶端設置了server地址，因此不需要跟服務端地址

[root@agent1 ~]# puppet agent --test

就可以申請證書了，由於我配置的自動簽發證書，所以直接就簽發了，在服務端執行

[root@master ~]# puppet cert list --all

Nagios服務器安裝
1.安裝Nagios相關依賴包

[root@master ~]# yum install -y httpd php gcc glibc glibc-common gd gd-devel openssl-devel

2.創建Nagios用戶與組

[root@master ~]# useradd -m nagios
[root@master ~]# passwd nagios

創建nagcmd用戶組以執行來自Web接口命令，並添加nagios和apache用戶到此用戶組

[root@master ~]# groupadd nagcmd
[root@master ~]# usermod -a -G nagcmd nagios
[root@master ~]# usermod -a -G nagcmd apache

3.下載Nagios和Plugins軟件包
http://www.nagios.org/download/下載Nagios Core和Nagios Plugins 4.編譯安裝Nagios

[root@master tmp]# tar zxf nagios-4.0.8.tar.gz
[root@master tmp]# cd nagios-4.0.8

#運行Nagios配置腳本，並把nagcmd更改為之前所創建的組

[root@master nagios-4.0.8]# ./configure --with-command-group=nagcmd

#編譯Nagios源碼

[root@master nagios-4.0.8]# make all

#安裝二進制文件、init腳本文件、sample配置文件，設置外部命令目錄權限

[root@master nagios-4.0.8]# make install
[root@master nagios-4.0.8]# make install-init
[root@master nagios-4.0.8]# make install-config
[root@master nagios-4.0.8]# make install-commandmode

5.修改配置文件
樣式配置文件位於/usr/local/nagios/etc目錄，可以更改email地址

[root@master nagios-4.0.8]# vim /usr/local/nagios/etc/objects/contacts.cfg

6.配置Web界面
在Apache的conf.d目錄中安裝Nagios Web配置文件

[root@master nagios-4.0.8]# make install-webconf

創建nagiosadmin帳號登錄Nagios Web接口

[root@master nagios-4.0.8]# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
[root@master nagios-4.0.8]# service httpd start
Starting httpd:                                            [  OK  ]
[root@master nagios-4.0.8]# chkconfig httpd on

開啟httpd服務使配置生效並設置開機自啟 7.編譯安裝Nagios Plugins

[root@master tmp]# tar zxvf nagios-plugins-2.0.3.tar.gz 
[root@master tmp]# cd nagios-plugins-2.0.3
[root@master nagios-plugins-2.0.3]# ./configure --with-nagios-user=nagios --with-nagios-group=nagios
[root@master nagios-plugins-2.0.3]# make && make install

8.編譯安裝Nrpe

[root@master tmp]# tar zxvf nrpe-2.15.tar.gz
[root@master nrpe-2.15]# ./configure
[root@master nrpe-2.15]# make all
[root@master nrpe-2.15]# make install-plugin
[root@master nrpe-2.15]# make install-daemon
[root@master nrpe-2.15]# make install-daemon-config

9.啟動Nagios
本機監控HTTP SSH的Notifications顯示警告錯誤，解決方法

[root@master ~]# vim /usr/local/nagios/etc/objects/localhost.cfg
# Define a service to check SSH on the local machine.
# Disable notifications for this service by default, as not all users may have SSH enabled.
define service{
use                             local-service         ; Name of service template to use
host_name                       localhost
service_description             SSH
check_command                   check_ssh
notifications_enabled           1  #改為1，即可
}
# Define a service to check HTTP on the local machine.
# Disable notifications for this service by default, as not all users may have HTTP enabled.
define service{
use                             local-service         ; Name of service template to use
host_name                       localhost
service_description             HTTP
check_command                   check_http
notifications_enabled           1  #改為1，即可 
}
[root@master ~]# touch /var/www/html/index.html

啟動Nagios之前測試配置文件

[root@master ~]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

啟動Nagios、nrpe並設置開機自啟

[root@master ~]# chkconfig nagios --add
[root@master ~]# chkconfig --list |grep nagios
nagios          0:off   1:off   2:off   3:on    4:on    5:on    6:off
[root@master ~]# chkconfig nagios on
[root@master ~]# service nagios start
Starting nagios: done.
[root@master ~]# echo "/usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d" >> /etc/rc.d/rc.local
[root@master nrpe-2.15]# /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d
[root@master nrpe-2.15]# netstat -tunpl |grep nrpe
tcp 0 0 0.0.0.0:5666 0.0.0.0:* LISTEN 70100/nrpe
tcp 0 0 :::5666 :::* LISTEN 70100/nrpe

執行/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1檢查連接是否正常 使用之前定義的nagiosadmin帳號與密碼登錄Nagios,地址：http://192.168.188.20/nagios/

創建Nagios客戶端監控
1.Puppet Master安裝相應模塊
Nagios沒有目前沒有提供官方軟件源，在批量部署時可以使用第三方epel源，采用Example42所提供的puppet-nrpe來實現Linux服務器批量部署。部署客戶端使用官方3個模塊：epel、nrpe、puppi。

epel模塊用於安裝nrpe軟件，nrpe模塊用於收集主機信息，puppi屬於Example42模塊組件，使用Example42模塊時都需要加載此模塊。

Puppi是一個Puppet模塊和CLI命令，他可以標准化和自動化快速部署應用程序，並提供快速和標准查詢命令，檢查系統資源。

[root@master ~]# git clone https://github.com/puppetlabs/puppetlabs-stdlib /etc/puppet/modules/stdlib
[root@master ~]# git clone https://github.com/example42/puppi /etc/puppet/modules/puppi
[root@master ~]# git clone https://github.com/example42/puppet-nrpe /etc/puppet/modules/nrpe
[root@master ~]# puppet module install stahnma/epel
[root@master ~]# vim /etc/puppet/puppet.conf
[master]
modulepath = /etc/puppet/modules/

2.創建agent節點組配置文件

[root@master ~]# mkdir /etc/puppet/manifests/nodes
[root@master ~]# vim /etc/puppet/manifests/nodes/agentgroup.pp
node /^agent\d+\.redking\.com$/ {
include stdlib
include epel
class { 'puppi': }
class { 'nrpe':
require => Class['epel'],
allowed_hosts => ['127.0.0.1',$::ipaddress,'192.168.188.20'],
template => 'nrpe/nrpe.cfg.erb',
}
}
[root@master ~]# vim /etc/puppet/manifests/site.pp
import "nodes/agentgroup.pp"

3.配置Nagios添加agent.redking.com主機監控
修改/usr/local/nagios/etc/objects/commands.cfg
command_name check_nrpe ——定義命令名稱為check_nrpe，services.cfg必須使用
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ ——用$USER1$代替/usr/local/nagios/libexec

這是定義實際運行的插件程序.這個命令行的書寫要完全按照check_nrpe這個命令的用法.不知道用法的就用check_nrpe –h查看； -c後面帶的$ARG1$參數是傳給nrpe daemon執行的檢測命令,它必須是nrpe.cfg中所定義的5條命令中的其中一條。

[root@master ~]# vim /usr/local/nagios/etc/objects/commands.cfg
# 'check_nrpe' command definition
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

修改/usr/local/nagios/etc/nagios.cfg

[root@master ~]# vim /usr/local/nagios/etc/nagios.cfg
cfg_file=/usr/local/nagios/etc/objects/agent1.redking.com.cfg
cfg_file=/usr/local/nagios/etc/objects/agnet2.redking.com.cfg
cfg_file=/usr/local/nagios/etc/objects/agent3.redking.com.cfg

增加agent1~3.redking.com.cfg配置文件

# vim /usr/local/nagios/etc/objects/agent1.redking.com.cfg
define host{
use             linux-server
host_name       agent1.redking.com
alias agent1.redking.com
address         192.168.188.31
}
 
define service{
use                     generic-service
host_name               agent1.redking.com
service_description     PING
check_command           check_ping!100.0,20%!500.0,60%
}

define service{
use                     generic-service
host_name               agent1.redking.com
service_description     Current Users
check_command           check_nrpe!check_users!10!5
}
 
define service{
use                     generic-service
host_name               agent1.redking.com
service_description     Current Load
check_command           check_nrpe!check_load!15,10,5!30,25,20
}
 
define service{
use                     generic-service
host_name               agent1.redking.com
service_description     Swap Usage
check_command           check_nrpe!check_swap!20!40
}

檢測Nagios服務並重啟使配置生效

[root@master ~]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg 
[root@master ~]# service nagios restart
[root@master ~]# service puppetmaster restart

客戶端測試

[root@agent1 ~]# puppet agent --test

客戶端自動部署nrpe 下面我們來看下客戶端自動化部署nrpe後采集信息的nagios監控界面





NRPE模塊中定義的nrpe.cfg包含大量腳本，我們可以直接拿來使用當然也可以自己修改nrpe.cfg.erb模板內容。在批量部署時可以分別采用自己編寫的模塊或者現有模塊來實現，利用現有模塊幾乎能實現系統管理日常工作中90%任務，剩余的10%我們可以根據生產業務來自己定制。
原文：http://redking.blog.51cto.com/27212/1612136