前言
LVS大家應該很熟悉,這款優秀的開源軟件基本成為了IP負載均衡的代言詞。但在實際的生產環境中會發現,LVS調度在大壓力下很容易就產生瓶頸, 其中瓶頸包括ipvs內核模塊的限制,CPU軟中斷,網卡性能等,當然這些都是可以調優的,關於LVS的調優,會在這裡詳細講 LVS調優攻略 。回到主題,那當無法避免的單台LVS調度機出現了性能瓶頸,有什麼辦法呢?在本文就來介紹如何橫向擴展LVS調度機
架構簡圖
如上圖三層設備的路由表,VIP地址1.1.1.1對應nexthop有三個地址,這三個地址是三台lvs調度機的地址。這樣便可達到效果:用戶訪問------>VIP------>三台LVS調度機------>分發到多台RealServe
架構優勢
1.LVS調度機自由伸縮,橫向擴展(最大8台,受限於三層設備允許的等價路由數目)
2.LVS調度資源全利用,All Active。不存在備份機
部署方法
1.硬件資源准備
三層設備: 本文用的是h3c 5800三層交換機
LVS調度機三台: 192.168.0.2 192.168.2.2 192.168.3.2
Realserver三台: 1.1.1.100 1.1.1.101 1.1.1.102
2.三層設備OSPF配置
#查找與三層交換與lvs調度相連的端口,在本文端口分別為 g1/0/2 g1/0/3 g1/0/6 #把g1/0/2改為三層端口,並配上IP interface GigabitEthernet1/0/2 port link-mode route ip address 192.168.0.1 255.255.255.0 #配置ospf的參數, timer hello是發送hello包的間隔,timer dead是存活的死亡時間。默認是10,40。 #hello包是ospf裡面維持鄰居關系的報文,這裡配置是每秒發送一個,當到4秒還沒有收到這個報文,就會認為這個鄰居已經丟失,需要修改路由 ospf timer hello 1 ospf timer dead 4 ospf dr-priority 100 #如此類推,把g1/0/3 g1/0/6都配置上 interface GigabitEthernet1/0/3 port link-mode route ip address 192.168.3.1 255.255.255.0 ospf timer hello 1 ospf timer dead 4 ospf dr-priority 99 interface GigabitEthernet1/0/6 port link-mode route ip address 192.168.2.1 255.255.255.0 ospf timer hello 1 ospf timer dead 4 ospf dr-priority 98 #配置ospf ospf 1 area 0.0.0.0 network 192.168.0.0 0.0.0.255 network 192.168.3.0 0.0.0.255 network 192.168.2.0 0.0.0.2553.LVS調度機的OSPF配置
yum –y install quaggab.配置zerba.conf
vim /etc/quagga/zebra.conf hostname lvs-route-1 password xxxxxx enable password xxxxxx log file /var/log/zebra.log service password-encryptionc.配置ospfd.conf
vim /etc/quagga/ospfd.conf #ospf的配置類似於上面三層設備,注意需要把vip聲明出去(1.1.1.1) log file /var/log/ospf.log log stdout log syslog interface eth0 ip ospf hello-interval 1 ip ospf dead-interval 4 router ospf ospf router-id 192.168.0.1 log-adjacency-changes auto-cost reference-bandwidth 1000 network 1.1.1.1/32 area 0.0.0.0 network 192.168.0.0/24 area 0.0.0.0d.開啟IP轉發
sed –i ‘/net.ipv4.ip_forward/d’ /etc/sysctl.conf echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.confsysctl –pe.開啟服務
/etc/init.d/zebra start /etc/init.d/ospfd start chkconfig zebra on chkconfig ospfd on4.LVS keepalived配置
vim /etc/keepalived/keepalived.conf #keepalived的全局配置global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id Ospf_LVS_1 } #VRRP實例,在這個架構下所有的LVS調度機都配置成MASTER vrrp_instance LVS_Cluster{ ##創建實例 實例名為LVS_Cluster state MASTER #備份服務器上將MASTER改為BACKUP interface eth0 ##VIP 捆綁網卡 virtual_router_id 100 ##LVS_ID 在同一個網絡中,LVS_ID是唯一的 priority 100 #選舉的優先級,優先級大的為MASTER 備份服務上將100改為99 advert_int 1 #發送vrrp的檢查報文的間隔,單位秒 authentication { ##認證信息。可以是PASS或者AH auth_type PASS auth_pass 08856CD8 } virtual_ipaddress { 1.1.1.1 } } #LVS實例,在本文采用的是DR模式,WRR調度方式。其實在這種架構下也只能使用DR模式 virtual_server 1.1.1.1 80 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 60 protocol TCP real_server 1.1.1.100 80 { weight 1 # 權重 inhibit_on_failure # 若此節點故障,則將權重設為零(默認是從列表中移除) TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 1.1.1.101 80 { weight 1 # 權重 inhibit_on_failure # 若此節點故障,則將權重設為零(默認是從列表中移除) TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 1.1.1.102 80 { weight 1 # 權重 inhibit_on_failure # 若此節點故障,則將權重設為零(默認是從列表中移除) TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }b.啟動keepalived
/etc/init.d/keepalived start chkconfig keepalived on5.realserver配置
#!/bin/sh ### BEGIN INIT INFO # Provides: lvs_realserver # Default-Start: 3 4 5 # Default-Stop: 0 1 6 # Short-Description: LVS real_server service scripts # Description: LVS real_server start and stop controller ### END INIT INFO # Copyright 2013 lxcong # # chkconfig: - 20 80 # # Author: xxxx@xxxx #有多個虛擬IP,以空格分隔 SNS_VIP="1.1.1.1" . /etc/rc.d/init.d/functions if [[ -z "$SNS_VIP" ]];then echo 'Please set vips in '$0' with SNS_VIP!' fi start(){ num=0 for loop in $SNS_VIP do /sbin/ifconfig lo:$num $loop netmask 255.255.255.255 broadcast $loop /sbin/route add -host $loop dev lo:$num ((num++)) done echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -e -p >/dev/null 2>&1 } stop(){ num=0 for loop in $WEB_VIP do /sbin/ifconfig lo:$num down /sbin/route del -host $loop >/dev/null 2>&1 ((num++)) done echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -e -p >/dev/null 2>&1 } case "$1" in start) start echo "RealServer Start OK" ;; stop) stop echo "RealServer Stoped" ;; restart) stop start ;; *) echo "Usage: $0 {start|stop|restart}" exit 1 esac exit 0b.啟動服務
service lvs_realserver start chkconfig lvs_realserver on總結