Nginx + Tomcat + ngx

Nginx處理用戶請求的靜態頁面，tomcat處理用戶請求jsp頁面，來實現動態分離，前端nginx反向代理後端nginx+tomcat集群，實現負載均衡，這樣一來就能更好的提高並發，處理性能，並隱藏後端，提高安全

環境：

前端： Centos 192.168.0.211： nginx + Ngx_cache_purge

後端1： Centos 192.168.0.222： nginx + Tomcat

後端2： Centos 192.168.0.223：nginx + Tomcat

使用軟件：

Nginx： http://nginx.org/en/download.html

JDK： http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Tomcat： http://tomcat.apache.org/download-80.cgi

首先配置後端Tomcat：

1，JDK 配置：

[root@Tomcat ~]# tar zxf jdk-8u40-linux-i586.tar.gz
[root@Tomcat ~]# mv jdk1.8.0_40/ /usr/local/jdk
[root@Tomcat ~]# vi /etc/profile
JAVA_HOME=/usr/local/jdk
PATH=$PATH:$JAVA_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export JAVA_HOME PATHCLASSPATH
[root@Tomcat ~]# source /etc/profile
[root@Tomcat ~]# java -version  #顯示版本說明成功
java version"1.8.0_40"

2，Tomcat 安裝
先創建普通用戶，用來運行tomcat，如遇權限問題，可以先關閉selinux

[root@Tomcat ~]# useradd -s /sbin/nologin tomcat
[root@Tomcat ~]# passwd tomcat
[root@Tomcat ~]# tar zxf apache-tomcat-8.0.21.tar.gz
[root@Tomcat ~]# mv apache-tomcat-8.0.21 /usr/local/tomcat
[root@Tomcat ~]# chown tomcat.tomcat -R /usr/local/tomcat
[root@Tomcat ~]# su - tomcat /usr/local/tomcat/bin/startup.sh
[root@Tomcat ~]# echo "su - tomcat /usr/local/tomcat/bin/startup.sh" >> /etc/rc.local  #開機啟動

3, 安裝Nginx

[root@Tomcat ~]# useradd -s /sbin/nologin nginx
[root@Tomcat ~]# yum install –y make zlib-devel openssl-devel pcre pcre-devel
[root@Tomcat ~]# tar zxvf nginx-1.4.4.tar.gz
[root@Tomcat ~]# cd nginx-1.4.4
[root@Tomcat nginx-1.4.4]# ulimit -SHn 51200
[root@Tomcat nginx-1.4.4]# ./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module
[root@Tomcat nginx-1.4.4]# make && make install
[root@Tomcat nginx-1.4.4]# \cp -pa /usr/local/nginx/sbin/nginx /etc/init.d/
[root@Tomcat nginx-1.4.4]# chmod +x /etc/init.d/nginx
[root@Tomcat nginx-1.4.4]# echo "ulimit -SHn 51200" >> /etc/rc.d/rc.local
[root@Tomcat nginx-1.4.4]# echo "/etc/init.d/nginx" >> /etc/rc.d/rc.local #開機啟動

4，主配置文件 nginx.conf

user  www www;
worker_processes  1; #跟服務器cpu一致就可以了，不要超過cpu的的內核個數，超過將會增加服務器負荷
 
error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
 
pid        logs/nginx.pid;
 
worker_rlimit_nofile 51200;
 
events {
    use epoll;
    worker_connections  51200;
}
 
http {
    include       mime.types;
    default_type  application/octet-stream;
 
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
 
    #access_log  logs/access.log  main;
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    server_name_in_redirect off;
    client_max_body_size 10m; #允許客戶端請求的最大單文件字節數
    client_body_buffer_size 128k;  #緩沖區代理緩沖用戶端請求的最大字節數
 
    sendfile        on;
    tcp_nopush     on;
    tcp_nodelay    on;
 
    #keepalive_timeout  0;
    keepalive_timeout  60;
    set_real_ip_from 192.168.0.0/24; #允許被信任ip段
    real_ip_header    X-Real-IP;     #獲取前端訪問真實ip
 
    gzip  on;                        #開啟gzip壓縮
    gzip_min_length 1k;              
    gzip_buffers 4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 3;
    gzip_disable "MSIE [1-6].";
    gzip_types text/plain application/x-javascript text/css application/xml image/jpeg image/gif image/png;
    gzip_vary on;
    gzip_proxied any;
 
    proxy_redirect off;
    proxy_connect_timeout 300;        #nginx跟Tomcat連接超時時間(代理連接超時)
    proxy_send_timeout 300;           #連接成功後，後端服務響應時間(代理發送超時)
    proxy_read_timeout 300;           #連接成功後，後端服務響應時間(代理接收超時)
    proxy_buffer_size 4k;             #設置代理服務器（nginx）保存用戶頭信息的緩沖區大小
    proxy_buffers 6 64k;              #proxy_buffers緩沖區，網頁平均在64k以下的話，這樣設置
    proxy_busy_buffers_size 128k;     #高負荷下緩沖大小（proxy_buffers*2）
    proxy_temp_file_write_size 64k;   #設定緩存文件夾大小
    proxy_set_header   Host  $host;   #後端的Web服務器可以通過X-Forwarded-For獲取用戶真實IP
    proxy_set_header   X-Real-IP  $remote_addr;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
 
    open_file_cache max=204800 inactive=20s;
    open_file_cache_min_uses 1;
    open_file_cache_valid 30s;
 
    include vhost/*.conf;
}

5, 配置站點文件： tomcat.conf

[root@Tomcat conf]# mkdir vhost
[root@Tomcat conf]# cd vhost/
[root@Tomcat vhost]# vi tomcat.conf

upstream tomcat_server {
   server   192.168.0.222:8080;
}
 
server {
   listen 80;
   server_name 192.168.0.222;
   root /usr/local/tomcat/webapps/ROOT/;  #同tomcat一致
   index index.html index.jsp index.php;
 
location ~ .*.jsp$ {
proxy_next_upstream http_503 http_500 http_502 error timeout invalid_header;
proxy_pass http://tomcat_server;
}
 
#這裡使用的tomcat安裝環境，下面是為了登入tomcat管理
location ~ /manager/ {
proxy_pass http://tomcat_server;
}
 
location ~ /host-manager/ {
proxy_pass http://tomcat_server;
}
 
error_page 404   /404.html;
error_page 500 502 503 504 /50x.html;
}

上面的配置已經可以實現Nginx + Tomcat的動靜分離，另一台Tomcat也是這樣配置，上面我並沒有對靜態文件進行Cache，這是有道理的， 我打算把靜態文件在前端進行cache

1，如果後端也進行cache， 更新文件的時候，後端和前端都要進行清除cache，這樣顯得麻煩

2，如果後端也進行cache，必要到靜態文件的location中添加proxy_pass代理tomcat，這樣卻達不到動靜分離，不添加proxy_pass, 卻不能使用purge命中清除cache，當然也可以些shell運行，手動清除

=======================

配置前端： Centos 192.168.0.211： nginx + Ngx_cache_purge

前端的nginx配置與後端差不多，主要是編譯安裝的時候添加cache模塊，基礎看後端安裝，不多介紹了

1，先解壓nginx 和 Ngx_cache_purge，主要不同是安裝的時候添加了ngx_cache_purge模塊

[root@Nginx-C opt]# tar zxf ngx_cache_purge-2.0.tar.gz 
[root@Nginx-C opt]# tar zxf nginx-1.4.3.tar.gz 
[root@Nginx-C opt]# cd nginx-1.4.3
[root@Nginx-C nginx-1.4.3]# ./configure --user=www --group=www --add-module=../ngx_cache_purge-2.0 --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module
[root@Nginx-C nginx-1.4.3]# make && make install

2，配置主配置文件 nginx.conf

user  www www;
worker_processes  1;
 
error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
 
pid        logs/nginx.pid;
 
worker_rlimit_nofile 51200;
 
events {
    use epoll;
    worker_connections  51200;
}
 
http {
    include       mime.types;
    default_type  application/octet-stream;
 
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
 
    #access_log  logs/access.log  main;
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    server_name_in_redirect off;
    client_max_body_size 10m;
    client_body_buffer_size 128k;
 
    sendfile        on;
    tcp_nopush     on;
    tcp_nodelay    on;
 
    #keepalive_timeout  0;
    keepalive_timeout  60;
 
    gzip  on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 3;
    gzip_disable "MSIE [1-6].";
    gzip_types text/plain application/x-javascript text/css application/xml image/jpeg image/gif image/png; #添加圖片壓縮
    gzip_vary on;
    gzip_proxied any;
 
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    proxy_read_timeout 300;
    proxy_buffer_size 64k;
    proxy_buffers 4 64k;
    proxy_busy_buffers_size 128k;
    proxy_temp_file_write_size 128k;
    proxy_set_header   Host             $host;
    proxy_set_header   X-Real-IP        $remote_addr; #傳遞真實ip給後端
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
 
    open_file_cache max=204800 inactive=20s;
    open_file_cache_min_uses 1;
    open_file_cache_valid 30s;
 
    proxy_cache_path /cache/proxy_cache levels=1:2 keys_zone=cache_one:100m inactive=1d max_size=30g; #100m和30G，按照服務要求，適當增大
    proxy_temp_path /cache/proxy_temp;
 
    include vhost/*.conf;
}

3, 配置站點文件

upstream tomcat_server {
   server   192.168.0.222 weight=1 max_fails=2 fail_timeout=30s; #添加ip，不是tomcat的8080端口
}
 
server {
   listen 80;
   server_name 192.168.0.211;
   index index.html index.jsp index.php;
 
   location / {
   proxy_next_upstream http_503 http_500 http_502 error timeout invalid_header;
   proxy_cache cache_one;
   add_header  Nginx-Cache "$upstream_cache_status";
   proxy_cache_key $host$uri$is_args$args;
   proxy_set_header Accept-Encoding "";
   proxy_pass http://tomcat_server;
   proxy_cache_valid  200 304 12h;
   proxy_cache_valid  301 302 1m;
   proxy_cache_valid  any 1m;
   expires 1d;
 }
 
#jsp,do文件不進行cache
location ~ .*\.(jsp|do)$ {
    proxy_set_header Accept-Encoding ""; #只添加了一個， 其他的都添加到主配置文件了，以後添加站點不用在重復寫
    proxy_pass http://tomcat_server;
    }
 
location ~ /purge(/.*) {
    allow 127.0.0.1;
    allow 192.168.0.0/24;
    deny all;
    proxy_cache_purge cache_one $host$1$is_args$args;
 }
 
location /ngx_status
    {
    stub_status on;
    access_log off;
    allow 127.0.0.1;
    allow 192.168.0.0/24; #自己的ip地址
    deny all;
    }
 
}

最後是性能測試
這裡使用的是ab壓力測試工具，後面會介紹如何單獨安裝ab測試工具
1，前端

[root@Tomcat ~]# ab -c 1000 -n 4000 http://192.168.0.211/docs/security-howto.html
====
Requests per second:    3304.24 [#/sec] (mean)
Time per request:       302.642 [ms] (mean)
Time per request:       0.303 [ms] (mean, across all concurrent requests)
Transfer rate:          110426.03 [Kbytes/sec] received

2，直接測試後端

[root@Nginx-C vhost]# ab -c 1000 -n 4000 http://192.168.0.222/docs/security-howto.html
====
Requests per second:    3416.84 [#/sec] (mean)
Time per request:       292.668 [ms] (mean)
Time per request:       0.293 [ms] (mean, across all concurrent requests)
Transfer rate:          114681.80 [Kbytes/sec] received

3，直接測試tomcat

[root@Nginx-C vhost]# ab -c 1000 -n 4000 http://192.168.0.222:8080/docs/security-howto.html
====
Requests per second:    1995.18 [#/sec] (mean)
Time per request:       501.209 [ms] (mean)
Time per request:       0.501 [ms] (mean, across all concurrent requests)
Transfer rate:          66449.32 [Kbytes/sec] received

前端的壓力測試比nginx+tomcat動靜分離要小點，但是實現了負載， 明顯要比tomcat單獨處理要強很多
原文:http://my.oschina.net/harlanblog/blog/395350