我不知道如何解决此问题。我收到所有“nginx 上游”代理将调用传递给 haproxy fast_thin 和 slow_thin(服务器 127.0.0.1:3100 和服务器 127.0.0.1:3200)的“503 服务不可用”http 错误,这些代理在 6 个 Thin 服务器(127.0.0.1:3000 .. 3005)上进行负载平衡。/blog 等静态文件目前没有问题。失败的原因是:端口 80 上的 nginx - 3100 和 3200 上的 haproxy - 3000 .. 3005 上的 thin,然后是 Rails。这是 /etc/nginx/nginx.conf:
user nginx;
worker_processes 2;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
tcp_nodelay on;
include /etc/nginx/conf.d/*.conf;
}
然后是/etc/nginx/conf.d/default.conf
upstream fast_thin {
server 127.0.0.1:3100;
}
upstream slow_thin {
server 127.0.0.1:3200;
}
server {
listen 80;
server_name www.gitwatcher.com;
rewrite ^/(.*) http://gitwatcher.com/$1 permanent;
}
server {
listen 80;
server_name gitwatcher.com;
access_log /var/www/gitwatcher/log/access.log;
error_log /var/www/gitwatcher/log/error.log;
root /var/www/gitwatcher/public;
# index index.html;
location /about {
proxy_pass http://fast_thin;
break;
}
location /trends {
proxy_pass http://slow_thin;
break;
}
location /categories {
proxy_pass http://slow_thin;
break;
}
location /signout {
proxy_pass http://slow_thin;
break;
}
location /auth/github {
proxy_pass http://slow_thin;
break;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://slow_thin;
break;
}
}
}
然后 haproxy 配置文件 /etc/haproxy/haproxy.cfg :
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
#chroot /usr/share/haproxy
user haproxy
group haproxy
daemon
#debug
#quiet
nbproc 1 # number of processing cores
defaults
log global
retries 3
maxconn 2000
contimeout 5000
mode http
clitimeout 60000 # maximum inactivity time on the client side
srvtimeout 30000 # maximum inactivity time on the server side
timeout connect 4000 # maximum time to wait for a connection attempt to a server to succeed
option httplog
option dontlognull
option redispatch
option httpclose # disable keepalive (HAProxy does not yet support the HTTP keep-alive mode)
option abortonclose # enable early dropping of aborted requests from pending queue
option httpchk # enable HTTP protocol to check on servers health
option forwardfor # enable insert of X-Forwarded-For headers
balance roundrobin # each server is used in turns, according to assigned weight
stats enable # enable web-stats at /haproxy?stats
stats auth haproxy:pr0xystats # force HTTP Auth to view stats
stats refresh 5s # refresh rate of stats page
listen rails_proxy 127.0.0.1:3100
# - equal weights on all servers
# - maxconn will queue requests at HAProxy if limit is reached
# - minconn dynamically scales the connection concurrency (bound my maxconn) depending on size of HAProxy queue
# - check health every 20000 microseconds
server web1 127.0.0.1:3000 weight 1 minconn 3 maxconn 6 check inter 20000
server web1 127.0.0.1:3001 weight 1 minconn 3 maxconn 6 check inter 20000
server web1 127.0.0.1:3002 weight 1 minconn 3 maxconn 6 check inter 20000
listen slow_proxy 127.0.0.1:3200
# cluster for slow requests, lower the queues, check less frequently
server slow1 127.0.0.1:3003 weight 1 minconn 1 maxconn 3 check inter 40000
server slow2 127.0.0.1:3004 weight 1 minconn 1 maxconn 3 check inter 40000
server slow3 127.0.0.1:3005 weight 1 minconn 1 maxconn 3 check inter 40000
以及 Thin 配置文件 /etc/thin/gitwatcher.yml :
---
chdir: /var/www/gitwatcher
environment: production
address: 0.0.0.0
port: 3000
timeout: 300
log: log/thin.log
pid: tmp/pids/thin.pid
max_conns: 1024
max_persistent_conns: 100
require: []
wait: 30
servers: 3
daemonize: true
如果我查看开放的监听端口,我会得到以下信息:
root@fullness:/var/www/gitwatcher# lsof | grep TCP | egrep "nginx|haproxy|thin"
nginx 834 root 8u IPv4 921 0t0 TCP *:http (LISTEN)
nginx 835 nginx 8u IPv4 921 0t0 TCP *:http (LISTEN)
nginx 837 nginx 8u IPv4 921 0t0 TCP *:http (LISTEN)
haproxy 1908 haproxy 4u IPv4 11699 0t0 TCP localhost:3100 (LISTEN)
haproxy 1908 haproxy 6u IPv4 11701 0t0 TCP localhost:3200 (LISTEN)
root@fullness:/var/www/gitwatcher#
iptables -L 获取以下信息:
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere tcp dpt:22222
ACCEPT tcp -- anywhere anywhere tcp dpt:http
ACCEPT tcp -- anywhere anywhere tcp dpt:https
ACCEPT all -- anywhere anywhere
DROP all -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
有什么帮助吗?
更新:尝试按照建议的方式进行故障排除没有帮助,因为我只收到“无法连接”错误,没有更多:
root@fullness:/var/www/gitwatcher/log# ps -ef | grep thin
root 3740 1 4 15:20 ? 00:00:19 thin server (0.0.0.0:3000)
root 3809 1 5 15:20 ? 00:00:22 thin server (0.0.0.0:3001)
root 3834 1 6 15:20 ? 00:00:26 thin server (0.0.0.0:3002)
root 4166 2274 2 15:27 pts/1 00:00:00 grep --color=auto thin
root@fullness:/var/www/gitwatcher/log# curl http://localhost:3000/trends
curl: (7) couldn't connect to host
root@fullness:/var/www/gitwatcher/log# curl http://localhost:3001/trends
curl: (7) couldn't connect to host
root@fullness:/var/www/gitwatcher/log# curl http://localhost:3002/trends
curl: (7) couldn't connect to host
root@fullness:/var/www/gitwatcher/log#
更新 :
root@fullness:/var/www/gitwatcher# netstat -a | egrep "3000|3001|3002"
tcp 0 0 *:3000 *:* LISTEN
tcp 0 0 *:3001 *:* LISTEN
tcp 0 0 *:3002 *:* LISTEN
root@fullness:/var/www/gitwatcher#
答案1
如果您尝试诊断上游的 503,请执行以下操作之一:
在本地机器上,curl
直接向其中一个上游服务器请求有效的 URL,然后查看结果:
curl http://localhost:3000/example
如果您无法执行此操作,请临时更新配置,以便上游监听外部 IP 并在浏览器中访问 IP:端口。
然后查看请求的日志。这应该可以让你确定服务器错误的位置。