504 超时使用太多 cUrl 来自已关闭的网站

504 超时使用太多 cUrl 来自已关闭的网站

我有一台配备 32 GB RAM / 8 个处理器的专用服务器。我在我的所有网站上都使用 cUrl php 显示一些代码。我创建 cUrl 的网站是一个 VPS,我在其中使用另一个带有多个代理的 cUrl,该代理会切换每个请求。

因此我使用如下方式:在站点 A(专用服务器)上使用 curl 访问站点 B(VPS)。cUrl 代码如下所示:

$url = "https://siteB.com/code";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 7);
curl_setopt ($ch, CURLOPT_TIMEOUT, 7);
$curl_res = curl_exec($ch);
curl_close ($ch);

我的问题是,在一个网站上(在本例中为网站 A),每天有大约 30 万名访问者,如果网站 B 由于某种原因没有响应,则网站 A 会出现 504 超时,而专用服务器上的其他网站则正常运行。

站点 AI 的错误日志中有如下错误:

25329#25329: *3623 upstream timed out (110: Connection timed out) while reading response header from upstream

当我检查 Nginx 时,我发现从 40-50 次写入(通常是)开始,写入过程太多,从这里开始就会超时

Active connections: 1756 
server accepts handled requests
11367 11367 11377 
Reading: 0 Writing: 1395 Waiting: 361

我的 nginx 配置是:

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    multi_accept on;
    use epoll;
    worker_connections 65535;
}

http {
    ## Basic Settings ##
    client_body_buffer_size        128k;
    client_body_timeout            5s; # Use 5s for high-traffic sites
    client_header_timeout          5s; # Use 5s for high-traffic sites
    client_max_body_size           1024m;
    keepalive_timeout              30s;
    open_file_cache                max=200000 inactive=20s;
    open_file_cache_errors         on;
    open_file_cache_min_uses       2;
    open_file_cache_valid          30s;
    port_in_redirect               off;
    reset_timedout_connection      on;
    send_timeout                   5s;
    sendfile                       on;
    server_name_in_redirect        off;
    server_names_hash_bucket_size  1024;
    server_names_hash_max_size     1024;
    server_tokens                  off;
    tcp_nodelay                    on;
    tcp_nopush                     on;
    types_hash_max_size            2048;

    ## DNS Resolver ##
    # If in China, enable the OpenDNS entry that matches your network connectivity (IPv4 only or IPv4 & IPv6)
    # OpenDNS (IPv4 & IPv6)
    #resolver                      208.67.222.222 208.67.220.220 [2620:0:ccc::2] [2620:0:ccd::2];
    # OpenDNS (IPv4 only)
    #resolver                      208.67.222.222 208.67.220.220;
    # Google Public DNS (IPv4 & IPv6)
    #resolver                      8.8.8.8 8.8.4.4 [2001:4860:4860::8888] [2001:4860:4860::8844];
    # Google Public DNS (IPv4 only) [default]
    resolver                       8.8.8.8 8.8.4.4;

    ## Real IP Forwarding ##
    set_real_ip_from 127.0.0.1;

    # CloudFlare IPs
    # List from: https://www.cloudflare.com/ips-v4
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 104.16.0.0/12;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 131.0.72.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    # List from: https://www.cloudflare.com/ips-v6
    set_real_ip_from 2400:cb00::/32;
    set_real_ip_from 2405:8100::/32;
    set_real_ip_from 2405:b500::/32;
    set_real_ip_from 2606:4700::/32;
    set_real_ip_from 2803:f800::/32;
    set_real_ip_from 2c0f:f248::/32;
    set_real_ip_from 2a06:98c0::/29;

    # Replace with correct visitor IP
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

    ## MIME ##
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ## Logging Settings ##
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ## Gzip Settings ##
    gzip on;
    gzip_buffers 16 8k;
    gzip_comp_level 5;
    gzip_disable "msie6";
    gzip_min_length 256;
    gzip_proxied any;
    gzip_types
        application/atom+xml
        application/javascript
        application/json
        application/ld+json
        application/manifest+json
        application/rss+xml
        application/vnd.geo+json
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-javascript
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        font/opentype
        image/bmp
        image/svg+xml
        image/x-icon
        text/cache-manifest
        text/css
        text/javascript
        text/plain
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/x-component
        text/x-cross-domain-policy
        text/x-js
        text/xml;
    gzip_vary on;

    # Security Headers
    add_header        X-XSS-Protection "1; mode=block" always;
    add_header        X-Content-Type-Options "nosniff" always;

    # Info
    add_header        X-Nginx-Cache-Status $upstream_cache_status;
    add_header        X-Server-Powered-By "Engintron";

    # Proxy Cache Settings
    proxy_cache_path  /var/cache/nginx/engintron_dynamic levels=1:2 keys_zone=engintron_dynamic:20m inactive=10m max_size=512m;
    proxy_cache_path  /var/cache/nginx/engintron_static levels=1:2 keys_zone=engintron_static:20m inactive=10m max_size=512m;
    proxy_temp_path   /var/cache/nginx/engintron_temp;

    ## Virtual Host Configs ##
    include /etc/nginx/conf.d/*.conf;
}

我想问的是,我如何让 cUrl 检查网站 B 是否速度慢或宕机,如果 4-5-6 秒内没有响应,则停止该请求?或者我可以增加配置中的某些内容?我还看到平均负载不超过 1.5(这是所有服务器的最大值)。

现在系统状态:

Processors: 8 CPU(s)

RAM: 31.19 GB

Memory Usage:
              total        used        free      shared  buff/cache   available
Mem:          31940        2155        2301         686       27484       28449
Swap:          1021          22         999

据我所知,某处存在一个限制,如果达到 X 个连接,它就会变慢或超时,但不知道是什么原因。

在我的专用服务器上,除了 nginx 的配置外,我所有的设置配置都默认来自 WHM Cpanel。

PS:我对服务器一点都不擅长,是否有人可以指导我。

编辑

仍然有一个站点瘫痪(将超时时间修改为相同后),其余站点运行正常。

这是现在我的服务器上的连接:

Active connections: 4382 
server accepts handled requests
 457784 457784 478795 
Reading: 0 Writing: 4071 Waiting: 310 

通常最多有 1000 个活动连接...

答案1

您将超时时间send_timeout(以及其他几个超时时间)设置为 5 秒,但调用外部网站的 PHP 代码最多需要等待 7 秒才能响应。显然,这行不通。您需要更改其中一个或两个。

相关内容