Apache 运行时出现“FetchError 没有后端连接”错误

Apache 运行时出现“FetchError 没有后端连接”错误
[centos@ip-172-35-25-65 ~]$  varnishlog
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635280998 1.0
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635281001 1.0
   10 SessionOpen  c 127.0.0.2 55870 127.0.0.2:80
   10 ReqStart     c 127.0.0.2 55870 894208400
   10 RxRequest    c GET
   10 RxURL        c /
   10 RxProtocol   c HTTP/1.0
   10 RxHeader     c X-Real-IP: 198.95.75.75
   10 RxHeader     c X-Forwarded-For: 198.95.75.75
   10 RxHeader     c X-Forwarded-Proto: https
   10 RxHeader     c X-Forwarded-Port: 80
   10 RxHeader     c Host: staging03.cherry.com
   10 RxHeader     c Connection: close
   10 RxHeader     c Cache-Control: max-age=0
   10 RxHeader     c Authorization: Basic aGc6am9objEyMw==
   10 RxHeader     c Upgrade-Insecure-Requests: 1
   10 RxHeader     c User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36
   10 RxHeader     c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
   10 RxHeader     c Accept-Encoding: gzip, deflate
   10 RxHeader     c Accept-Language: en-US,en;q=0.9,fr;q=0.8
   10 RxHeader     c Cookie: ajs_anonymous_id=%22424f4cd9-cbbc-4ead-83b1-273cb21cf453%22; _fbp=fb.1.1630002144579.2012566540; __qca=P0-1416512434-1630002144589; _edwvts=708154457303700204; _gid=GA1.2.1572498662.1635275261; ajs_user_id=%224543534%40mimpi99.com%22; _gcl_au=1.1.
   10 VCL_call     c recv pass
   10 VCL_call     c hash
   10 Hash         c /
   10 Hash         c staging03.cherry.com
   10 Hash         c 80
   10 Hash         c ajs_anonymous_id=%22424f4cd9-cbbc-4ead-83b1-273cb21cf453%22; _fbp=fb.1.1630002144579.2012566540; __qca=P0-1416512434-1630002144589; _edwvts=708154457303700204; _gid=GA1.2.1572498662.1635275261; ajs_user_id=%224543534%40mimpi99.com%22; _gcl_au=1.1.1880042
   10 VCL_return   c hash
   10 VCL_call     c pass pass
   10 FetchError   c no backend connection
   10 VCL_call     c error deliver
   10 VCL_call     c deliver deliver
   10 TxProtocol   c HTTP/1.1
   10 TxStatus     c 503
   10 TxResponse   c Service Unavailable
   10 TxHeader     c Server: Varnish
   10 TxHeader     c Content-Type: text/html; charset=utf-8
   10 TxHeader     c Retry-After: 5
   10 TxHeader     c Content-Length: 392
   10 TxHeader     c Accept-Ranges: bytes
   10 TxHeader     c Date: Tue, 26 Oct 2021 20:43:23 GMT
   10 TxHeader     c X-Varnish: 894208400
   10 TxHeader     c Via: 1.1 varnish
   10 TxHeader     c Connection: close
   10 TxHeader     c X-Age: 0
   10 TxHeader     c X-Cache: MISS
   10 Length       c 392
   10 ReqEnd       c 894208400 1635281003.852778196 1635281003.852984428 0.000073195 0.000165701 0.000040531
   10 SessionClose c error
   10 StatSess     c 127.0.0.2 55870 0 1 1 0 1 0 273 392
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635281004 1.0
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635281007 1.0
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635281010 1.0
    0 CLI          - Rd ping
    0 CLI          - Wr 200 19 PONG 1635281013 1.0

我尝试记录从客户端收到消息时发生的事情:

Error 503 Service Unavailable
Service Unavailable

Guru Meditation:
XID: 894208400

现在,我认为这是因为 Apache 没有运行,因为当我关闭 varnish 时,我收到来自 nginx 的 502 网关错误。无论如何,我阅读了错误日志:

[Tue Oct 26 14:53:47 2021] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0
[Tue Oct 26 14:53:47 2021] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Tue Oct 26 14:53:47 2021] [notice] Digest: generating secret for digest authentication ...
[Tue Oct 26 14:53:47 2021] [notice] Digest: done
[Tue Oct 26 14:53:47 2021] [notice] FastCGI: process manager initialized (pid 23090)
[Tue Oct 26 14:53:47 2021] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fastcgi/2.4.6 configured -- resuming normal operations
[Tue Oct 26 14:53:52 2021] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Tue Oct 26 14:53:52 2021] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico, referer: http://staging03.hgreg.com/
[Tue Oct 26 15:01:21 2021] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Tue Oct 26 15:01:42 2021] [notice] caught SIGTERM, shutting down
[Tue Oct 26 15:01:42 2021] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0
[Tue Oct 26 15:01:42 2021] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Tue Oct 26 15:01:42 2021] [notice] Digest: generating secret for digest authentication ...
[Tue Oct 26 15:01:42 2021] [notice] Digest: done
[Tue Oct 26 15:01:42 2021] [notice] FastCGI: process manager initialized (pid 23299)
[Tue Oct 26 15:01:42 2021] [notice] Apache/2.2.15 (Unix) DAV/2 mod_fastcgi/2.4.6 configured -- resuming normal operations
[Tue Oct 26 15:11:56 2021] [notice] caught SIGTERM, shutting down

我看到了 SIGTERM,正在关闭,所以我认为也许我应该重新启动 Apache,然后我就这样做了,但是我得到了同样的错误,并且 error_log 中没有新的日志。

[centos@ip-172-35-25-65 ~]$ sudo service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[centos@ip-172-35-25-65 ~]$ date
Tue Oct 26 17:12:32 EDT 2021
[centos@ip-172-35-25-65 ~]$ 

现在,我运行了一个 puppet 配置,但它没有完全运行,但我有相同的文件。所以我想知道问题可能是什么。由于所有带有 conf 的文件都已加载,因此加载的 Apache 配置文件之一如下所示:

<VirtualHost *>
    ServerName preprod.staging03.cherry.com

    
    
    ServerAlias betacherry.staging03.cherry.com staging03.cherry.com
    
    

    DocumentRoot /home/staging03/version/preprod.staging03.cherry.com
    ServerAdmin [email protected]

    SetEnv environment preprod
    SetEnv project staging03

    UseCanonicalName Off
    #CustomLog /var/log/httpd/preprod.staging03.cherry.com_log combined
    #CustomLog /var/log/httpd/preprod.staging03.cherry.com-bytes_log "%{%s}t %I .\n%{%s}t %O ."

    ## User cherry # Needed for Cpanel::ApacheConf
    UserDir disabled
    UserDir enabled staging03
    
      #<IfModule mod_suphp.c>
    #    suPHP_UserGroup staging03 staging03
    #</IfModule>
    
    SuexecUserGroup staging03 staging03
    
    <directory "/home/staging03/version">
        AddHandler php5-fcgi .php
        Action php5-fcgi /php5-fcgi-staging03
        AllowOverride All

        
        AuthType Basic
        AuthName "staging03-preprod"
        AuthUserFile "/etc/httpd/conf.d/htpasswd.staging03"
        require valid-user

        satisfy any
        deny from all

        Order deny,allow
        SetEnvIf X-Hg-Internal-IP 1 HgInternalIP=1
        Allow from env=HgInternalIP

        SetEnvIf User-Agent "Amazon CloudFront" AmazonCloudFront
        Allow from env=AmazonCloudFront

        SetEnvIf User-Agent "^(.*)Lighthouse(.*)$" Lighthouse=1
        Allow from env=Lighthouse
        
    </directory>
    <IfModule concurrent_php.c>
        php5_admin_value open_basedir "/home/staging03:/usr/lib/php:/usr/local/lib/php:/tmp"
    </IfModule>
    <IfModule !concurrent_php.c>
        <IfModule mod_php5.c>
            php_admin_value open_basedir "/home/staging03:/usr/lib/php:/usr/local/lib/php:/tmp"
        </IfModule>
        <IfModule sapi_apache2.c>
            php_admin_value open_basedir "/home/staging03:/usr/lib/php:/usr/php4/lib/php:/usr/local/lib/php:/usr/local/php4/lib/php:/tmp"
        </IfModule>
    </IfModule>
    <IfModule !mod_disable_suexec.c>
        <IfModule !mod_ruid2.c>
            SuexecUserGroup staging03 staging03
        </IfModule>
    </IfModule>
    <IfModule mod_ruid2.c>
        RMode config
        RUidGid staging03 staging03
    </IfModule>
    <IfModule itk.c>
        # For more information on MPM ITK, please read:
        #   http://mpm-itk.sesse.net/
        AssignUserID staging03 staging03
    </IfModule>
</VirtualHost>

那么我应该查看哪些文件以及如何检查问题是否出在 Apache 上,因为我们有 nginx 路由到 varnish,然后路由到 Apache,所以我认为问题出在 Apache 上,但是我没有从日志中得到任何有用的信息,Apache 运行时没有任何问题,它只是没有为页面提供服务,而 Varnish 由于某种原因无法访问 Apache?

我正在运行 CENTOS 6,并且我还有另一台具有相同配置且运行良好的服务器,但是当我比较 etc 文件夹的差异时,我并没有看到任何明显的差异。

答案1

根据您的日志,我可以看到 Varnish 和 Apache 都在同一台机器上运行。Varnish 应该在端口上运行,80而 Apache 应该在端口上运行8080

显然还有一个 Nginx 正在运行,所以我假设它用于 TLS 终止,在端口上运行443

步骤 1:确保 Apache 成功监听端口 8080

运行sudo netstat -plnt以查明每个服务使用了哪些端口。

确保httpd服务正在监听端口8080,并通过运行来验证这一点curl -I localhost:8080

第 2 步:在 VCL 文件中添加后端的健康探测

标准 VCL 不提供针对您的默认后端的运行状况探测。使用以下 VCL 代码,您可以持续监控后端运行状况:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .probe = {
        .url = "/";
        .timeout = 2s;
        .interval = 5s;
        .window = 10;
        .threshold = 5;
   }
}

添加探测器并加载新的 VCL 后,您可以调用以下命令根据探测器检查后端的健康状况:

varnishlog -g raw -i backend_health

如果输出包含Still sick,则表明后端不可用,并且状态代码可能会告诉您原因。

步骤 3:升级 Varnish 服务器

我忍不住注意到了RxHeaderVSL 输出中的类似术语。这清楚地表明您正在使用不再受支持的旧版 Varnish。

即使在非常老的 Varnish 版本中,也RxHeader被替换为ReqHeader

我的建议:升级到Varnish 6.0 LTS。Varnish 的这个 LTS 版本经常会修复一些错误并发布安全补丁。请参阅https://www.varnish-software.com/developers/tutorials/installing-varnish-centos/了解如何在 CentOS 上安装此版本。

相关内容