☰

HAProxy 和后端服务器之间丢失了一半的数据

tag-icon • tag-icon ubuntu haproxy
HAProxy 和后端服务器之间丢失了一半的数据

我在 Ubuntu 上有这样的设置

浏览器--> HAProxy-->后端服务器

后端服务器是一个 ASP.NET Core Web 应用程序。

99.9％ 的时间它都可以正常工作，除了上传二进制文件（带有多部分表单数据的简单 POST）时，在这种情况下我会收到错误：

System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component. 

   at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)

   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)

   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)

   at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)

   at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgeryTokenStore.GetRequestTokensAsync(HttpContext httpContext)

   at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgery.ValidateRequestAsync(HttpContext httpContext)

   at Microsoft.AspNetCore.Mvc.ViewFeatures.Filters.ValidateAntiforgeryTokenAuthorizationFilter.OnAuthorizationAsync(AuthorizationFilterContext context)

   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)

   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)

   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)

   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

所以，我一直在尝试探究原因。

我直接尝试了 ASP.NET Core 后端，它可以工作。

因此我将传入的数据记录到后端，并发现如果我直接这样做，那么我会得到预期的请求标头，然后在正文中获得上传的文件。

然后我通过 HAProxy 将传入数据记录到后端，发现一半的数据丢失了。

即后端看到的第一个数据似乎是

04 E2 22 FC 60 FF 2B E1  BF 85 D2 75 F9 44 94 86

但这些字节大约占了我文件的一半，即 25,126 字节。我根本看不到任何标头信息。

我准备接受我的日志记录不完善且不一定准确。似乎请求以某种方式被分成了两个，或者缓冲区已被填满然后被覆盖。

可能是什么问题？

global

    chroot /var/lib/haproxy
    log /dev/log    local0 info

    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group users
    daemon
        
    lua-load /home/user/haproxy-mapping.lua

    ssl-default-bind-ciphers ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS:!AESCCM
    ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
    tune.ssl.default-dh-param 2048


defaults
    log     global
    mode    http

    timeout connect 30000
    timeout client  50000
    timeout server  50000
    option forwardfor
    option http-server-close #have also try http-keep-alive


frontend httpfront
    mode http
    bind *:80
    redirect scheme https code 301 if !{ ssl_fc }


frontend web_front_end

    bind *:443 ssl crt /home/.....file.pem
    mode http

     log /var/lib/haproxy/dev/log local0 info

        # Rate limiting
        stick-table  type ip  size 100k  expire 600s  store http_req_rate(60s) #store up to 100k requests for 60s, see if over 60s there are more than 600
        http-request track-sc0 src
        http-request deny deny_status 429 if { sc_http_req_rate(0) gt 600 }


    # Ensure we have a clean state to start with
    http-request del-header X-SERVER-SNI

    # Set the concatenated value of the SNI value to a temporary header
    http-request set-header X-SERVER-SNI haproxy.%[ssl_fc_sni] if { ssl_fc_sni -m found }

    # Set the value of the header to a transaction-level variable
    http-request set-var(txn.fc_sni) ssl_fc_sni #hdr(X-SERVER-SNI) if { hdr(X-SERVER-SNI) -m found }

    #use Lua code to determine which backend to send to
    use_backend %[lua.legacy_new_backend]





backend backendnodes_1_https
    balance roundrobin
    option forwardfor
    server node1 127.0.0.1:446 ssl verify none sni var(txn.fc_sni)

答案1

现在看来 HAProxy 配置可以正常工作

option http-keep-alive

代替

option http-server-close

我发誓我以前尝试过它，但它确实有帮助。

相关内容