nginx proxy_cache:限制同一URL的并行请求

nginx proxy_cache:限制同一URL的并行请求

我们使用 nginx 作为反向代理从上游服务器获取文件。这些文件不是动态的(至少不是基于每个请求的),并且(有时)延迟很高。

我想将对同一个 URL 的请求限制为 1 个。

我想避免的行为示例:

127.0.0.1 - - [03/Jan/2013:16:08:15 +0100] "GET /part-00132.ts HTTP/1.0" 200 1506068 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:28 +0100] "GET /part-00133.ts HTTP/1.0" 200 1189476 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:29 +0100] "GET /part-00133.ts HTTP/1.0" 200 1189476 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17" "-"
127.0.0.1 - - [03/Jan/2013:16:08:34 +0100] "GET /part-00133.ts HTTP/1.0" 200 1189476 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:44 +0100] "GET /part-00134.ts HTTP/1.0" 200 1762876 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17" "-"
127.0.0.1 - - [03/Jan/2013:16:08:53 +0100] "GET /part-00135.ts HTTP/1.0" 200 1627704 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:59 +0100] "GET /part-00136.ts HTTP/1.0" 200 1252456 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:03 +0100] "GET /part-00134.ts HTTP/1.0" 200 1762876 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:04 +0100] "GET /part-00137.ts HTTP/1.0" 200 1120292 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:07 +0100] "GET /part-00135.ts HTTP/1.0" 200 1627704 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17" "-"
127.0.0.1 - - [03/Jan/2013:16:09:12 +0100] "GET /part-00134.ts HTTP/1.0" 200 1762876 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:14 +0100] "GET /part-00136.ts HTTP/1.0" 200 1252456 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:17 +0100] "GET /part-00135.ts HTTP/1.0" 200 1627704 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:17 +0100] "GET /part-00138.ts HTTP/1.0" 200 1248884 "-" "Prefetch" "-"

它应该是这样的:

127.0.0.1 - - [03/Jan/2013:16:08:15 +0100] "GET /part-00132.ts HTTP/1.0" 200 1506068 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:28 +0100] "GET /part-00133.ts HTTP/1.0" 200 1189476 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:44 +0100] "GET /part-00134.ts HTTP/1.0" 200 1762876 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17" "-"
127.0.0.1 - - [03/Jan/2013:16:08:53 +0100] "GET /part-00135.ts HTTP/1.0" 200 1627704 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:08:59 +0100] "GET /part-00136.ts HTTP/1.0" 200 1252456 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:04 +0100] "GET /part-00137.ts HTTP/1.0" 200 1120292 "-" "Prefetch" "-"
127.0.0.1 - - [03/Jan/2013:16:09:17 +0100] "GET /part-00138.ts HTTP/1.0" 200 1248884 "-" "Prefetch" "-"

更新:

用一张图稍微描述一下这个问题:

            nginx              upstream
           +-----+            +--------+
           |     |            |        |
   A +----->     +-----------------+   |
           |     |            |    |   |
   B +----->     +--------------+  |   |
           |     |            | |  |   |
     <-----+     <--------------|--+   |
           |     |            | |      |
     <-----+     <--------------+      |
           |     |            |        |
   C +----->     +---+        |        |
           |     |   |        |        |
     <-----+     <---+        |        |
           |     |            |        |
           +-----+            +--------+

所以我希望客户端B等待A响应。

答案1

摘自Nginx如何禁用多线程下载

使用 nginxlimit_conn配置示例

limit_zone one $binary_remote_addr 10m;

server {
  location /download/ {
    limit_conn one 1;
  }
}

不仅仅是下载管理器的Range请求会多路复用下载(这不会禁用标头);现代浏览器加载页面时也会多路复用请求。除了需要专门阻止多路复用下载的地方外,Range在任何地方使用此配置时都要非常小心。location

答案2

一个可能的解决方案是缓存。当使用 nginx 的代理传递或 fcgi 传递时,您可以选择将返回数据缓存一段时间。您可以在文档中阅读更多相关信息:代理通行证fcgi 版本。两者的工作方式实际上完全相同。

我还建议查阅一些相关教程,因为从文档中了解缓存操作的各个部分如何协同工作并不完全是易事。

但主要思想是,您从 URL 中创建一个代理“密钥”(就您而言)。然后您可以将其设置为缓存 10 分钟左右。因此,后续调用将从本地缓存中提供服务,而不是再次获取它。

然后,您可以通过设置来阻止更新期间的后续请求:

proxy_cache_use_stale updating;

相关内容