![如何配置 Nginx 来代理超过 200K 的 MQTT 连接?](https://linux22.com/image/788666/%E5%A6%82%E4%BD%95%E9%85%8D%E7%BD%AE%20Nginx%20%E6%9D%A5%E4%BB%A3%E7%90%86%E8%B6%85%E8%BF%87%20200K%20%E7%9A%84%20MQTT%20%E8%BF%9E%E6%8E%A5%EF%BC%9F.png)
我需要代理大量 MQTT 连接(超过 200K),并将它们分发到后端服务器的多个端口上。以下是我的 Nginx 配置。
load_module /usr/lib/nginx/modules/ngx_stream_module.so;
user www-data;
pid /run/nginx.pid;
worker_rlimit_nofile 204800;
worker_cpu_affinity auto;
worker_processes auto;
events {
accept_mutex on; # Multiple workers should accept connection
worker_connections 206200; # Total connections with upstream and clients
multi_accept off; # Accept one connection in each worker
}
stream {
upstream backend {
server backendserver.com:1885;
server backendserver.com:1886;
server backendserver.com:1887;
server backendserver.com:1888;
server backendserver.com:1889;
server backendserver.com:1890;
server backendserver.com:1891;
server backendserver.com:1892;
server backendserver.com:1893;
server backendserver.com:1894;
server backendserver.com:1895;
server backendserver.com:1896;
server backendserver.com:1897;
server backendserver.com:1898;
server backendserver.com:1899;
server backendserver.com:1900;
server backendserver.com:1901;
server backendserver.com:1902;
server backendserver.com:1903;
server backendserver.com:1904;
}
server {
listen 8883;
proxy_pass backend;
}
}
我还设置了以下系统配置:
sysctl -w fs.file-max=11000000
sysctl -w fs.nr_open=11000000
sysctl -w net.core.somaxconn=65535
sysctl -w net.ipv4.tcp_max_syn_backlog=65535
sysctl -w net.ipv4.ip_local_port_range="1025 65535"
sysctl -w net.ipv4.tcp_mem="100000000 100000000 100000000"
sysctl -w net.ipv4.tcp_rmem='2048 2048 2048'
sysctl -w net.ipv4.tcp_wmem='4096 4096 4096'
sysctl -w net.core.rmem_default=4096
sysctl -w net.core.wmem_default=4608
sysctl -w net.core.rmem_max=4096
sysctl -w net.core.wmem_max=4608
我也已设置ulimit -n 999999
。
但我只能创建大约 128K-130K 个连接。之后,Nginx 会记录如下错误:
2023/08/16 06:46:32 [error] 4172305#4172305: *206569 recv() failed (104: Unknown error) while proxying and reading from upstream, client: 72.xxx.xxx.xxx, server: 0.0.0.0:8885, upstream: "139.xxx.xxx.xxx:8889", bytes from/to client:984/5224, bytes from/to upstream:5224/984
我检查了直接连接到所有后端端口并且它们工作正常。
此外,当我的块中只有一个条目时upstream backend
,我只能创建大约 64K 个连接。当我添加第二个条目时,我可以达到大约 128K-130K。但是添加更多条目并不能让我超过该连接数。
我已经尝试过执行以下操作:
- 设定
worker_rlimit_nofile
和worker_connections
为 999999。 - 添加更多具有不同端口的服务器块。
- 10
tcp_rmem
倍tcp_wmem
,,,,,,。rmem_default
wmem_default
rmem_max
wmem_max
- 环境
multi_accept on;
如何让我的 Nginx 服务器代理超过 130K 个连接(至少 200K)?