我有一个在 hyperv vm 上运行的 minikube K8S 集群。该集群有一个 nginx 入口控制器，用于将服务暴露给集群外部。集群外部的请求通过主机上运行的 nginx 暴露给互联网。
在集群内部，我还有一个 kube-registry，其中 kube-registry-proxy 正在运行，用于存储我的 docker 镜像。
但是，当我尝试推送较大的图像（约 32mb）时，会出现问题。在这种情况下，我收到 413 错误，并在主机 nginx 日志文件中出现以下错误（出于隐私原因已删除）：
<redactedIP> - - [<redactedLocalDateTime>] "PATCH /v2/<redactedImageName>/blobs/uploads/<redactedUuid>?_state=<redactedState>%3D HTTP/1.1" 413 183 "-" "docker/19.03.12 go/go1.13.10 git-commit/48a66213fe kernel/4.19.114 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.12 \x5C(windows\x5C))"
当我查看 k8s 集群中的 ingress-nginx-controller 日志时，我看到以下消息：
[error] 371#371: *1004283 client intended to send too large body: 9930410 bytes, client: 127.0.0.1, server: , request: "PATCH /v2/<redactedImageName>/blobs/uploads/<redactedUuid> HTTP/1.1", host: ".<redactedHost>"
查看我的 kube-registry 日志，我根本看不到 PATCH 请求。这说明问题出在我集群中的 ingress-nginx-controller 上。
我研究了该问题并发现了一些线索，关于使用 nginx configmap设置client_max_body_size和：client_body_buffer_size
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx-controller
namespace: ingress-nginx
data:
body-size: "1024m"
proxy-body-size: "1024m"
client-max-body-size: "1024m"
client-body-buffer-size: "1024m"
这是在 ingress-nginx-controller 的部署中使用参数设置--configmap的
我还尝试使用 kube-registry 入口处的注释来设置它：
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
labels:
app: kube-registry-ingress
name: kube-registry-ingress
namespace: kube-system
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-body-size: "1024m"
nginx.org/client-max-body-size: "1024m"
...
我还确保client_max_body_size并client_body_buffer_size在主机 nginx 配置中设置。
我验证了来自 ingress-nginx-controller 的 nginx 配置已设置：
http {
client_body_buffer_size 1024m;
...
## start server <redactedDomain>
server {
server_name <redactedDomain> ;
listen 80 ;
listen 443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "kube-system";
set $ingress_name "kube-registry-ingress";
set $service_name "kube-registry-proxy";
set $service_port "80";
set $location_path "/";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "kube-system-kube-registry-proxy-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1024m;
...
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
proxy_buffer_size 8k;
proxy_buffers 4 8k;
proxy_max_temp_file_size 1024m;
proxy_request_buffering on;
...
}
}
## end server <redactedDomain>
但是，这似乎没有帮助。我client-max-body-size也尝试过设置为 0。
我的问题是：如何防止最有可能来自 ingress-nginx-controller 的 413 HTTP 状态代码？
答案1
这可以在Docker 文档：
http {
...
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
...
}
因此看来解决方案就是关闭限制。