我们的 Apache Kafka 服务器面临着非常大的问题，我们的 Kafka 服务器出现了褶皱，因为“打开的文件太多”
我们有生产 Kafka 集群7台机器，而 Kafka 版本是0.1
操作系统版本 - RHEL 7.5
我们已经将“kafka_user_nofile_limit“ 到50万，而原始值为128000
但我们再次遭遇高价，经纪商崩溃了
另外，我们可以清楚地看到 Kafka 日志中有许多行“打开文件过多”
我们看到了以下内容吉拉案件
https://issues.apache.org/jira/browse/KAFKA-3317
但我们没有看到这个 jira 的解决方案是什么，或者这个主要问题的解决方法是什么？
来自吉拉
Bug
Status:OPEN
Priority: Major
Resolution:Unresolved
Affects Version/s:
0.9.0.0, 0.9.0.1
Fix Version/s:None
Component/s:None
Labels:None
Environment:CentOS release 6.5 (Final)
kafka_2.11-0.9.0.1
Flags:
Important
Description
there is much open files in my java app client which write msg to kafka broker, i used
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.9.0.1</version>
</dependency>
new api to write msg. when i use
lsof -p 8780 | wc -l
19184
lsof -p 8780 | grep XmlIpcRegSvc | wc -l
4920
lsof -p 8780 | grep pipe | wc -l
9576
lsof -p 8780 | grep eventpoll | wc -l
4792
where 8780 is my java app pid.
java 37121 app *796u IPv6 960673997 0t0 TCP mad183:50213->mad180:XmlIpcRegSvc (ESTABLISHED)
there are many ESTABLISHED XmlIpcRegSvc , and seems not closed.
and It likes use ipv6 .
我的配置
more /etc/security/limits.d/kafka.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
kafka - nofile 500000
kafka - nproc 65536