☰

Apache Kafka 单节点 Docker-如何将侦听器设置为 Docker 主机名

tag-icon • tag-icon docker kafka
Apache Kafka 单节点 Docker-如何将侦听器设置为 Docker 主机名

我正在使用 Apache Kafka 3.7.0docker 镜像，如下所述这里，并进行了一些细微的调整：

docker run --network=internal --name my-kafka apache/kafka:3.7.0

我的意图是仅在内部网络内访问它，并通过其主机名访问它my-kafka。就此命令而言，它似乎运行良好，并且根据日志开箱即用：

INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper)
INFO Initialized snapshots with IDs SortedSet() from /tmp/kraft-combined-logs/__cluster_metadata-0 (kafka.r>
INFO [LogLoader partition=__cluster_metadata-0, dir=/tmp/kraft-combined-logs] Producer state recovery took >
INFO [LogLoader partition=__cluster_metadata-0, dir=/tmp/kraft-combined-logs] Reloading from producer snaps>

但是，默认情况下，广告侦听器仅针对配置localhost，从而断开来自客户端（通过主机名连接my-kafka）的连接。我尝试设置一些环境变量来设置具有相关主机名的新侦听器：

docker run --network=internal --name my-kafka -e KAFKA_LISTENERS=LISTENER_FRED://my-kafka:9092 -e KAFKA_ADVERTISED_LISTENERS=LISTENER_FRED://my-kafka:9092 -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=LISTENER_FRED:PLAINTEXT apache/kafka:3.7.0

这会导致异常：Exception in thread "main" org.apache.kafka.common.config.ConfigException: Missing required configuration 'zookeeper.connect'。它似乎尝试使用 Zookeeper 而不是 Kraft 运行，但仅在提供了环境变量时才会发生？

据我所知，我正确设置了广告监听器，并且根据最新的docker镜像，其行为与其他人的预期略有不同？我该怎么做才能让我的Kafka服务器正常工作？

答案1

也许这个示例 docker-compose 可以帮助您处理环境变量：

https://github.com/apache/kafka/blob/trunk/docker/examples/jvm/single-node/plaintext/docker-compose.yml

我在尝试使用以下配置在 k3d 集群内配置 kafka 部署（仅用于本地开发）时遇到了类似的问题（请注意服务和端口：kafka.kafka.svc:9094由集群内的消费者使用）。

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-properties
  namespace: kafka
data:
  KAFKA_NODE_ID: '1'
  KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
  KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT_HOST://localhost:9092,PLAINTEXT://kafka.kafka.svc:9094'
  KAFKA_PROCESS_ROLES: 'broker,controller'
  KAFKA_CONTROLLER_QUORUM_VOTERS: '1@localhost:29093'
  KAFKA_LISTENERS: 'CONTROLLER://:29093,PLAINTEXT_HOST://:9092,PLAINTEXT://:9094'
  KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
  KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
  CLUSTER_ID: '4L6g3nShT-eMCtK--X86sw'
  KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: '1'
  KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: '0'
  KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: '1'
  KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: '1'
  KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka
  namespace: kafka
  labels:
    app: kakfa
spec:
  replicas: 1

  revisionHistoryLimit: 3

  selector:
    matchLabels:
      app: kakfa

  template:
    metadata:
      labels:
        app: kakfa

    spec:
      containers:
        - name: server
          image: apache/kafka:3.7.0
          imagePullPolicy: IfNotPresent

          envFrom:
            - configMapRef:
                name: kafka-properties

          ports:
            - containerPort: 9092
              name: internal

            - containerPort: 9094
              name: external

          resources:
            requests:
              memory: "64Mi"
              cpu: "500m"

            limits:
              memory: "5Gi"
              cpu: "3"
---
apiVersion: v1
kind: Service
metadata:
  name: kafka
  namespace: kafka
  labels:
    app: kakfa
spec:
  selector:
    app: kakfa

  ports:
    - name: internal
      port: 9092
      protocol: TCP
      targetPort: 9092

    - name: external
      port: 9094
      protocol: TCP
      targetPort: 9094

相关内容