Kafdrop – WebUI for Kafka

Kafdrop is a webUI for Apache Kafka. Kafdrop is an open-source tool which displays the information like broker details, creates a topic, delete a topic, browses messages and view ACLs. It is a lightweight tool and very easy to set up.

What is Kafka?

Apache Kafka is an open-source platform. Kafka was originally developed by Linkedin and was later incubated as the Apache Project. It can process over 1 million messages per second.

Need for Kafdrop:

Kafka is an amazing platform for processing a huge number of messages very quickly. However, Kafka has one disadvantage that it does not come with an inbuilt User Interface where the users can see the information related to Kafka.

Kafdrop helps us in solving this problem. It gives us a simple, lightweight, and easy-to-use User Interface where one can not only see the required information but can also create and delete Kafka topics.

Steps to install Kafdrop:

Prerequisites:

To install Kafdrop you need the following:

• Kafka 2.0 or later
• Zookeeper 3.4.5 or later
• Java 8

The source code for Kafdrop is available at https://github.com/obsidiandynamics/kafdrop

Kafdrop can be installed by executing a JAR file or via docker or on Kubernetes.

In this tutorial we will learn about installing Kafdrop via docker or on Kubernetes.

KafDrop Via Docker:

KafDrop Images are available at: hub.docker.com/r/obsidiandynamics/kafdrop.

To launch the Kafdrop run the following command:

docker run -d --rm -p 9000:9000 \
    -e KAFKA_BROKERCONNECT=<kafka_broker_ip:port,kafka_broker_ip:port> \
    -e JVM_OPTS="-Xms32M -Xmx64M" \
    -e SERVER_SERVLET_CONTEXTPATH="/" \
    obsidiandynamics/kafdrop

Now, you can access the Kafdrop UI by opening http://localhost:9000 in your browser.

KafDrop via K8S Manifest file:

You can also install KafDrop on the Kubernetes cluster with the helm of the manifest file. Create a YAML file called kafdrop-deployment.yaml with the following content in it:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-kafdrop-deployment
  namespace: "kafdrop"
  labels:
    app: kafka-kafdrop
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-kafdrop
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: kafka-kafdrop
    spec:
      volumes:
        - name: tz-config
          hostPath:
            path: /usr/share/zoneinfo/Asia/Kolkata
      containers:
        - image: obsidiandynamics/kafdrop
          imagePullPolicy: Always
          name: kafka-kafdrop
          volumeMounts:
            - name: tz-config
              mountPath: /etc/localtime
          resources:
            limits:
              cpu: 200m
              memory: 1Gi
            requests:
              cpu: 200m
              memory: 1Gi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          ports:
            - containerPort: 5010
              name: server
            - containerPort: 5012
              name: jmx
          env:
            - name: JVM_OPTS
              value: "-Xms512M -Xms512M"
            - name: SERVER_SERVLET_CONTEXTPATH
              value: "/"
            - name: KAFKA_BROKERCONNECT
              value: "<kafka_broker_ip>:9092"
      restartPolicy: Always

To create the deployment run the following command:

kubectl apply -f kafdrop-deployment.yaml

By default, the topic creation and deletion are enabled via KafDrop. If you want to disable topic creation and topic deletion, add the following in the env section of the YAML file:

- name: CMD_ARGS
  value: "--topic.deleteEnabled=false --topic.createEnabled=false"

To access the Kafdrop UI from the browser of your local machine you need to create a Kubernetes service that will point to the deployment create in the previous step. Create kafdrop-service.yaml and add the following lines in it:

---
apiVersion: v1
kind: Service
metadata:
  name: kafka-kafdrop-service
  namespace: "kafdrop"
  labels:
    app: kafka-kafdrop
  annotations:
    cloud.google.com/load-balancer-type: "Internal"
spec:
  ports:
    - protocol: "TCP"
      port: 9000
      name: server
  selector:
    app: kafka-kafdrop
  type: LoadBalancer

To create the service run the following command:

kubectl apply -f kafdrop-service.yaml

Wait for the service to come up. Once the service is started you can access the Kafdrop UI from the browser at http://<kubernetes_service_ip>:9000

With this your KafDrop is up and running.

Check Details of Kafka Cluster:

To check the details about the cluster in the browser, it will show the details like total topics, topic name, partitions, broker details as shown in the below image:

Check details of Kafka Topic:

On the KafDrop Home page click on any Kafka topic for which you want to check the details. It will open up a page with details like partitions count, replication factor, offset lag, under-replicated partition as shown in the below image

To read about how to create a Kafka HA cluster please refer to Kafka and Zookeeper HA Cluster Setup.

Cheers !!