Kafka and ZooKeeper contains Podman

Introduction

Apache Kafka and ZooKeeper is a distributed data store with optimization for ingesting and processing streaming data. Streaming data generated thousands. A streaming platform needs to handle this constant influx of data sequentially and incrementally process the data. Visit Apache Kafka’s site for more info. Also, try the tutorial Apache Kafka WebUI for those who want Web interface.

Kafka and ZooKeeper provide main functions to their users:

  • Publish and subscribe the streams of records
  • Effectively store streams data records in the order in which they were generated
  • Process streams of records in real-time

Kafka primarily builds real-time streaming data pipelines and applications that adapt to the data streams. It combines messaging, storage, and stream processing to allow storage and analysis of historical and real-time data. 

Zookeeper is distributed systems configuration managing tool. It delivers multiple features for broadcast applications like distributed configuration management, leader election, consensus handling, coordination, locks, etc. Zookeeper stores its data on tree-like structures. It was introduced as Data Tree, and nodes were presented as nodes. Zookeeper follows standard Unix notations for file paths.

Pull Kafka and Zookeeper images

Use pull command from podman options.

# podman pull bitnami/kafka
# podman pull bitnami/zookeeper

Create zookeeper container pod

# podman pod create --name kafkapp -p 2181:2181 -p 9092:9092 --network bridge

Launch the Zookeeper server instance

Use the --pod kafkapp argument to the podman run command to attach the Zookeeper container to the network from the pod bridge.

# podman run --pod kafkapp --name zookeeper-server -e ALLOW_ANONYMOUS_LOGIN=yes -d bitnami/zookeeper:latest

Launch the Apache Kafka server instance

Use the --pod kafkapp argument to the podman run command to attach the Apache Kafka container to the network from the pod bridge.

# podman run --pod kafkapp --name kafka-server -e ALLOW_PLAINTEXT_LISTENER=yes -e KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper-server:2181 -d bitnami/kafka:latest

Launch your Apache Kafka and ZooKeeper client instance

Finally, we create a new container instance to launch the Apache Kafka client and connect to the server created in the previous step:

# podman run -it --rm --pod kafkapp -e KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper-server:2181 bitnami/kafka:latest kafka-topics.sh --list  --bootstrap-server kafka-server:9092
Neil
Treat your password like your toothbrush. Don’t let anybody else use it, and get a new one every six months.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest articles

You might also likeRELATED

1 DNS server container Podman dirty easy

Introduction Linux distributions. So, what is a DNS? A DNS server is a service that helps resolve a fully qualified domain name (FQDN) into an IP address...

1 Nagios xi server container Fast and Easy

Introduction Provides monitoring of all mission-critical infrastructure components, including applications, services, operating systems, network protocols, systems metrics, and network infrastructure. In addition, hundreds of third-party...

Remote VS Code fast and dirty easy 100%

Introduction Visual Studio Code or VS Code is a lightweight but powerful source code editor which runs on your desktop and is available for Windows,...