Setup
Prerequisite CLI tools
The following CLI tools are required for running the exercises in this tutorial. Please have them installed and configured before you get started with any of the tutorial chapters.
| Tool | macOS | Fedora | windows |
|---|---|---|---|
|
|||
|
|
||
|
|||
Minikube v1.25.1 |
|||
|
|||
|
|||
|
|||
|
Download or |
||
|
|||
|
IMPORTANT: An alternative for running |
The following CLI tools are optional for running the exercises in this tutorial. Although they are used in the tutorial, you could use others without any problem.
| Tool | macOS | Fedora | windows |
|---|---|---|---|
|
|
||
watch |
|
|
|
kubectx and kubens |
|
||
|
Download & Configure Tutorial Sources
Before we start setting up the environment, let’s clone the tutorial sources and set the TUTORIAL_HOME environment variable to point to the root directory of the tutorial:
git clone https://github.com/redhat-developer-demos/kafka-tutorial.git kafka-tutorialexport TUTORIAL_HOME="$(pwd)/kafka-tutorial"cd $TUTORIAL_HOMESetup Kafka
Assuming you’ve got Docker & Docker Compose installed.
cd $TUTORIAL_HOME/apps/it
docker-compose up --remove-orphans...
kafka_1 | [2020-04-27 05:51:56,560] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
kafka_1 | [2020-04-27 05:51:56,561] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
kafka_1 | [2020-04-27 05:51:56,597] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
kafka_1 | [2020-04-27 05:51:56,641] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
kafka_1 | [2020-04-27 05:51:56,656] INFO [SocketServer brokerId=0] Started data-plane processors for 1 acceptors (kafka.network.SocketServer)
kafka_1 | [2020-04-27 05:51:56,666] INFO Kafka version: 2.4.0 (org.apache.kafka.common.utils.AppInfoParser)
kafka_1 | [2020-04-27 05:51:56,666] INFO Kafka commitId: 77a89fcf8d7fa018 (org.apache.kafka.common.utils.AppInfoParser)
kafka_1 | [2020-04-27 05:51:56,667] INFO Kafka startTimeMs: 1587966716657 (org.apache.kafka.common.utils.AppInfoParser)
kafka_1 | [2020-04-27 05:51:56,669] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)Open a new terminal to run Kafka commands:
Verify that Zookeeper and Apache Kafka containers are up and running:
docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5152b0eabe99 strimzi/kafka:0.17.0-kafka-2.4.0 "sh -c 'bin/kafka-se…" 43 seconds ago Up 42 seconds 0.0.0.0:9092->9092/tcp, 0.0.0.0:29092->29092/tcp kafka
e3417f63ac61 strimzi/kafka:0.17.0-kafka-2.4.0 "sh -c 'bin/zookeepe…" 43 seconds ago Up 43 seconds 0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp it_zookeeper_1| Notice that Kafka container is exposing two ports to outside. The reason is that Kafka requires one port for accessing from host machine, and another one for accessing from Docker host machine. You can find a full explanation of this at Kafka Listeners. |
Verify that the Kafka cluster is correctly started and configured by running the following command:
kcat -b localhost:9092 -Ldocker run -it --network=host edenhill/kcat:1.7.0 kcat -b localhost:29092 -LMetadata for all topics (from broker 0: localhost:9092/0):
1 brokers:
broker 0 at localhost:9092 (controller)
0 topics: