[Fixed] Kafka Error org.apache.kafka.common.errors.NotLeaderOrFollowerException NOT_LEADER_OR_FOLLOWER – Apache-kafka

The Problem:

While trying to write to a newly setup Kafka cluster on localhost using the kafka-console-producer tool, an error message is displayed:

[2023-08-17 02:35:07,842] WARN [Producer clientId=console-producer] Got error produce response with correlation id 4 on topic-partition TutorialTopic-0, retrying (2 attempts left). Error: NOT_LEADER_OR_FOLLOWER (org.apache.kafka.clients.producer.internals.Sender)

…

[2023-08-17 02:35:08,151] ERROR Error when sending message to topic TutorialTopic with key: null, value: 12 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.NotLeaderOrFollowerException: For requests intended only for the leader, this error indicates that the broker is not the current leader. For requests intended for any replica, this error indicates that the broker is not a replica of the topic partition.

….

It was confirmed that both ports 9092 and 2181 are accessible using telnet. Additionally, the command kafka-topics –describe –topic TutorialTopic –bootstrap-server localhost:9092 shows that the topic TutorialTopic exists with 1 partition, 1 replication factor, and leader 0.

The Solutions:

Solution 1: Verify Topic Before Publishing Messages

Before publishing messages to a topic, ensure it exists using the `kafka-topics –create` command. This step is essential to establish a leader for the partition, enabling message publishing. Without this step, the producer will repeatedly attempt to send messages until the topic is created.

Additionally, it’s recommended to set `auto.topics.create=false` in a production environment to prevent automatic topic creation. This safeguard ensures topics are explicitly created and controlled, avoiding unexpected topic proliferation.