org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms

We Are Going To Discuss About org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms. So lets Start this Java Article.

org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms

  1. org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms

    I was having this same problem today. I'm a newbie at Kafka and was simply trying to get a sample Java producer and consumer running. I was able to get the consumer working, but kept getting the same “topic not present in metadata” error as you, with the producer.

  2. org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms

    I was having this same problem today. I'm a newbie at Kafka and was simply trying to get a sample Java producer and consumer running. I was able to get the consumer working, but kept getting the same “topic not present in metadata” error as you, with the producer.

Solution 1

I was having this same problem today. I’m a newbie at Kafka and was simply trying to get a sample Java producer and consumer running. I was able to get the consumer working, but kept getting the same “topic not present in metadata” error as you, with the producer.

Finally, out of desperation, I added some code to my producer to dump the topics. When I did this, I then got runtime errors because of missing classes in packages jackson-databind and jackson-core. After adding them, I no longer got the “topic not present” error. I removed the topic-dumping code I temporarily added, an it still worked.

Advertisements

Original Author Of This Content

Solution 2

First off I want to say thanks to Bobb Dobbs for his answer, I was also struggling with this for a while today. I just want to add that the only dependency I had to add is jackson-databind. This is the only dependency I have in my project, besides kafka-clients.

Update: I’ve learned a bit more about what’s going on. kafka-clients sets the scope of its jackson-databind dependency as “provided,” which means it expects it to be provided at runtime by the JDK or a container. See this article for more details on the provided maven scope.

This scope is used to mark dependencies that should be provided at runtime by JDK or a container, hence the name.
A good use case for this scope would be a web application deployed in some container, where the container already provides some libraries itself.

I’m not sure the exact reasoning on setting its scope to provided, except that maybe this library is something people normally would want to provide themselves to keep it up to the latest version for security fixes, etc.

Advertisements

Original Author Of This Content

Solution 3

This error also can appear because of destination Kafka instance “died” or URL to it is wrong.

In such a case a thread that sends message to Kafka will be blocked on max.block.ms time which defaults exactly to 60000 ms.

You can check whether it is because of above property by passing changed value:

Properties props = new Properties();
...(among others)
props.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 30000); # 30 sec or any other value of your choice 

If TimeoutException is thrown after your specified time, then you should check whether your URL to Kafka is correct or Kafka instance is alive.

Advertisements

Original Author Of This Content

Solution 4

I also had similar issue, where I was trying this on my local environment on my macbook. It was quite frustrating and I tried a few approaches

  1. Stopped Zookeeper, Stopped Kafka, restarted ZK and Kafka. (Didn’t help)
  2. Stopped ZK. Deleted ZK data directory. Deleted Kafka logs.dirs and restarted Kafka (Didn’t help)
  3. Restarted my macbook – This did the trick.

I have used Kafka in production for more than 3 years, but didn’t face this problem on the cluster, happened only on my local environment. However, restarting fixes it for me.

Advertisements

Original Author Of This Content

Conclusion

Advertisements

So This is all About This Tutorial. Hope This Tutorial Helped You. Thank You.

Also Read,

Siddharth

I am an Information Technology Engineer. I have Completed my MCA And I have 4 Year Plus Experience, I am a web developer with knowledge of multiple back-end platforms Like PHP, Node.js, Python and frontend JavaScript frameworks Like Angular, React, and Vue.

Leave a Comment