kafka-python no broker available with kafka on localhost

Revision history
Tags: kafka python docker

All of a sudden I was having problems getting a script based on kafka-python to work properly. Even the example code from the upstrem repo didn’t work. However, I still had success with my Node.js scripts, so the problem had to be with the Python lib.

Immediately when creating a new KafkaConsumer or KafkaProducer an exception was thrown stating

No brokers available

Both the consumer and producer constructors in kafka-python accepts a long list of keyword arguments, one of which is api_version. Apparently, when no api_version is set, an attempt is made to automatically detect the kafka version of the broker.

The stack trace printed along with the exception also hints about where the exception originated from, and it is when attempting to detect a version.

Specify the version explicitly, and keep on programming

KAFKA_HOSTS = ['localhost:9092']
KAFKA_VERSION = (0, 10)
producer = KafkaProducer(bootstrap_servers=KAFKA_HOSTS, api_version=KAFKA_VERSION)

topic = "foobars"
consumer = KafkaConsumer(topic, bootstrap_servers=KAFKA_HOSTS, api_version=KAFKA_VERSION)

This may be a problem between kafka-python and the current Kafka installation I am using within a docker image, wurstmeister/kafka-docker, but I have not bothered to confirm this at this time. At least it works!

References

If you have any comments or feedback, please send me an e-mail. (stig at stigok dotcom).

Did you find any typos, incorrect information, or have something to add? Then please propose a change to this post.

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.