I have been working on deploying Kafka on a Kubernetes cluster using the Confluent platform and it has been much harder than I expected.
Deploying their zookeeper in a StatefulSet seemed impossible so I had to switch to using the Google version of zookeeper. I just could not get it to deploy with a serverId that was unique and the Google example made this trivial.
Kafka was easily deployed once Zookeeper was working.
Kafka Connect has been very difficult and I have tried quite a few combinations. Nothing seems to work. Basically, it times out.
If you want to see the latest update to the YAML and some notes to this struggle I have it on Github: https://github.com/cinq/confluent-kafka-k8s
Advertisements
