Struggling with Kafka Connect

I have been working on deploying Kafka on a Kubernetes cluster using the Confluent platform and it has been much harder than I expected.

Deploying their zookeeper in a StatefulSet seemed impossible so I had to switch to using the Google version of zookeeper. I just could not get it to deploy with a serverId that was unique and the Google example made this trivial.

Kafka was easily deployed once Zookeeper was working.

Kafka Connect has been very difficult and I have tried quite a few combinations. Nothing seems to work. Basically, it times out.

If you want to see the latest update to the YAML and some notes to this struggle I have it on Github: https://github.com/cinq/confluent-kafka-k8s

 

Advertisements

Published by

m5c

Java developper that loves photography and good coffee

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s