How to configure MongoDB official source connector for Kafka Connect running on a kubernetes cluster

My Kafka cluster runs on kubernetes and I am using a custom image to run Kafka Connect with required mongoDB official source and sink connectors.

My mongoDB instance also runs on kubernetes. My issue is, I am unable to connect my live DB with Kafka Connect.

My connector config currently looks like this,

curl -X PUT \
     -H "Content-Type: application/json" \
     --data '{
               "connector.class":"com.mongodb.kafka.connect.MongoSourceConnector",
                    "tasks.max": "1",
                    "connection.uri": "mongodb://192.168.190.132:27017,192.168.190.137:27017",
                    "database": "tractor",
                    "collection": "job",
                    "topic.prefix": "testing-mongo"
                    
          }' \
     http://10.108.202.171:8083/connectors/mongo_source_job/config

2 answers

  • answered 2020-09-24 19:51 alex

    Can you try connecting to MongoDB service using the service name?

    kubectl get service -n <namespace>
    

    Use the above to get the services in the namespace of MongoDB and use the service name instead of the Ip's you have and see if that works?

  • answered 2020-09-24 19:57 unitom

    Thanks for your reply. The issue was the stemming from TLS. I modified my config as follows,

    "connection.uri": "mongodb://192.168.190.132:27017,192.168.190.137:27017/?tlsInsecure=true"
    

    Its working now!