How does schema evolution in confluent schema registry work?

I've configured my confluent schema registry's compatibility to BACKWARD_TRANSITIVE. I'm using confluent jdbc connector to pull incremental changes from MySQL DB. Suppose my initial schema v1 looks like this

{"connect.name": "customers",
 "type": "record",
 "name": "user",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": "int"},
     {"name": "favorite_color", "type": "string", "default": null}
 ]
}

I perform following actions on the customers table in my database and schedule jdbc connect to pull data from database after each action (I also pushed some DML after each action to capture events into Kafka topic)

  1. I drop favorite_color (optional) column from customers table
  2. I add a new (optional) column address to customers table

When 1 is performed I observe no change in my AVRO schema. However, when action 2 is performed, I notice that the schema version bumps up to v2 with below changes

{"connect.name": "customers",
 "type": "record",
 "name": "user",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": "int"},
     {"name": "address", "type": "string", "default": null}
 ]
}

favorite_color column got dropped from schema and address column got introduced. I noticed similar behaviour for non optional columns as well.

  • Why didn't schema registry update the schema when action 1 was performed?
  • Why did it update when 2 was performed?
  • Could you please walk me through the kind of schema changes gets updated eagerly and the ones updated lazily?

The behaviour is same irrespective of compatibility types.