DC replication via Kafka Connect for Apache Ignite

classic Classic list List threaded Threaded
2 messages Options
adipro adipro
Reply | Threaded
Open this post in threaded view
|

DC replication via Kafka Connect for Apache Ignite

We are trying to implement the replication of data between two Ignite data
clusters.

For this purpose, we are using Kafka Connect.

We have followed the things mentioned in this document ->
https://dzone.com/articles/linking-apache-ignite-and-apache-kafka-for-highly

Everything is working fine till we use one cache and PUT operation.

But when I use the same for REMOVED operation, in the consumer thread of the
connector, I can see the CacheEvent record, but the data is not being
removed from the Sink Cluster nodes.

When checked the remove functionality is missing in IgniteSinkTask class.

Is IgniteSink connector only for injecting data? The data record type at
receiving end is of type CacheEvent. Based on operation it should put or
remove right? As of now only PUT is working.

Can someone please help with this case?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ibelyakov ibelyakov
Reply | Threaded
Open this post in threaded view
|

Re: DC replication via Kafka Connect for Apache Ignite

Ignite Kafka sink connector doesn't support removals processing. I've created
a ticket to implement this improvement:
https://issues.apache.org/jira/browse/IGNITE-13442

But, as I know, such functionality is implemented in GridGain Kafka
Connector, you can find more details regarding it and data replication,
here:
https://www.gridgain.com/docs/latest/integrations/kafka/kc-ex-replication

Igor



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/