Issue in starting IgniteSinkConnector with Distributed Worker mode

classic Classic list List threaded Threaded
4 messages Options
rishi007bansod rishi007bansod
Reply | Threaded
Open this post in threaded view
|

Issue in starting IgniteSinkConnector with Distributed Worker mode

This post was updated on .
Hi,
As mentioned in post, https://dzone.com/articles/linking-apache-ignite-and-apache-kafka-for-highly, I am able to use connect-standalone.sh for ignite kafka connection. But now for scaling up these connectors across  cluster I am trying to use connect-distributed.sh from kafka. I have also attached *.properties files I am using.
     But in case of connect-distributed.sh i.e. distributed worker mode Ignite instance is not getting started at all. What might be issue in this case? Is kafka Distributed Worker mode supported in Ignite?      

connect-distributed.properties                     
ignite-connector.properties
dkarachentsev dkarachentsev
Reply | Threaded
Open this post in threaded view
|

Re: Issue in starting IgniteSinkConnector with Distributed Worker mode

Hi,

I'm not quite familiar with Kafka connector, but I'm going to figure it out and come back to help you.

Thanks!

-Dmitry.
dkarachentsev dkarachentsev
Reply | Threaded
Open this post in threaded view
|

Re: Issue in starting IgniteSinkConnector with Distributed Worker mode

In reply to this post by rishi007bansod
Hi,

Could you please attach logs and thread dumps (in case of hang) if problem is still relevant?

-Dmitry.
Roman Shtykh Roman Shtykh
Reply | Threaded
Open this post in threaded view
|

Re: Issue in starting IgniteSinkConnector with Distributed Worker mode

The connector is not created and no connection to your Ignite cluster is done in a distributed mode, if you pass connector configuration on the command line.
In a distributed mode, you have to use REST API, like below.

$ curl -X POST -H "Content-Type: application/json" --data '{"name": "string-ignite-connector", "config": {"connector.class":"org.apache.ignite.stream.kafka.connect.IgniteSinkConnector", "tasks.max":"2","topics":"test","cacheName":"cache1","igniteCfg":"path_to_/ignite.xml"}}' http://localhost:8083/connectors

Hope this helps.

Roman


On Tuesday, March 14, 2017 4:45 PM, dkarachentsev <[hidden email]> wrote:


Hi,

Could you please attach logs and thread dumps (in case of hang) if problem
is still relevant?

-Dmitry.



--
Sent from the Apache Ignite Users mailing list archive at Nabble.com.