Ignite YARN deployment - how to use TCP IP Discovery?

classic Classic list List threaded Threaded
2 messages Options
Seshan, Manoj N. (TR Tech, Content & Ops) Seshan, Manoj N. (TR Tech, Content & Ops)
Reply | Threaded
Open this post in threaded view
|

Ignite YARN deployment - how to use TCP IP Discovery?

We are using Ignite as a Distributed In-Memory cache, deployed using YARN on a Hadoop Cluster.  We have configured Zookeeper Discovery, and this is working fine.

 

Given this is a small 20 node Ignite cluster, Zookeeper Discovery seems overkill. Would it be possible to switch to TCP Discovery? Multicast Finding is not an option, as that is disabled. Static IP Finding would also not work, as the Ignite Containers are dynamically allocated by YARN to arbitrary nodes of the Hadoop Cluster.

 

Rgds

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

aealexsandrov aealexsandrov
Reply | Threaded
Open this post in threaded view
|

Re: Ignite YARN deployment - how to use TCP IP Discovery?

Hi,

I guess you can try collect all IP addresses from all nodes that managed by YARN (where you are going to start Ignite) and add them all to addresses into TcpDiscoveryVmIpFinder part of Ignite configuration.

Also you should provide the guarantee that each such hosts will be able to connect each other.

BR,
Andrei

11/5/2019 9:15 PM, Seshan, Manoj N. (TR Tech, Content & Ops) пишет:

We are using Ignite as a Distributed In-Memory cache, deployed using YARN on a Hadoop Cluster.  We have configured Zookeeper Discovery, and this is working fine.

 

Given this is a small 20 node Ignite cluster, Zookeeper Discovery seems overkill. Would it be possible to switch to TCP Discovery? Multicast Finding is not an option, as that is disabled. Static IP Finding would also not work, as the Ignite Containers are dynamically allocated by YARN to arbitrary nodes of the Hadoop Cluster.

 

Rgds

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572