How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

classic Classic list List threaded Threaded
4 messages Options
Seshan, Manoj N. (TR Tech, Content & Ops) Seshan, Manoj N. (TR Tech, Content & Ops)
Reply | Threaded
Open this post in threaded view
|

How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

Please help, on how we can achieve this with the Ignite Yarn client

 

  1. How exactly do we specify Cluster1 & Cluster2? Would we use the “igniteInstanceName” property & Spring Configuration

<property name="igniteInstanceName" value="Cluster1"/>

 

2)  How would the Ignite Client specify which Cluster (Cluster1 or Cluster2) to work with?

 

Rgds

 

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

Hello!

You can start two clusters if they have different Discovery settings, e.g., one cluster is active on ports 47500..47520, other on 48500..47520.

You should configure clients accordingly. You can also change connector configurations for thin clients' connecting.

Regards,
--
Ilya Kasnacheev


пн, 30 сент. 2019 г. в 13:31, Seshan, Manoj N. (TR Tech, Content & Ops) <[hidden email]>:

Please help, on how we can achieve this with the Ignite Yarn client

 

  1. How exactly do we specify Cluster1 & Cluster2? Would we use the “igniteInstanceName” property & Spring Configuration

<property name="igniteInstanceName" value="Cluster1"/>

 

2)  How would the Ignite Client specify which Cluster (Cluster1 or Cluster2) to work with?

 

Rgds

 

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

Seshan, Manoj N. (TR Tech, Content & Ops) Seshan, Manoj N. (TR Tech, Content & Ops)
Reply | Threaded
Open this post in threaded view
|

RE: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

Thanks. A couple of follow-up questions:

 

  1. There doesn’t seem to be a way to specify the Port Range, without specifying IP addresses? Could you help more specifically?
  2. How would we support multiple clusters with the Zookeeper Discovery method?
  3. Could we use different MultiCast Groups, or a different Multicast Port to setup the multiple clusters?

 

Rgds

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

From: Ilya Kasnacheev <[hidden email]>
Sent: Monday, September 30, 2019 4:16 PM
To: [hidden email]
Subject: Re: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

 

Hello!

 

You can start two clusters if they have different Discovery settings, e.g., one cluster is active on ports 47500..47520, other on 48500..47520.

 

You should configure clients accordingly. You can also change connector configurations for thin clients' connecting.

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 13:31, Seshan, Manoj N. (TR Tech, Content & Ops) <[hidden email]>:

Please help, on how we can achieve this with the Ignite Yarn client

 

  1. How exactly do we specify Cluster1 & Cluster2? Would we use the “igniteInstanceName” property & Spring Configuration

<property name="igniteInstanceName" value="Cluster1"/>

 

2)  How would the Ignite Client specify which Cluster (Cluster1 or Cluster2) to work with?

 

Rgds

 

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

Hello!

1. Yes, you should do both.
2. I'm not sure, I guess you will have to supply different Zk clusters.
3. It should probably work, but testing is needed.

Regards,
--
Ilya Kasnacheev


вт, 1 окт. 2019 г. в 17:00, Seshan, Manoj N. (TR Tech, Content & Ops) <[hidden email]>:

Thanks. A couple of follow-up questions:

 

  1. There doesn’t seem to be a way to specify the Port Range, without specifying IP addresses? Could you help more specifically?
  2. How would we support multiple clusters with the Zookeeper Discovery method?
  3. Could we use different MultiCast Groups, or a different Multicast Port to setup the multiple clusters?

 

Rgds

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572

 

From: Ilya Kasnacheev <[hidden email]>
Sent: Monday, September 30, 2019 4:16 PM
To: [hidden email]
Subject: Re: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

 

Hello!

 

You can start two clusters if they have different Discovery settings, e.g., one cluster is active on ports 47500..47520, other on 48500..47520.

 

You should configure clients accordingly. You can also change connector configurations for thin clients' connecting.

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 13:31, Seshan, Manoj N. (TR Tech, Content & Ops) <[hidden email]>:

Please help, on how we can achieve this with the Ignite Yarn client

 

  1. How exactly do we specify Cluster1 & Cluster2? Would we use the “igniteInstanceName” property & Spring Configuration

<property name="igniteInstanceName" value="Cluster1"/>

 

2)  How would the Ignite Client specify which Cluster (Cluster1 or Cluster2) to work with?

 

Rgds

 

 

Manoj Seshan - Senior Architect

Platform Content Technology, Bangalore

cid:image001.gif@01C95541.6801BF70

Voice: +91-98806 72987  +91-80-67492572