Starting many Ignite nodes at once

classic Classic list List threaded Threaded
6 messages Options
Aleksei Valikov Aleksei Valikov
Reply | Threaded
Open this post in threaded view
|

Starting many Ignite nodes at once

Hi,

is it possible to start several Ignite node at once?

When I execute ./ignite.sh I'm getting just 1 node started. Assume I'm having a large machine with, say, 16 CPU Cores and I'd like to start 16 nodes. Is it possible via configuration?

What would be even better, if I could start N-M (for instance 16-1=15) nodes, where N is the total number of CPU Cores available and M is a number of CPU Cores I'd like to reserve for management purposes.

We're going to execute some third-party software which is (a) not intrinsically parallelizable (just uses 1 CPU Core) and (b) uses 100% CPU when it runs. So if we have a 16-Core machine, we'd like to be able to start this software 15 times (via 15 Ignite nodes) and resevre 1 CPU for management tasks like health checks, monitoring and so on.

Is there a way to do it? Or should we just start ignite.sh 15 times and script around?

Best wishes,
Alexey
Alexey Kuznetsov Alexey Kuznetsov
Reply | Threaded
Open this post in threaded view
|

Re: Starting many Ignite nodes at once

You could write a simple .sh file that will run ignite.sh in loop?

On Fri, Jul 31, 2015 at 3:12 PM, Aleksei Valikov <[hidden email]> wrote:
Hi,

is it possible to start several Ignite node at once?

When I execute ./ignite.sh I'm getting just 1 node started. Assume I'm having a large machine with, say, 16 CPU Cores and I'd like to start 16 nodes. Is it possible via configuration?

What would be even better, if I could start N-M (for instance 16-1=15) nodes, where N is the total number of CPU Cores available and M is a number of CPU Cores I'd like to reserve for management purposes.

We're going to execute some third-party software which is (a) not intrinsically parallelizable (just uses 1 CPU Core) and (b) uses 100% CPU when it runs. So if we have a 16-Core machine, we'd like to be able to start this software 15 times (via 15 Ignite nodes) and resevre 1 CPU for management tasks like health checks, monitoring and so on.

Is there a way to do it? Or should we just start ignite.sh 15 times and script around?

Best wishes,
Alexey



--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
Aleksei Valikov Aleksei Valikov
Reply | Threaded
Open this post in threaded view
|

Re: Starting many Ignite nodes at once

Hi,

well, this is what I meand with "script around".

I just thought it might have been a natural feature in Ignite. And maybe nodes may profit from being started in one JVM.

But as I understand, this is not the case, there is no OOTB feature for this?

Best wishes,
Alexey

On Fri, Jul 31, 2015 at 11:48 AM, Alexey Kuznetsov <[hidden email]> wrote:
You could write a simple .sh file that will run ignite.sh in loop?

On Fri, Jul 31, 2015 at 3:12 PM, Aleksei Valikov <[hidden email]> wrote:
Hi,

is it possible to start several Ignite node at once?

When I execute ./ignite.sh I'm getting just 1 node started. Assume I'm having a large machine with, say, 16 CPU Cores and I'd like to start 16 nodes. Is it possible via configuration?

What would be even better, if I could start N-M (for instance 16-1=15) nodes, where N is the total number of CPU Cores available and M is a number of CPU Cores I'd like to reserve for management purposes.

We're going to execute some third-party software which is (a) not intrinsically parallelizable (just uses 1 CPU Core) and (b) uses 100% CPU when it runs. So if we have a 16-Core machine, we'd like to be able to start this software 15 times (via 15 Ignite nodes) and resevre 1 CPU for management tasks like health checks, monitoring and so on.

Is there a way to do it? Or should we just start ignite.sh 15 times and script around?

Best wishes,
Alexey



--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Alexey Kuznetsov Alexey Kuznetsov
Reply | Threaded
Open this post in threaded view
|

Re: Starting many Ignite nodes at once

1) Starting several nodes in one JVM usually useful for testing not in production

2) If you need to launch several nodes at onces OOTB you could create an issue in Ignite JIRA and submit a patch (or wait for someone to do it).

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Starting many Ignite nodes at once

Aleksei,

You can use Visor command line tool for this. It can start nodes both on local machine and on remote hosts via SSH.

Start Visor:

  ./ignitevisorcmd.sh

See help for 'start' command:

  help start

-Val

P.S. Alexey, I'm really surprised you haven't suggested this ;-)
Alexey Kuznetsov Alexey Kuznetsov
Reply | Threaded
Open this post in threaded view
|

Re: Starting many Ignite nodes at once

Val,

Indeed I forgot about this :) 

And even more, Visor command line tool could execute commands in batch mode.

Alexey, take a look here https://issues.apache.org/jira/browse/IGNITE-185 (see comments to issue with examples of batch mode).


I also found that we have no documentation about Visor cmd in our documentation.

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com