newbie question: how best to pass Ignite to a ComputeJob

classic Classic list List threaded Threaded
2 messages Options
jonathonbell jonathonbell
Reply | Threaded
Open this post in threaded view
|

newbie question: how best to pass Ignite to a ComputeJob

Please forgive what may be silly question, but...

I'm using the ComputeTask/ComputeJob framework to arrange for m Upstream ComputeJobs to send messages to n Downstream ComputeJobs, these latter running on certain nodes of my cluster that i wish to be in control of.

My (perhaps naive) plan is for both the U-job and D-job to each hold onto an IgniteMessaging object that was created for them by the parent ComputeTask at construction time, and that forms a sort of 'private channel' through which U-jobs can feed data to a specific D-job.

Assuming this design is reasonable, what is the recommended method for passing a specific IgniteMessaging object (or an Ignite object itself, for that matter) to each CompuetJob?

* through the job's constructor?

   public Map<Job,ClusterNode> map(List<ClusterNode> subgrid,Ignite ignite) throws IgniteException
   {
      Map<Job,ClusterNode> map = new HashMap<>();
      for (ClusterNode n : subgrid)
      {
          IgniteMessaging m = ...
          map.put(new Job(m),n);
          ...
      }

      return map;
   }

This would lead to the IgniteMessaging object 'm' (and all that it references) being serialized across the network to the node on which the job actually runs, no? Is that a bad thing?

* through the ComputeTaskSession mechanism?

I'm new to IoC dependency injection, but perhaps this is the preferred method for passing Task and Job specific data through to the jobs at execute() time?

* by first 'naming' the cluster and then having job 'find' the cluster by name from within its executed method?

* some other mechanism?

Many thanks

Jonathon
alexey.goncharuk alexey.goncharuk
Reply | Threaded
Open this post in threaded view
|

Re: newbie question: how best to pass Ignite to a ComputeJob

I think the most straightforward and easy way would be to use resource injection. Simply declare an Ignite field in your task/job classes and annotate it with @IgniteInstanceResource. The corresponding instance of Ignite will be injected upon task/job execution. You can check MessagingExample for the use of @IgniteIntanceResource.

Passing an instance of IgniteMessaging to your classes will also work and it actually will not cause any additional serialization; only the name of your grid will be written. However, this approach, as well as getting an existing instance of Ignite from Ignition, requires proper (identical) naming of grid on all cluster nodes. The approach with injected resources will work regardless of grid naming.

Hope this helps,
AG

2015-06-16 10:45 GMT-07:00 jonathon <[hidden email]>:
Please forgive what may be silly question, but...

I'm using the ComputeTask/ComputeJob framework to arrange for /m/ Upstream
ComputeJobs to send messages to /n/ Downstream ComputeJobs, these latter
running on certain nodes of my cluster that i wish to be in control of.

My (perhaps naive) plan is for both the U-job and D-job to each hold onto an
/IgniteMessaging/ object that was created for them by the parent ComputeTask
at construction time, and that forms a sort of 'private channel' through
which U-jobs can feed data to a specific D-job.

Assuming this design is reasonable, what is the recommended method for
passing a specific IgniteMessaging object (or an Ignite object itself, for
that matter) to each CompuetJob?

* through the job's constructor?


This would lead to the IgniteMessaging object 'm' (and all that it
references) being serialized across the network to the node on which the job
actually runs, no? Is that a bad thing?

* through the ComputeTaskSession mechanism?

I'm new to IoC dependency injection, but perhaps this is the preferred
method for passing Task and Job specific data through to the jobs at
execute() time?

* by first 'naming' the cluster and then having job 'find' the cluster by
name from within its executed method?

* some other mechanism?

Many thanks

Jonathon



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/newbie-question-how-best-to-pass-Ignite-to-a-ComputeJob-tp503.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.