FW: class loading, peer class loading, jars, fun times in ignite

classic Classic list List threaded Threaded
6 messages Options
Scott Cote Scott Cote
Reply | Threaded
Open this post in threaded view
|

FW: class loading, peer class loading, jars, fun times in ignite

Whoops – sent to the wrong list …

 

From: Scott Cote
Sent: Tuesday, May 28, 2019 1:04 PM
To: [hidden email]
Subject: class loading, peer class loading, jars, fun times in ignite

 

I am fairly certain that I don’t know how to use peer class loading properly.

 

Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:

 

Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):

 

try (Ignite ignite = Ignition.start("examples/config/example-ignite.xml")) {

    IgniteCache<Integer, MyWrapperOfString> cache = ignite.getOrCreateCache("myCacheName");

 

    // Store keys in cache (values will end up on different cache nodes).

    for (int i = 0; i < 10; i++)

        cache.put(i,new MyWrapperOfString( Integer.toString(i)));

 

    for (int i = 0; i < 10; i++)

        System.out.println("Got [key=" + i + ", val=" + cache.get(i) + ']');

}

 

 

Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….

 

However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.

 

Should I have to do this?

If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?

I thought the peer class loading is supposed to solve this problem.

 

I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?

 

Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .

 

Cheers.

 

SCott

 

Павлухин Иван Павлухин Иван
Reply | Threaded
Open this post in threaded view
|

Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott,

As far as I know, peer class loading does not work for data classes
(which are stored in a cache). It works for tasks sended for execution
using IgniteCompute.

It is only a partial answer. Could you describe your use case in more details?

вт, 28 мая 2019 г. в 23:35, Scott Cote <[hidden email]>:

>
> Whoops – sent to the wrong list …
>
>
>
> From: Scott Cote
> Sent: Tuesday, May 28, 2019 1:04 PM
> To: [hidden email]
> Subject: class loading, peer class loading, jars, fun times in ignite
>
>
>
> I am fairly certain that I don’t know how to use peer class loading properly.
>
>
>
> Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:
>
>
>
> Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):
>
>
>
> try (Ignite ignite = Ignition.start("examples/config/example-ignite.xml")) {
>
>     IgniteCache<Integer, MyWrapperOfString> cache = ignite.getOrCreateCache("myCacheName");
>
>
>
>     // Store keys in cache (values will end up on different cache nodes).
>
>     for (int i = 0; i < 10; i++)
>
>         cache.put(i,new MyWrapperOfString( Integer.toString(i)));
>
>
>
>     for (int i = 0; i < 10; i++)
>
>         System.out.println("Got [key=" + i + ", val=" + cache.get(i) + ']');
>
> }
>
>
>
>
>
> Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….
>
>
>
> However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.
>
>
>
> Should I have to do this?
>
> If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?
>
> I thought the peer class loading is supposed to solve this problem.
>
>
>
> I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?
>
>
>
> Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .
>
>
>
> Cheers.
>
>
>
> SCott
>
>



--
Best regards,
Ivan Pavlukhin
Scott Cote Scott Cote
Reply | Threaded
Open this post in threaded view
|

RE: FW: class loading, peer class loading, jars, fun times in ignite

Ivan,

I think that you gave me the right answer and confirmed my suspicion - that peer class loading is only for just an executable and not data.

Can I assist by helping edit the documentation on the Apache Ignite site to add clarity on when a jar is needed in the lib folder?

Also - I'll have to come up with a plan to flush out of date java data classes, or maybe you guys have some techniques that allow for online migration of v1 of a java data class to v2 of a java data class without having to shutdown the whole set of vm's.

Our system is pretty simple.


We use the cache's.

Not the streamers.

Not the callables ...

It’s a big fantastic cache 😊

(With some stuff that I built on top of it - like a priority queue framework)

SCott

-----Original Message-----
From: Павлухин Иван <[hidden email]>
Sent: Wednesday, May 29, 2019 1:16 AM
To: [hidden email]
Subject: Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott,

As far as I know, peer class loading does not work for data classes (which are stored in a cache). It works for tasks sended for execution using IgniteCompute.

It is only a partial answer. Could you describe your use case in more details?

вт, 28 мая 2019 г. в 23:35, Scott Cote <[hidden email]>:

>
> Whoops – sent to the wrong list …
>
>
>
> From: Scott Cote
> Sent: Tuesday, May 28, 2019 1:04 PM
> To: [hidden email]
> Subject: class loading, peer class loading, jars, fun times in ignite
>
>
>
> I am fairly certain that I don’t know how to use peer class loading properly.
>
>
>
> Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:
>
>
>
> Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):
>
>
>
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
>
>     IgniteCache<Integer, MyWrapperOfString> cache =
> ignite.getOrCreateCache("myCacheName");
>
>
>
>     // Store keys in cache (values will end up on different cache nodes).
>
>     for (int i = 0; i < 10; i++)
>
>         cache.put(i,new MyWrapperOfString( Integer.toString(i)));
>
>
>
>     for (int i = 0; i < 10; i++)
>
>         System.out.println("Got [key=" + i + ", val=" + cache.get(i) +
> ']');
>
> }
>
>
>
>
>
> Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….
>
>
>
> However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.
>
>
>
> Should I have to do this?
>
> If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?
>
> I thought the peer class loading is supposed to solve this problem.
>
>
>
> I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?
>
>
>
> Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .
>
>
>
> Cheers.
>
>
>
> SCott
>
>



--
Best regards,
Ivan Pavlukhin
Dmitriy Pavlov Dmitriy Pavlov
Reply | Threaded
Open this post in threaded view
|

Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott, 

actually, users are encouraged to suggest edits to the documentation. 


Sincerely,
Dmitriy Pavlov

ср, 29 мая 2019 г. в 15:54, Scott Cote <[hidden email]>:
Ivan,

I think that you gave me the right answer and confirmed my suspicion - that peer class loading is only for just an executable and not data.

Can I assist by helping edit the documentation on the Apache Ignite site to add clarity on when a jar is needed in the lib folder?

Also - I'll have to come up with a plan to flush out of date java data classes, or maybe you guys have some techniques that allow for online migration of v1 of a java data class to v2 of a java data class without having to shutdown the whole set of vm's.

Our system is pretty simple.


We use the cache's.

Not the streamers.

Not the callables ...

It’s a big fantastic cache 😊

(With some stuff that I built on top of it - like a priority queue framework)

SCott

-----Original Message-----
From: Павлухин Иван <[hidden email]>
Sent: Wednesday, May 29, 2019 1:16 AM
To: [hidden email]
Subject: Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott,

As far as I know, peer class loading does not work for data classes (which are stored in a cache). It works for tasks sended for execution using IgniteCompute.

It is only a partial answer. Could you describe your use case in more details?

вт, 28 мая 2019 г. в 23:35, Scott Cote <[hidden email]>:
>
> Whoops – sent to the wrong list …
>
>
>
> From: Scott Cote
> Sent: Tuesday, May 28, 2019 1:04 PM
> To: [hidden email]
> Subject: class loading, peer class loading, jars, fun times in ignite
>
>
>
> I am fairly certain that I don’t know how to use peer class loading properly.
>
>
>
> Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:
>
>
>
> Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):
>
>
>
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
>
>     IgniteCache<Integer, MyWrapperOfString> cache =
> ignite.getOrCreateCache("myCacheName");
>
>
>
>     // Store keys in cache (values will end up on different cache nodes).
>
>     for (int i = 0; i < 10; i++)
>
>         cache.put(i,new MyWrapperOfString( Integer.toString(i)));
>
>
>
>     for (int i = 0; i < 10; i++)
>
>         System.out.println("Got [key=" + i + ", val=" + cache.get(i) +
> ']');
>
> }
>
>
>
>
>
> Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….
>
>
>
> However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.
>
>
>
> Should I have to do this?
>
> If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?
>
> I thought the peer class loading is supposed to solve this problem.
>
>
>
> I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?
>
>
>
> Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .
>
>
>
> Cheers.
>
>
>
> SCott
>
>



--
Best regards,
Ivan Pavlukhin
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: FW: class loading, peer class loading, jars, fun times in ignite

In reply to this post by Scott Cote
Hello!

What kind of migrations are you expecting? I guess you could add or remove fields just by doing rolling upgrade (restart nodes to V2 one by one until no nodes with V1 are left).

Trying to rename fields is not recommended and changing fields' type is not supported.

Regards,
--
Ilya Kasnacheev


ср, 29 мая 2019 г. в 15:54, Scott Cote <[hidden email]>:
Ivan,

I think that you gave me the right answer and confirmed my suspicion - that peer class loading is only for just an executable and not data.

Can I assist by helping edit the documentation on the Apache Ignite site to add clarity on when a jar is needed in the lib folder?

Also - I'll have to come up with a plan to flush out of date java data classes, or maybe you guys have some techniques that allow for online migration of v1 of a java data class to v2 of a java data class without having to shutdown the whole set of vm's.

Our system is pretty simple.


We use the cache's.

Not the streamers.

Not the callables ...

It’s a big fantastic cache 😊

(With some stuff that I built on top of it - like a priority queue framework)

SCott

-----Original Message-----
From: Павлухин Иван <[hidden email]>
Sent: Wednesday, May 29, 2019 1:16 AM
To: [hidden email]
Subject: Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott,

As far as I know, peer class loading does not work for data classes (which are stored in a cache). It works for tasks sended for execution using IgniteCompute.

It is only a partial answer. Could you describe your use case in more details?

вт, 28 мая 2019 г. в 23:35, Scott Cote <[hidden email]>:
>
> Whoops – sent to the wrong list …
>
>
>
> From: Scott Cote
> Sent: Tuesday, May 28, 2019 1:04 PM
> To: [hidden email]
> Subject: class loading, peer class loading, jars, fun times in ignite
>
>
>
> I am fairly certain that I don’t know how to use peer class loading properly.
>
>
>
> Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:
>
>
>
> Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):
>
>
>
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
>
>     IgniteCache<Integer, MyWrapperOfString> cache =
> ignite.getOrCreateCache("myCacheName");
>
>
>
>     // Store keys in cache (values will end up on different cache nodes).
>
>     for (int i = 0; i < 10; i++)
>
>         cache.put(i,new MyWrapperOfString( Integer.toString(i)));
>
>
>
>     for (int i = 0; i < 10; i++)
>
>         System.out.println("Got [key=" + i + ", val=" + cache.get(i) +
> ']');
>
> }
>
>
>
>
>
> Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….
>
>
>
> However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.
>
>
>
> Should I have to do this?
>
> If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?
>
> I thought the peer class loading is supposed to solve this problem.
>
>
>
> I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?
>
>
>
> Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .
>
>
>
> Cheers.
>
>
>
> SCott
>
>



--
Best regards,
Ivan Pavlukhin
Dave Harvey Dave Harvey
Reply | Threaded
Open this post in threaded view
|

Re: FW: class loading, peer class loading, jars, fun times in ignite

In reply to this post by Dmitriy Pavlov
With these symptoms when the programmer understands the rules, there is still a somewhat frequent bug where  "cache.withKeepBinary() "is what is required, rather than simply "cache".

On Wed, May 29, 2019 at 12:29 PM Dmitriy Pavlov <[hidden email]> wrote:
Hi Scott, 

actually, users are encouraged to suggest edits to the documentation. 


Sincerely,
Dmitriy Pavlov

ср, 29 мая 2019 г. в 15:54, Scott Cote <[hidden email]>:
Ivan,

I think that you gave me the right answer and confirmed my suspicion - that peer class loading is only for just an executable and not data.

Can I assist by helping edit the documentation on the Apache Ignite site to add clarity on when a jar is needed in the lib folder?

Also - I'll have to come up with a plan to flush out of date java data classes, or maybe you guys have some techniques that allow for online migration of v1 of a java data class to v2 of a java data class without having to shutdown the whole set of vm's.

Our system is pretty simple.


We use the cache's.

Not the streamers.

Not the callables ...

It’s a big fantastic cache 😊

(With some stuff that I built on top of it - like a priority queue framework)

SCott

-----Original Message-----
From: Павлухин Иван <[hidden email]>
Sent: Wednesday, May 29, 2019 1:16 AM
To: [hidden email]
Subject: Re: FW: class loading, peer class loading, jars, fun times in ignite

Hi Scott,

As far as I know, peer class loading does not work for data classes (which are stored in a cache). It works for tasks sended for execution using IgniteCompute.

It is only a partial answer. Could you describe your use case in more details?

вт, 28 мая 2019 г. в 23:35, Scott Cote <[hidden email]>:
>
> Whoops – sent to the wrong list …
>
>
>
> From: Scott Cote
> Sent: Tuesday, May 28, 2019 1:04 PM
> To: [hidden email]
> Subject: class loading, peer class loading, jars, fun times in ignite
>
>
>
> I am fairly certain that I don’t know how to use peer class loading properly.
>
>
>
> Am using Apache Ignite 2.7.  If I have a node running on 192.168.1.2 with a peer class loading enabled, and I start up a second node – 192.168.1.3, client mode enabled and peer class loading enabled, then I expected the following:
>
>
>
> Running the snippet (based on https://apacheignite.readme.io/docs/getting-started#section-first-ignite-data-grid-application ) on the client (192.168.1.3):
>
>
>
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
>
>     IgniteCache<Integer, MyWrapperOfString> cache =
> ignite.getOrCreateCache("myCacheName");
>
>
>
>     // Store keys in cache (values will end up on different cache nodes).
>
>     for (int i = 0; i < 10; i++)
>
>         cache.put(i,new MyWrapperOfString( Integer.toString(i)));
>
>
>
>     for (int i = 0; i < 10; i++)
>
>         System.out.println("Got [key=" + i + ", val=" + cache.get(i) +
> ']');
>
> }
>
>
>
>
>
> Would cause the cache of “MyWrapperOfString” instances to be available on 192.168.1.2 and on 192.168.1.3 .   Also be able to observe the cache using visor, etc ….
>
>
>
> However – I instead get an error that the class “MyWrapperOfString” is not available on 192.168.1.2.   Now if I take the jar that the class is packed, and place it in the lib folder, all is happy.
>
>
>
> Should I have to do this?
>
> If yes – how do I update the jar if I have a cluster of nodes doing this?   Do I have to shutdown the entire cluster in order to not have class loader problems?
>
> I thought the peer class loading is supposed to solve this problem.
>
>
>
> I think it would be VERY INSTRUCTIVE for the snippet that I anchored to not use a standard java library cache object, but to demonstrate the need to package value object into a jar and stuff it into the lib folder (If this is what is expected).     Running lambdas that use basic java primitives is cool, but is this the normal?
>
>
>
> Switching up …. Is there interest in me creating class loader that would load java classes into the vm that could be incorporated into ignite?   So instead of reading a jar, you load the class bytes into a cache .  You want to hot load a new class?  Fine ! pump into the DISTRIBUTED_CLASS_PATH_CACHE .
>
>
>
> Cheers.
>
>
>
> SCott
>
>



--
Best regards,
Ivan Pavlukhin


Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.