ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

classic Classic list List threaded Threaded
10 messages Options
rtroilo rtroilo
Reply | Threaded
Open this post in threaded view
|

ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hey Guys,

I have a strange problem with the Transform part of a ScanQuery that I get a deserialize object exception for the SerializedLambda.
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]

Strange is that it works very well in other cases like:

try (Ignite ignite = Ignition.start("ignite.xml")) {
  IgniteCache<Integer,String> cache = ignite.createCache("ScanTransformTest");
  cache.put(1, "Hello");

  // 1  Just a broadcast with a IgniteClosure works!
  compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
       
  // 2 Running the ScanQuery on the remote server. Works too!
  compute.broadcast(() -> {
    try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), (a) -> 234L)){  
      for( Long row : cursor) {
        return row;
      }
    }
    return 0;
  }).forEach(System.out::println);
   

  // 3 But running it from the client side it fails!
  try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) -> 345L)){  
    for( Long row : cursor) {
      System.out.println(row);
      break;
    }
  }

}

Also changing the Lambda to a actually class implementing IgniteClosure results in the same problem.

Any idea what I'm doing wrong?

We are running ignite ver. 2.8.0#20200226-sha1:341b01df
With java openjdk 11.0.7 2020-04-14

Thank you!

Best,
Rafael


Error Log:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:49)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read field [name=capturingClass]
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
        at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
        ... 20 more
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: ScanQueryTest
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
        ... 21 more
Caused by: java.lang.ClassNotFoundException: ScanQueryTest
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 29 more





--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hi Rafael,

Lamdas and anonymous classes capture and serialize their Inner/OuterClass with all the fields, and classes of the fields of those fields, etc. In your case, ScanQueryClass was captured during the lamda serialization and that class was not found during deserialization on the other side:

Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read field [name=capturingClass]
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
        at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
        ... 20 more
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: ScanQueryTest
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
        ... 21 more
Caused by: java.lang.ClassNotFoundException: ScanQueryTest
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 29 more

My advice is to avoid using lambdas and anonymous classes due to this specificity. Instead, you can create "private static classes" and Ignite will be serializing an instance of that class only:

private static class MyFilter implements IgniteClosure {
....
}

Lamdas are nice for demos and simple code samples where Inner/OuterClasses are simple Java classes. While in real production environments, things can get messy.

-
Denis


On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <[hidden email]> wrote:
Hey Guys,

I have a strange problem with the Transform part of a ScanQuery that I get a deserialize object exception for the SerializedLambda.
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]

Strange is that it works very well in other cases like:

try (Ignite ignite = Ignition.start("ignite.xml")) {
  IgniteCache<Integer,String> cache = ignite.createCache("ScanTransformTest");
  cache.put(1, "Hello");

  // 1  Just a broadcast with a IgniteClosure works!
  compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);

  // 2 Running the ScanQuery on the remote server. Works too!
  compute.broadcast(() -> {
    try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), (a) -> 234L)){ 
      for( Long row : cursor) {
        return row;
      }
    }
    return 0;
  }).forEach(System.out::println);


  // 3 But running it from the client side it fails!
  try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) -> 345L)){ 
    for( Long row : cursor) {
      System.out.println(row);
      break;
    }
  }

}

Also changing the Lambda to a actually class implementing IgniteClosure results in the same problem.

Any idea what I'm doing wrong?

We are running ignite ver. 2.8.0#20200226-sha1:341b01df
With java openjdk 11.0.7 2020-04-14

Thank you!

Best,
Rafael


Error Log:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:49)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda]
        at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read field [name=capturingClass]
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
        at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
        ... 20 more
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: ScanQueryTest
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
        at org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
        at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
        ... 21 more
Caused by: java.lang.ClassNotFoundException: ScanQueryTest
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 29 more





--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
rtroilo rtroilo
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hi Denis,

thank you for your quick response. Unfortunately even avoiding lambdas or anonymous classes didn't solve this problem.
And was my first thought for this problem too.

With:

MyFilter.java

import org.apache.ignite.lang.IgniteClosure;

public class MyFilter<T> implements IgniteClosure<T, Long> {

  private static final long serialVersionUID = 1L;
  private final long l;

  public MyFilter(long l) {
    this.l = l;
  }

  @Override
  public Long apply(T e) {
    return l;
  }

}


and

try (Ignite ignite = Ignition.start("ignite.xml")) {
   IgniteCache<Integer, String> cache = ignite.getOrCreateCache("ScanTransformTest");
  ...
   // 2 Running the ScanQuery on the remote server. Works!
  compute.broadcast(() -> {
    try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
      for (Long row : cursor) {
        return row;
      }
    }
    return 0;
  })
 
  // 3 Still does not work.
  try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new MyFilter<>(345L))) {
    for (Long row : cursor) {
      System.out.println(row);
      break;
    }
  }
}

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config
...
<property name="peerClassLoadingEnabled" value="true"/>
...
it is enabled.

I'm also not sure why the same class (IgniteClosure) works in the other case but not in the Transformer or whats the differnt in case 2 to 3.
I guess making the MyFilter class available on the server will solve this problem, but should not peerClassLoading work here?

Is there any information  I can provide or test I should do to help to solve this problem.

Thank you very much.


Stack Trace:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:57)
Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: MyFilter
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: java.lang.ClassNotFoundException: MyFilter
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 21 more



On 7/1/20 7:09 PM, Denis Magda wrote:

> Hi Rafael,
>
> Lamdas and anonymous classes capture and serialize their Inner/OuterClass
> with all the fields, and classes of the fields of those fields, etc. In
> your case, ScanQueryClass was captured during the lamda serialization and
> that class was not found during deserialization on the other side:
>
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> read field [name=capturingClass]
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>         at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>         ... 20 more
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> ScanQueryTest
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>         at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>         ... 21 more
> *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>         at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>         at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>         at java.base/java.lang.Class.forName0(Native Method)
>         at java.base/java.lang.Class.forName(Class.java:398)
>         at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>         at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>         ... 29 more
>
> My advice is to avoid using lambdas and anonymous classes due to this
> specificity. Instead, you can create "private static classes" and Ignite
> will be serializing an instance of that class only:
>
> private static class MyFilter implements IgniteClosure {
> ....
> }
>
> Lamdas are nice for demos and simple code samples where Inner/OuterClasses
> are simple Java classes. While in real production environments, things can
> get messy.
>
> -
> Denis
>
>
> On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <[hidden email]>
> wrote:
>
>> Hey Guys,
>>
>> I have a strange problem with the Transform part of a ScanQuery that I get
>> a deserialize object exception for the SerializedLambda.
>>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>
>> Strange is that it works very well in other cases like:
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>   IgniteCache<Integer,String> cache =
>> ignite.createCache("ScanTransformTest");
>>   cache.put(1, "Hello");
>>
>>   // 1  Just a broadcast with a IgniteClosure works!
>>   compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
>>
>>   // 2 Running the ScanQuery on the remote server. Works too!
>>   compute.broadcast(() -> {
>>     try(QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), (a) -> 234L)){
>>       for( Long row : cursor) {
>>         return row;
>>       }
>>     }
>>     return 0;
>>   }).forEach(System.out::println);
>>
>>
>>   // 3 But running it from the client side it fails!
>>   try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> 345L)){
>>     for( Long row : cursor) {
>>       System.out.println(row);
>>       break;
>>     }
>>   }
>>
>> }
>>
>> Also changing the Lambda to a actually class implementing IgniteClosure
>> results in the same problem.
>>
>> Any idea what I'm doing wrong?
>>
>> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> With java openjdk 11.0.7 2020-04-14
>>
>> Thank you!
>>
>> Best,
>> Rafael
>>
>>
>> Error Log:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>         at
>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>         at
>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>         at
>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>         at ScanQueryTest.main(ScanQueryTest.java:49)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>         at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>         at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>         at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>         at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>         ... 14 more
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> read field [name=capturingClass]
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>         ... 20 more
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> ScanQueryTest
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>         ... 21 more
>> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>>         at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>         at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>         at java.base/java.lang.Class.forName0(Native Method)
>>         at java.base/java.lang.Class.forName(Class.java:398)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>         at
>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>         ... 29 more
>>
>>
>>
>>
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg University
>>
>> https://heigit.org | [hidden email] | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>

--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hello!

Did you enable peer class loading? You need to either enable peer class loading, or put your filter class(es) to all server nodes.

Regards,
--
Ilya Kasnacheev


чт, 2 июл. 2020 г. в 15:09, Rafael Troilo <[hidden email]>:
Hi Denis,

thank you for your quick response. Unfortunately even avoiding lambdas or anonymous classes didn't solve this problem.
And was my first thought for this problem too.

With:

MyFilter.java

import org.apache.ignite.lang.IgniteClosure;

public class MyFilter<T> implements IgniteClosure<T, Long> {

  private static final long serialVersionUID = 1L;
  private final long l;

  public MyFilter(long l) {
    this.l = l;
  }

  @Override
  public Long apply(T e) {
    return l;
  }

}


and

try (Ignite ignite = Ignition.start("ignite.xml")) {
   IgniteCache<Integer, String> cache = ignite.getOrCreateCache("ScanTransformTest");
  ...
   // 2 Running the ScanQuery on the remote server. Works!
  compute.broadcast(() -> {
    try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
      for (Long row : cursor) {
        return row;
      }
    }
    return 0;
  })

  // 3 Still does not work.
  try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new MyFilter<>(345L))) {
    for (Long row : cursor) {
      System.out.println(row);
      break;
    }
  }
}

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config
...
<property name="peerClassLoadingEnabled" value="true"/>
...
it is enabled.

I'm also not sure why the same class (IgniteClosure) works in the other case but not in the Transformer or whats the differnt in case 2 to 3.
I guess making the MyFilter class available on the server will solve this problem, but should not peerClassLoading work here?

Is there any information  I can provide or test I should do to help to solve this problem.

Thank you very much.


Stack Trace:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:57)
Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: MyFilter
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: java.lang.ClassNotFoundException: MyFilter
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 21 more



On 7/1/20 7:09 PM, Denis Magda wrote:
> Hi Rafael,
>
> Lamdas and anonymous classes capture and serialize their Inner/OuterClass
> with all the fields, and classes of the fields of those fields, etc. In
> your case, ScanQueryClass was captured during the lamda serialization and
> that class was not found during deserialization on the other side:
>
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> read field [name=capturingClass]
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>         at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>         ... 20 more
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> ScanQueryTest
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>         at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>         ... 21 more
> *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>         at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>         at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>         at java.base/java.lang.Class.forName0(Native Method)
>         at java.base/java.lang.Class.forName(Class.java:398)
>         at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>         at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>         ... 29 more
>
> My advice is to avoid using lambdas and anonymous classes due to this
> specificity. Instead, you can create "private static classes" and Ignite
> will be serializing an instance of that class only:
>
> private static class MyFilter implements IgniteClosure {
> ....
> }
>
> Lamdas are nice for demos and simple code samples where Inner/OuterClasses
> are simple Java classes. While in real production environments, things can
> get messy.
>
> -
> Denis
>
>
> On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <[hidden email]>
> wrote:
>
>> Hey Guys,
>>
>> I have a strange problem with the Transform part of a ScanQuery that I get
>> a deserialize object exception for the SerializedLambda.
>>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>
>> Strange is that it works very well in other cases like:
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>   IgniteCache<Integer,String> cache =
>> ignite.createCache("ScanTransformTest");
>>   cache.put(1, "Hello");
>>
>>   // 1  Just a broadcast with a IgniteClosure works!
>>   compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
>>
>>   // 2 Running the ScanQuery on the remote server. Works too!
>>   compute.broadcast(() -> {
>>     try(QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), (a) -> 234L)){
>>       for( Long row : cursor) {
>>         return row;
>>       }
>>     }
>>     return 0;
>>   }).forEach(System.out::println);
>>
>>
>>   // 3 But running it from the client side it fails!
>>   try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> 345L)){
>>     for( Long row : cursor) {
>>       System.out.println(row);
>>       break;
>>     }
>>   }
>>
>> }
>>
>> Also changing the Lambda to a actually class implementing IgniteClosure
>> results in the same problem.
>>
>> Any idea what I'm doing wrong?
>>
>> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> With java openjdk 11.0.7 2020-04-14
>>
>> Thank you!
>>
>> Best,
>> Rafael
>>
>>
>> Error Log:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>         at
>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>         at
>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>         at
>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>         at ScanQueryTest.main(ScanQueryTest.java:49)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>         at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>         at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>         at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>         at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>         ... 14 more
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> read field [name=capturingClass]
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>         ... 20 more
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> ScanQueryTest
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>         ... 21 more
>> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>>         at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>         at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>         at java.base/java.lang.Class.forName0(Native Method)
>>         at java.base/java.lang.Class.forName(Class.java:398)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>         at
>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>         ... 29 more
>>
>>
>>
>>
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg University
>>
>> https://heigit.org | [hidden email] | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>

--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Ilya,

Yes, Rafael confirmed that the peer-class-loading is set.

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config

Is this a known limitation or regression? Should we file an feature request or bug? 

Denis

On Thursday, July 2, 2020, Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Did you enable peer class loading? You need to either enable peer class loading, or put your filter class(es) to all server nodes.

Regards,
--
Ilya Kasnacheev


чт, 2 июл. 2020 г. в 15:09, Rafael Troilo <[hidden email]>:
Hi Denis,

thank you for your quick response. Unfortunately even avoiding lambdas or anonymous classes didn't solve this problem.
And was my first thought for this problem too.

With:

MyFilter.java

import org.apache.ignite.lang.IgniteClosure;

public class MyFilter<T> implements IgniteClosure<T, Long> {

  private static final long serialVersionUID = 1L;
  private final long l;

  public MyFilter(long l) {
    this.l = l;
  }

  @Override
  public Long apply(T e) {
    return l;
  }

}


and

try (Ignite ignite = Ignition.start("ignite.xml")) {
   IgniteCache<Integer, String> cache = ignite.getOrCreateCache("ScanTransformTest");
  ...
   // 2 Running the ScanQuery on the remote server. Works!
  compute.broadcast(() -> {
    try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
      for (Long row : cursor) {
        return row;
      }
    }
    return 0;
  })

  // 3 Still does not work.
  try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new MyFilter<>(345L))) {
    for (Long row : cursor) {
      System.out.println(row);
      break;
    }
  }
}

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config
...
<property name="peerClassLoadingEnabled" value="true"/>
...
it is enabled.

I'm also not sure why the same class (IgniteClosure) works in the other case but not in the Transformer or whats the differnt in case 2 to 3.
I guess making the MyFilter class available on the server will solve this problem, but should not peerClassLoading work here?

Is there any information  I can provide or test I should do to help to solve this problem.

Thank you very much.


Stack Trace:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:57)
Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: MyFilter
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: java.lang.ClassNotFoundException: MyFilter
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 21 more



On 7/1/20 7:09 PM, Denis Magda wrote:
> Hi Rafael,
>
> Lamdas and anonymous classes capture and serialize their Inner/OuterClass
> with all the fields, and classes of the fields of those fields, etc. In
> your case, ScanQueryClass was captured during the lamda serialization and
> that class was not found during deserialization on the other side:
>
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> read field [name=capturingClass]
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>         at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>         ... 20 more
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> ScanQueryTest
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>         at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>         ... 21 more
> *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>         at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>         at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>         at java.base/java.lang.Class.forName0(Native Method)
>         at java.base/java.lang.Class.forName(Class.java:398)
>         at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>         at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>         ... 29 more
>
> My advice is to avoid using lambdas and anonymous classes due to this
> specificity. Instead, you can create "private static classes" and Ignite
> will be serializing an instance of that class only:
>
> private static class MyFilter implements IgniteClosure {
> ....
> }
>
> Lamdas are nice for demos and simple code samples where Inner/OuterClasses
> are simple Java classes. While in real production environments, things can
> get messy.
>
> -
> Denis
>
>
> On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <[hidden email]>
> wrote:
>
>> Hey Guys,
>>
>> I have a strange problem with the Transform part of a ScanQuery that I get
>> a deserialize object exception for the SerializedLambda.
>>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>
>> Strange is that it works very well in other cases like:
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>   IgniteCache<Integer,String> cache =
>> ignite.createCache("ScanTransformTest");
>>   cache.put(1, "Hello");
>>
>>   // 1  Just a broadcast with a IgniteClosure works!
>>   compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
>>
>>   // 2 Running the ScanQuery on the remote server. Works too!
>>   compute.broadcast(() -> {
>>     try(QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), (a) -> 234L)){
>>       for( Long row : cursor) {
>>         return row;
>>       }
>>     }
>>     return 0;
>>   }).forEach(System.out::println);
>>
>>
>>   // 3 But running it from the client side it fails!
>>   try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> 345L)){
>>     for( Long row : cursor) {
>>       System.out.println(row);
>>       break;
>>     }
>>   }
>>
>> }
>>
>> Also changing the Lambda to a actually class implementing IgniteClosure
>> results in the same problem.
>>
>> Any idea what I'm doing wrong?
>>
>> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> With java openjdk 11.0.7 2020-04-14
>>
>> Thank you!
>>
>> Best,
>> Rafael
>>
>>
>> Error Log:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>         at
>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>         at
>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>         at
>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>         at ScanQueryTest.main(ScanQueryTest.java:49)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>         at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>         at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>         at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>         at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>         ... 14 more
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> read field [name=capturingClass]
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>         ... 20 more
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> ScanQueryTest
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>         ... 21 more
>> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>>         at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>         at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>         at java.base/java.lang.Class.forName0(Native Method)
>>         at java.base/java.lang.Class.forName(Class.java:398)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>         at
>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>         ... 29 more
>>
>>
>>
>>
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg University
>>
>> https://heigit.org | [hidden email] | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>

--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger


--
-
Denis

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hello!

I would like to start with a reproducer. Rafael, can you please throw together a runnable stand-alone reproducer of this issue?

Thanks,
--
Ilya Kasnacheev


чт, 2 июл. 2020 г. в 17:28, Denis Magda <[hidden email]>:
Ilya,

Yes, Rafael confirmed that the peer-class-loading is set.

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config

Is this a known limitation or regression? Should we file an feature request or bug? 

Denis

On Thursday, July 2, 2020, Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Did you enable peer class loading? You need to either enable peer class loading, or put your filter class(es) to all server nodes.

Regards,
--
Ilya Kasnacheev


чт, 2 июл. 2020 г. в 15:09, Rafael Troilo <[hidden email]>:
Hi Denis,

thank you for your quick response. Unfortunately even avoiding lambdas or anonymous classes didn't solve this problem.
And was my first thought for this problem too.

With:

MyFilter.java

import org.apache.ignite.lang.IgniteClosure;

public class MyFilter<T> implements IgniteClosure<T, Long> {

  private static final long serialVersionUID = 1L;
  private final long l;

  public MyFilter(long l) {
    this.l = l;
  }

  @Override
  public Long apply(T e) {
    return l;
  }

}


and

try (Ignite ignite = Ignition.start("ignite.xml")) {
   IgniteCache<Integer, String> cache = ignite.getOrCreateCache("ScanTransformTest");
  ...
   // 2 Running the ScanQuery on the remote server. Works!
  compute.broadcast(() -> {
    try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
      for (Long row : cursor) {
        return row;
      }
    }
    return 0;
  })

  // 3 Still does not work.
  try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new MyFilter<>(345L))) {
    for (Long row : cursor) {
      System.out.println(row);
      break;
    }
  }
}

Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client ignite config
...
<property name="peerClassLoadingEnabled" value="true"/>
...
it is enabled.

I'm also not sure why the same class (IgniteClosure) works in the other case but not in the Transformer or whats the differnt in case 2 to 3.
I guess making the MyFilter class available on the server will solve this problem, but should not peerClassLoading work here?

Is there any information  I can provide or test I should do to help to solve this problem.

Thank you very much.


Stack Trace:
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
        at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
        at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
        at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
        at org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
        at ScanQueryTest.main(ScanQueryTest.java:57)
Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
        at org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
        at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: MyFilter
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
        at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
        at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
        at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
        ... 14 more
Caused by: java.lang.ClassNotFoundException: MyFilter
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
        at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
        at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
        ... 21 more



On 7/1/20 7:09 PM, Denis Magda wrote:
> Hi Rafael,
>
> Lamdas and anonymous classes capture and serialize their Inner/OuterClass
> with all the fields, and classes of the fields of those fields, etc. In
> your case, ScanQueryClass was captured during the lamda serialization and
> that class was not found during deserialization on the other side:
>
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> read field [name=capturingClass]
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>         at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>         ... 20 more
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> ScanQueryTest
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>         at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>         at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>         at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>         ... 21 more
> *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>         at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>         at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>         at java.base/java.lang.Class.forName0(Native Method)
>         at java.base/java.lang.Class.forName(Class.java:398)
>         at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>         at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>         at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>         ... 29 more
>
> My advice is to avoid using lambdas and anonymous classes due to this
> specificity. Instead, you can create "private static classes" and Ignite
> will be serializing an instance of that class only:
>
> private static class MyFilter implements IgniteClosure {
> ....
> }
>
> Lamdas are nice for demos and simple code samples where Inner/OuterClasses
> are simple Java classes. While in real production environments, things can
> get messy.
>
> -
> Denis
>
>
> On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <[hidden email]>
> wrote:
>
>> Hey Guys,
>>
>> I have a strange problem with the Transform part of a ScanQuery that I get
>> a deserialize object exception for the SerializedLambda.
>>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>
>> Strange is that it works very well in other cases like:
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>   IgniteCache<Integer,String> cache =
>> ignite.createCache("ScanTransformTest");
>>   cache.put(1, "Hello");
>>
>>   // 1  Just a broadcast with a IgniteClosure works!
>>   compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
>>
>>   // 2 Running the ScanQuery on the remote server. Works too!
>>   compute.broadcast(() -> {
>>     try(QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), (a) -> 234L)){
>>       for( Long row : cursor) {
>>         return row;
>>       }
>>     }
>>     return 0;
>>   }).forEach(System.out::println);
>>
>>
>>   // 3 But running it from the client side it fails!
>>   try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> 345L)){
>>     for( Long row : cursor) {
>>       System.out.println(row);
>>       break;
>>     }
>>   }
>>
>> }
>>
>> Also changing the Lambda to a actually class implementing IgniteClosure
>> results in the same problem.
>>
>> Any idea what I'm doing wrong?
>>
>> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> With java openjdk 11.0.7 2020-04-14
>>
>> Thank you!
>>
>> Best,
>> Rafael
>>
>>
>> Error Log:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>         at
>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>         at
>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>         at
>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>         at ScanQueryTest.main(ScanQueryTest.java:49)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>         at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>         at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>         at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>         at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>         at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>         at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>         at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>         at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>         ... 14 more
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> read field [name=capturingClass]
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>         at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>         ... 20 more
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> ScanQueryTest
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>         at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>         at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>         at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>         ... 21 more
>> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>>         at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>         at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>         at java.base/java.lang.Class.forName0(Native Method)
>>         at java.base/java.lang.Class.forName(Class.java:398)
>>         at
>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>         at
>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>         at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>         ... 29 more
>>
>>
>>
>>
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg University
>>
>> https://heigit.org | [hidden email] | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>

--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger


--
-
Denis

rtroilo rtroilo
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hi Ilya, hey Denis,

thank you both of you for taking a look into this problem.

How would you like get the reproducing stand-alone runnable?

I attached a single file which reproduces the problem.

Do you like to have it in any other way?


Some comments:
In this issue, it does not matter if the Transformer Class is a static member class or a separate class it self. Both produce the exception.

The Transformer (IgniteClosure) class works fine in a broadcast.

  // simple IgniteClosure broadcast
  ignite.compute().broadcast(new Transformer<>(), "broadcast").forEach(System.out::println);

and for a scanquery with transform from within remote job (like this broadcast)

  // broadcast which return the result the scanquery transform
  ignite.compute().broadcast(() -> {
    return cache.query(new ScanQuery<>(), new Transformer<>()).getAll();
  }).stream().flatMap(List::stream).forEach(System.out::println);

but does not work on the client side directly

  // does throw a ClassNotFound Exception about the Transformer Class.
  cache.query(new ScanQuery<>(), new Transformer<>()).getAll().forEach(System.out::println);


Thank you.

Let me know if I can do more to help you to find the problem.

Best,
Rafael




On 7/2/20 5:39 PM, Ilya Kasnacheev wrote:
> Hello!
>
> I would like to start with a reproducer. Rafael, can you please throw
> together a runnable stand-alone reproducer of this issue?
>
> Thanks,
>

--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University

https://heigit.org | [hidden email] | phone +49-6221-533 484

Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany

Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger

ScanQueryWithTransform.java (2K) Download Attachment
Rafael Troilo Rafael Troilo
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Sorry I messed up the ScanQueryWithTransform.java a little bit.
This is the clean version.

Thank you.


On 7/3/20 2:19 PM, Rafael Troilo wrote:

> Hi Ilya, hey Denis,
>
> thank you both of you for taking a look into this problem.
>
> How would you like get the reproducing stand-alone runnable?
>
> I attached a single file which reproduces the problem.
>
> Do you like to have it in any other way?
>
>
> Some comments:
> In this issue, it does not matter if the Transformer Class is a static member class or a separate class it self. Both produce the exception.
>
> The Transformer (IgniteClosure) class works fine in a broadcast.
>
>   // simple IgniteClosure broadcast
>   ignite.compute().broadcast(new Transformer<>(), "broadcast").forEach(System.out::println);
>
> and for a scanquery with transform from within remote job (like this broadcast)
>
>   // broadcast which return the result the scanquery transform
>   ignite.compute().broadcast(() -> {
>     return cache.query(new ScanQuery<>(), new Transformer<>()).getAll();
>   }).stream().flatMap(List::stream).forEach(System.out::println);
>
> but does not work on the client side directly
>
>   // does throw a ClassNotFound Exception about the Transformer Class.
>   cache.query(new ScanQuery<>(), new Transformer<>()).getAll().forEach(System.out::println);
>
>
> Thank you.
>
> Let me know if I can do more to help you to find the problem.
>
> Best,
> Rafael
>
>
>
>
> On 7/2/20 5:39 PM, Ilya Kasnacheev wrote:
>> Hello!
>>
>> I would like to start with a reproducer. Rafael, can you please throw
>> together a runnable stand-alone reproducer of this issue?
>>
>> Thanks,
>>
>
--
Rafael Troilo, Dipl.-Inform. (FH)
   GIScience Research Group
 
   Heidelberg Institute for Geoinformation Technology

   [hidden email]
   http://giscience.uni-hd.de
   http://www.geog.uni-heidelberg.de/gis/heigit.html
 
   Berliner Str. 45 (Mathematikon), D-69120 Heidelberg, Germany
   fon: +49(0)6221 / 54 / 19704

ScanQueryWithTransform.java (1K) Download Attachment
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Hello!

There is now a ticket about this issue, which I indeed can see:

While it is not fixed, you will have to make sure that filter code is deployed to all nodes.

Regards,
--
Ilya Kasnacheev


пт, 3 июл. 2020 г. в 15:50, Rafael Troilo <[hidden email]>:
Sorry I messed up the ScanQueryWithTransform.java a little bit.
This is the clean version.

Thank you.


On 7/3/20 2:19 PM, Rafael Troilo wrote:
> Hi Ilya, hey Denis,
>
> thank you both of you for taking a look into this problem.
>
> How would you like get the reproducing stand-alone runnable?
>
> I attached a single file which reproduces the problem.
>
> Do you like to have it in any other way?
>
>
> Some comments:
> In this issue, it does not matter if the Transformer Class is a static member class or a separate class it self. Both produce the exception.
>
> The Transformer (IgniteClosure) class works fine in a broadcast.
>
>   // simple IgniteClosure broadcast
>   ignite.compute().broadcast(new Transformer<>(), "broadcast").forEach(System.out::println);
>
> and for a scanquery with transform from within remote job (like this broadcast)
>
>   // broadcast which return the result the scanquery transform
>   ignite.compute().broadcast(() -> {
>     return cache.query(new ScanQuery<>(), new Transformer<>()).getAll();
>   }).stream().flatMap(List::stream).forEach(System.out::println);
>
> but does not work on the client side directly
>
>   // does throw a ClassNotFound Exception about the Transformer Class.
>   cache.query(new ScanQuery<>(), new Transformer<>()).getAll().forEach(System.out::println);
>
>
> Thank you.
>
> Let me know if I can do more to help you to find the problem.
>
> Best,
> Rafael
>
>
>
>
> On 7/2/20 5:39 PM, Ilya Kasnacheev wrote:
>> Hello!
>>
>> I would like to start with a reproducer. Rafael, can you please throw
>> together a runnable stand-alone reproducer of this issue?
>>
>> Thanks,
>>
>

--
Rafael Troilo, Dipl.-Inform. (FH)
   GIScience Research Group

   Heidelberg Institute for Geoinformation Technology

   [hidden email]
   http://giscience.uni-hd.de
   http://www.geog.uni-heidelberg.de/gis/heigit.html

   Berliner Str. 45 (Mathematikon), D-69120 Heidelberg, Germany
   fon: +49(0)6221 / 54 / 19704
Rafael Troilo Rafael Troilo
Reply | Threaded
Open this post in threaded view
|

Re: ScanQuery Transform: fail to deserialze! ignite 2.8.0, java 11

Thank you Ilya for creating the issue.

And a big thank you that you already fixed it :-).

Best,
Rafael

On 7/3/20 5:56 PM, Ilya Kasnacheev wrote:

> Hello!
>
> There is now a ticket about this issue, which I indeed can see:
> https://issues.apache.org/jira/browse/IGNITE-13212
>
> While it is not fixed, you will have to make sure that filter code is
> deployed to all nodes.
>
> Regards,
>

--
Rafael Troilo, Dipl.-Inform. (FH)
   GIScience Research Group
 
   Heidelberg Institute for Geoinformation Technology

   [hidden email]
   http://giscience.uni-hd.de
   http://www.geog.uni-heidelberg.de/gis/heigit.html
 
   Berliner Str. 45 (Mathematikon), D-69120 Heidelberg, Germany
   fon: +49(0)6221 / 54 / 19704