ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

classic Classic list List threaded Threaded
5 messages Options
ifedorenkov-td ifedorenkov-td
Reply | Threaded
Open this post in threaded view
|

ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

This post was updated on .
Dear Igniters,

I have come across a bug in ContinuousQueryWithTransformer. The thing is that the query is failing to deliver some updates when a custom java class is being used as a transformed value. On attempt to serialize the custom type Ignite marshaller throws an exception, stated that the class hasn’t been registered yet. Sometimes it takes like 3-4 attempts for a marshaller to serialize the required object.
It would’ve been kind of ok if the event would have been eventually delivered, however that is not the case.

The code that I am using to reproduce the issue: https://gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f
The output:

Put account: Person#1
Put account: Person#2
Caught updated account: Person#1
Caught updated account: Person#2
Put account: Person#3
Caught updated account: Person#3
Put account: Person#4
Caught updated account: Person#4
Put account: Person#5
Caught updated account: Person#5
Put account: Person#6
Caught updated account: Person#6
Put account: Person#7
Caught updated account: Person#7
Put account: Person#8
Put account: Person#9
Caught updated account: Person#9
Put account: Person#10
Caught updated account: Person#10
Put account: Person#11
Caught updated account: Person#11
Put account: Person#12


As you can see the update for Person#8 has been missed.

Workaround: use Externalizable type, so that Ignite won’t fail to serialize the object from the very beginning

Best regards,
Ivan Fedorenkov
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hello!

Can you please share the complete runnable project? In your gist, POJO classes are missing.

Regards,
--
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 13:57, <[hidden email]>:
Dear Igniters,

I have come across a bug in ContinuousQueryWithTransformer. The thing is that the query is failing to deliver some updates when a custom java class is being used as a transformed value. On attempt to serialize the custom type Ignite marshaller throws an exception, stated that the class hasn’t been registered yet. Sometimes it takes like 3-4 attempts for a marshaller to serialize the required object.
It would’ve been kind of ok if the event would be eventually delivered, however that is not the case.

The code that I am using to reproduce the issue: https://gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f
The output:

Put account: Person#1
Put account: Person#2
Caught updated account: Person#1
Caught updated account: Person#2
Put account: Person#3
Caught updated account: Person#3
Put account: Person#4
Caught updated account: Person#4
Put account: Person#5
Caught updated account: Person#5
Put account: Person#6
Caught updated account: Person#6
Put account: Person#7
Caught updated account: Person#7
Put account: Person#8
Put account: Person#9
Caught updated account: Person#9
Put account: Person#10
Caught updated account: Person#10
Put account: Person#11
Caught updated account: Person#11
Put account: Person#12


As you can see the update for Person#8 has been missed.

Workaround: use Externalizable type, so that Ignite won’t fail to serialize the object from the very beginning

Best regards,
Ivan Fedorenkov
ifedorenkov-td ifedorenkov-td
Reply | Threaded
Open this post in threaded view
|

Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hi Ilya,

Sure, the complete code could be found here: https://github.com/ifedorenkov-tda/ignite-experiments/tree/continuous-query-with-transformer-missing-events
Just start 3-4 server nodes and then 1 client node.

Best regards,
Ivan Fedorenkov

From: Ilya Kasnacheev <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Wednesday, February 10, 2021 at 12:55 PM
To: "[hidden email]" <[hidden email]>
Subject: Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hello!

Can you please share the complete runnable project? In your gist, POJO classes are missing.

Regards,
--
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 13:57, <[hidden email]<mailto:[hidden email]>>:
Dear Igniters,

I have come across a bug in ContinuousQueryWithTransformer. The thing is that the query is failing to deliver some updates when a custom java class is being used as a transformed value. On attempt to serialize the custom type Ignite marshaller throws an exception, stated that the class hasn’t been registered yet. Sometimes it takes like 3-4 attempts for a marshaller to serialize the required object.
It would’ve been kind of ok if the event would be eventually delivered, however that is not the case.

The code that I am using to reproduce the issue: https://gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f<https://urldefense.com/v3/__https:/gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f__;!!PNmPxbqrAcYR!2CsruMvfwmxLzj1Q56eBNRhEskcShguDsA2lkrSlDMJBWIiN_0chTDxcvIXXE50dhRk0gwIpag$>
The output:

Put account: Person#1
Put account: Person#2
Caught updated account: Person#1
Caught updated account: Person#2
Put account: Person#3
Caught updated account: Person#3
Put account: Person#4
Caught updated account: Person#4
Put account: Person#5
Caught updated account: Person#5
Put account: Person#6
Caught updated account: Person#6
Put account: Person#7
Caught updated account: Person#7
Put account: Person#8
Put account: Person#9
Caught updated account: Person#9
Put account: Person#10
Caught updated account: Person#10
Put account: Person#11
Caught updated account: Person#11
Put account: Person#12


As you can see the update for Person#8 has been missed.

Workaround: use Externalizable type, so that Ignite won’t fail to serialize the object from the very beginning

Best regards,
Ivan Fedorenkov
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hello!

This does indeed look like a bug. I have filed https://issues.apache.org/jira/browse/IGNITE-14161

Regards,
--
Ilya Kasnacheev


ср, 10 февр. 2021 г. в 15:48, <[hidden email]>:
Hi Ilya,

Sure, the complete code could be found here: https://github.com/ifedorenkov-tda/ignite-experiments/tree/continuous-query-with-transformer-missing-events
Just start 3-4 server nodes and then 1 client node.

Best regards,
Ivan Fedorenkov

From: Ilya Kasnacheev <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Wednesday, February 10, 2021 at 12:55 PM
To: "[hidden email]" <[hidden email]>
Subject: Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hello!

Can you please share the complete runnable project? In your gist, POJO classes are missing.

Regards,
--
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 13:57, <[hidden email]<mailto:[hidden email]>>:
Dear Igniters,

I have come across a bug in ContinuousQueryWithTransformer. The thing is that the query is failing to deliver some updates when a custom java class is being used as a transformed value. On attempt to serialize the custom type Ignite marshaller throws an exception, stated that the class hasn’t been registered yet. Sometimes it takes like 3-4 attempts for a marshaller to serialize the required object.
It would’ve been kind of ok if the event would be eventually delivered, however that is not the case.

The code that I am using to reproduce the issue: https://gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f<https://urldefense.com/v3/__https:/gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f__;!!PNmPxbqrAcYR!2CsruMvfwmxLzj1Q56eBNRhEskcShguDsA2lkrSlDMJBWIiN_0chTDxcvIXXE50dhRk0gwIpag$>
The output:

Put account: Person#1
Put account: Person#2
Caught updated account: Person#1
Caught updated account: Person#2
Put account: Person#3
Caught updated account: Person#3
Put account: Person#4
Caught updated account: Person#4
Put account: Person#5
Caught updated account: Person#5
Put account: Person#6
Caught updated account: Person#6
Put account: Person#7
Caught updated account: Person#7
Put account: Person#8
Put account: Person#9
Caught updated account: Person#9
Put account: Person#10
Caught updated account: Person#10
Put account: Person#11
Caught updated account: Person#11
Put account: Person#12


As you can see the update for Person#8 has been missed.

Workaround: use Externalizable type, so that Ignite won’t fail to serialize the object from the very beginning

Best regards,
Ivan Fedorenkov
ivan.fedorenkov ivan.fedorenkov
Reply | Threaded
Open this post in threaded view
|

Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Thank you, Ilya!

ср, 10 февр. 2021 г. в 17:01, Ilya Kasnacheev <[hidden email]>:
Hello!

This does indeed look like a bug. I have filed https://issues.apache.org/jira/browse/IGNITE-14161

Regards,
--
Ilya Kasnacheev


ср, 10 февр. 2021 г. в 15:48, <[hidden email]>:
Hi Ilya,

Sure, the complete code could be found here: https://github.com/ifedorenkov-tda/ignite-experiments/tree/continuous-query-with-transformer-missing-events
Just start 3-4 server nodes and then 1 client node.

Best regards,
Ivan Fedorenkov

From: Ilya Kasnacheev <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Wednesday, February 10, 2021 at 12:55 PM
To: "[hidden email]" <[hidden email]>
Subject: Re: ContinuousQueryWithTransformer fails to deliver an update when a custom java class is being used as a transformed value

Hello!

Can you please share the complete runnable project? In your gist, POJO classes are missing.

Regards,
--
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 13:57, <[hidden email]<mailto:[hidden email]>>:
Dear Igniters,

I have come across a bug in ContinuousQueryWithTransformer. The thing is that the query is failing to deliver some updates when a custom java class is being used as a transformed value. On attempt to serialize the custom type Ignite marshaller throws an exception, stated that the class hasn’t been registered yet. Sometimes it takes like 3-4 attempts for a marshaller to serialize the required object.
It would’ve been kind of ok if the event would be eventually delivered, however that is not the case.

The code that I am using to reproduce the issue: https://gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f<https://urldefense.com/v3/__https:/gist.github.com/ifedorenkov-tda/069b33edcd4fdb0afd68862c1b46563f__;!!PNmPxbqrAcYR!2CsruMvfwmxLzj1Q56eBNRhEskcShguDsA2lkrSlDMJBWIiN_0chTDxcvIXXE50dhRk0gwIpag$>
The output:

Put account: Person#1
Put account: Person#2
Caught updated account: Person#1
Caught updated account: Person#2
Put account: Person#3
Caught updated account: Person#3
Put account: Person#4
Caught updated account: Person#4
Put account: Person#5
Caught updated account: Person#5
Put account: Person#6
Caught updated account: Person#6
Put account: Person#7
Caught updated account: Person#7
Put account: Person#8
Put account: Person#9
Caught updated account: Person#9
Put account: Person#10
Caught updated account: Person#10
Put account: Person#11
Caught updated account: Person#11
Put account: Person#12


As you can see the update for Person#8 has been missed.

Workaround: use Externalizable type, so that Ignite won’t fail to serialize the object from the very beginning

Best regards,
Ivan Fedorenkov