unable to start node

classic Classic list List threaded Threaded
12 messages Options
Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

unable to start node

Hi Igniters,

 

We have encountered below error in one of our cluster. After this, we even tried to clear the index and restarted but still the node doesn’t get started. It throws below errors.

 

[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error: "class o.a.i.IgniteCheckedException: Unexpected binary object class [type=class java.lang.String]" [50000-197]]]

 

Does this occurs if few records are corrupted? If yes, is there any way to correct it and start the node.

 

Also, attaching logs for reference.

 

Awaiting for response on this, thanks in advance !

 

 

Thanks and Regards,

Kamlesh Joshi

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


ignite-logs.log (1M) Download Attachment
Tom Black Tom Black
Reply | Threaded
Open this post in threaded view
|

Re: unable to start node

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.
Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node


Hi Tom,

We are using java 1.8.151 and Ignite 2.7.6.

Regards,
Kamlesh



From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.



"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

Let us know if it worked out for you. We'll document this procedure.

-
Denis


On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

Hi Tom,

We are using java 1.8.151 and Ignite 2.7.6.

Regards,
Kamlesh



From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.



"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.
Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

Denis

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:
[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

Let us know if it worked out for you. We'll document this procedure.

-
Denis


On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

Hi Tom,

We are using java 1.8.151 and Ignite 2.7.6.

Regards,
Kamlesh



From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.



"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

RE: [External]Re: unable to start node

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node

Kamlesh, 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

Denis

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

RE: [External]Re: unable to start node

Denis,

 

Yes, we do have table connected to that cache (we don’t update anything though SQL though) but its not created via CREATE TABLE command. We use XMLs to create caches.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 16:35
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh, 

 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

 

Denis

 

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node

Could you provide the schema of the table with all indexed fields?
Is it possible, that you by any chance inserted a String into this cache instead of a POJO with the fields like the table has?

Denis

чт, 3 сент. 2020 г. в 14:19, Kamlesh Joshi <[hidden email]>:

Denis,

 

Yes, we do have table connected to that cache (we don’t update anything though SQL though) but its not created via CREATE TABLE command. We use XMLs to create caches.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 16:35
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh, 

 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

 

Denis

 

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

RE: [External]Re: unable to start node

Denis,

 

Below is the cache config. We have used Binary Objects as well from some of thick clients to update cache.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 18:33
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Could you provide the schema of the table with all indexed fields?

Is it possible, that you by any chance inserted a String into this cache instead of a POJO with the fields like the table has?

 

Denis

 

чт, 3 сент. 2020 г. в 14:19, Kamlesh Joshi <[hidden email]>:

Denis,

 

Yes, we do have table connected to that cache (we don’t update anything though SQL though) but its not created via CREATE TABLE command. We use XMLs to create caches.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 16:35
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh, 

 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

 

Denis

 

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

Kamlesh Joshi Kamlesh Joshi
Reply | Threaded
Open this post in threaded view
|

RE: [External]Re: unable to start node

Denis,

 

Did u got any chance to go through this? Please update if you have..

 

Thanks and Regards,

Kamlesh Joshi

 

From: Kamlesh Joshi
Sent: 03 September 2020 19:54
To: [hidden email]
Subject: RE: [External]Re: unable to start node

 

Denis,

 

Below is the cache config. We have used Binary Objects as well from some of thick clients to update cache.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 18:33
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Could you provide the schema of the table with all indexed fields?

Is it possible, that you by any chance inserted a String into this cache instead of a POJO with the fields like the table has?

 

Denis

 

чт, 3 сент. 2020 г. в 14:19, Kamlesh Joshi <[hidden email]>:

Denis,

 

Yes, we do have table connected to that cache (we don’t update anything though SQL though) but its not created via CREATE TABLE command. We use XMLs to create caches.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 16:35
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh, 

 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

 

Denis

 

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: [External]Re: unable to start node

Hello!

I think it is possible that you have put a (String, CustomerChars) entry in this cache. Is this possible? Can you scan it to check?

Regards,
--
Ilya Kasnacheev


пн, 7 сент. 2020 г. в 09:14, Kamlesh Joshi <[hidden email]>:

Denis,

 

Did u got any chance to go through this? Please update if you have..

 

Thanks and Regards,

Kamlesh Joshi

 

From: Kamlesh Joshi
Sent: 03 September 2020 19:54
To: [hidden email]
Subject: RE: [External]Re: unable to start node

 

Denis,

 

Below is the cache config. We have used Binary Objects as well from some of thick clients to update cache.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 18:33
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Could you provide the schema of the table with all indexed fields?

Is it possible, that you by any chance inserted a String into this cache instead of a POJO with the fields like the table has?

 

Denis

 

чт, 3 сент. 2020 г. в 14:19, Kamlesh Joshi <[hidden email]>:

Denis,

 

Yes, we do have table connected to that cache (we don’t update anything though SQL though) but its not created via CREATE TABLE command. We use XMLs to create caches.

 

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 16:35
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh, 

 

So do you have a SQL table connected to this cache? Query entity or table created through CREATE TABLE command?

 

Denis

 

чт, 3 сент. 2020 г. в 12:09, Kamlesh Joshi <[hidden email]>:

Hi Denis,

 

We cleared all the indices of that particular node and tried starting but no luck. Shall we do this activity for entire cluster ?

 

CustomerCharsCache, we have multiple thick clients which writes to this cache. Similarly, we have multiple caches with same setup but never observed this issue earlier (in version 2.6.0). We migrated to Ignite 2.7.6 around 3 months back.

 

Please, let me know is there anything we should try to recover the node ?

 

Thanks and Regards,

Kamlesh Joshi

 

From: Denis Mekhanikov <[hidden email]>
Sent: 03 September 2020 14:08
To: user <[hidden email]>
Subject: Re: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

It seems that there is a String value in the CustomerCharsCache cache that is being rebalanced to the new node, where it tries to be indexed.

Kamlesh, do you have an SQL table that stores data in the CustomerCharsCache? Did you put any data to this cache using cache API directly?

 

Denis

 

чт, 3 сент. 2020 г. в 02:17, Denis Magda <[hidden email]>:

[hidden email][hidden email][hidden email], did you guys see this issue before? It looks like a bug in the SQL engine.

 

Kamlesh, in the meantime, try to follow the procedure described in this JIRA ticket (you need to clean the index that we'll be rebuild on the startup):

 

Let us know if it worked out for you. We'll document this procedure.


-

Denis

 

 

On Wed, Sep 2, 2020 at 10:07 AM Kamlesh Joshi <[hidden email]> wrote:

 

Hi Tom,

 

We are using java 1.8.151 and Ignite 2.7.6.

 

Regards,

Kamlesh

 

 


From: Tom Black <[hidden email]>
Sent: Wednesday, 2 September, 2020, 1:31 pm
To: [hidden email]
Subject: [External]Re: unable to start node

 

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Kamlesh Joshi wrote:
> */[2020-08-28T19:05:03,296][ERROR][sys-#940187%EDIFReplica%][] JVM will
> be halted immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=org.h2.message.DbException: General error:
> "class o.a.i.IgniteCheckedException: Unexpected binary object class
> [type=class java.lang.String]" [50000-197]]]/*

what ignite and java version are you using?

regards.

 


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."


"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."