Ignite 2.7.0: unresponsive: Found long running cache future -

classic Classic list List threaded Threaded
4 messages Options
Mahesh Renduchintala Mahesh Renduchintala
Reply | Threaded
Open this post in threaded view
|

Ignite 2.7.0: unresponsive: Found long running cache future -

Hi, 



After a while, an ignite node becomes unresponsive printing


>>Found long-running cache future


Please check the attached log. 


What does "Found long-running cache future" mean?

please throw some light. 


regards

mahesh



longpendingFuture.zip (195K) Download Attachment
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 2.7.0: unresponsive: Found long running cache future -

Igniters, 

Does this remind you any long-running transaction, starvation or deadlock? It's not obvious from the attached files what caused the issue. Mahesh please share all the logs from all the nodes and apps for analysis.


-
Denis


On Sun, Jul 28, 2019 at 8:50 PM Mahesh Renduchintala <[hidden email]> wrote:

Hi, 



After a while, an ignite node becomes unresponsive printing


>>Found long-running cache future


Please check the attached log. 


What does "Found long-running cache future" mean?

please throw some light. 


regards

mahesh



longpendingFuture.log (3M) Download Attachment
Mahesh Renduchintala Mahesh Renduchintala
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 2.7.0: unresponsive: Found long running cache future -

Denis, 


It is quite difficult to capture all clients logs from our production environment. 

I gave you the server log.


I can devise a better test if you can say explain this error?

What does "long running cache futures" mean?


regards
mahesh
Loredana Radulescu Ivanoff Loredana Radulescu Ivanoff
Reply | Threaded
Open this post in threaded view
|

Re: Ignite 2.7.0: unresponsive: Found long running cache future -

In reply to this post by dmagda
Hello,

I have recently encountered this "long-running cache future" message while doing stress tests on Ignite (we run it embedded inside an application server). I put CPU pressure on two out of four nodes using stress-ng, that rendered the two nodes irresponsive for the duration of the test, and then one of the other two "healthy" nodes started logging the "long running future" message.

In my opinion, it could be related to a cache operation that cannot complete due to another node(nodes?) that do not respond in time. I hope that helps a little.

On Tue, Jul 30, 2019 at 3:52 PM Denis Magda <[hidden email]> wrote:
Igniters, 

Does this remind you any long-running transaction, starvation or deadlock? It's not obvious from the attached files what caused the issue. Mahesh please share all the logs from all the nodes and apps for analysis.


-
Denis


On Sun, Jul 28, 2019 at 8:50 PM Mahesh Renduchintala <[hidden email]> wrote:

Hi, 



After a while, an ignite node becomes unresponsive printing


>>Found long-running cache future


Please check the attached log. 


What does "Found long-running cache future" mean?

please throw some light. 


regards

mahesh