Ignite Client Node Not Responding - Hung

classic Classic list List threaded Threaded
2 messages Options
Ravi Makwana Ravi Makwana
Reply | Threaded
Open this post in threaded view
|

Ignite Client Node Not Responding - Hung

Hi,

We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs.

Recently we have noticed that our application is not responding and all the calls  failed.

App server has 32 GB RAM & We are specifying JVM Heap = 8 GB

I am sharing a client log and could you please suggest why the client node is not responding & what is the possible cause and how to resolve this issue?

Thanks & Regards,

Logs.rar (276K) Download Attachment
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Ignite Client Node Not Responding - Hung

Hello!

What I have noticed in your logs is a huge number of network errors:
        Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=vrnv02ax04705.INT.CARLSONWAGONLIT.COM/10.212.120.67:51605, err=Failed to read remote n
ode recovery handshake (connection closed).]^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462)^M
               ... 14 more^M
       Caused by: class org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$HandshakeException: Failed to read remote node recovery handshake (connection closed).^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3803)^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3313)^M
               ... 14 more^M
       Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=vrnv02ax04705.INT.CARLSONWAGONLIT.COM/10.212.120.67:51605, err=Failed to read remote n
ode recovery handshake (connection closed).]^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462)^M
               ... 14 more^M
       Caused by: class org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$HandshakeException: Failed to read remote node recovery handshake (connection closed).^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3803)^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3313)^M
               ... 14 more^M
       Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=vrnv02ax04705.INT.CARLSONWAGONLIT.COM/10.212.120.67:51605, err=Failed to read remote n
ode recovery handshake (connection closed).]^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462)^M
               ... 14 more^M
       Caused by: class org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$HandshakeException: Failed to read remote node recovery handshake (connection closed).^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3803)^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3313)^M
               ... 14 more^M
       Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=vrnv02ax04705.INT.CARLSONWAGONLIT.COM/10.212.120.67:51605, err=Failed to read remote n
ode recovery handshake (connection closed).]^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462)^M
               ... 14 more^M
       Caused by: class org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$HandshakeException: Failed to read remote node recovery handshake (connection closed).^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3803)^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3313)^M
               ... 14 more^M
       Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address [addr=vrnv02ax04705.INT.CARLSONWAGONLIT.COM/10.212.120.67:51605, err=Failed to read remote n
ode recovery handshake (connection closed).]^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3462)^M
               ... 14 more^M
       Caused by: class org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$HandshakeException: Failed to read remote node recovery handshake (connection closed).^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeTcpHandshake(TcpCommunicationSpi.java:3803)^M
               at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3313)^M
               ... 14 more^M


Have you tried making sure that the nodes are in fact connectable by Communication?

Regards,
--
Ilya Kasnacheev


пн, 2 нояб. 2020 г. в 13:17, Ravi Makwana <[hidden email]>:
Hi,

We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs.

Recently we have noticed that our application is not responding and all the calls  failed.

App server has 32 GB RAM & We are specifying JVM Heap = 8 GB

I am sharing a client log and could you please suggest why the client node is not responding & what is the possible cause and how to resolve this issue?

Thanks & Regards,