Getting lots of Accepted "incoming communication connection"

classic Classic list List threaded Threaded
4 messages Options
javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Getting lots of Accepted "incoming communication connection"

Hi getting allot of these messages. It seems to be coming from a single client. The client was restarted briefly because of updates...

The client is an apache flink streaming job.

[21:38:12,775][INFO][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918]
[21:38:12,776][WARNING][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Close incoming connection, unknown node [nodeId=3f5f544a-957d-460a-bf62-3bf5141afa7e, ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=5063179, bytesSent=3190214, bytesRcvd0=84, bytesSent0=36, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=xxxxxx, finished=false, heartbeatTs=1580938692770, hashCode=1501311954, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#25%xxxxxx%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=42 lim=42 cap=32768], inRecovery=null, outRecovery=null, super=GridNioSessionImpl [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918, createTime=1580938692770, closeTime=0, bytesSent=18, bytesRcvd=42, bytesSent0=18, bytesRcvd0=42, sndSchedTime=1580938692770, lastSndTime=1580938692770, lastRcvTime=1580938692770, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@1778eda4, directMode=true], GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Getting lots of Accepted "incoming communication connection"

Hello!

I suspect interference from different clusters, or too long timeouts on client preventing it from understanding that it's dropped from cluster already.

Regards,
--
Ilya Kasnacheev


чт, 6 февр. 2020 г. в 00:44, John Smith <[hidden email]>:
Hi getting allot of these messages. It seems to be coming from a single client. The client was restarted briefly because of updates...

The client is an apache flink streaming job.

[21:38:12,775][INFO][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918]
[21:38:12,776][WARNING][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Close incoming connection, unknown node [nodeId=3f5f544a-957d-460a-bf62-3bf5141afa7e, ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=5063179, bytesSent=3190214, bytesRcvd0=84, bytesSent0=36, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=xxxxxx, finished=false, heartbeatTs=1580938692770, hashCode=1501311954, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#25%xxxxxx%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=42 lim=42 cap=32768], inRecovery=null, outRecovery=null, super=GridNioSessionImpl [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918, createTime=1580938692770, closeTime=0, bytesSent=18, bytesRcvd=42, bytesSent0=18, bytesRcvd0=42, sndSchedTime=1580938692770, lastSndTime=1580938692770, lastRcvTime=1580938692770, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@1778eda4, directMode=true], GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: Getting lots of Accepted "incoming communication connection"

Ok it happened for a few hours, 4 or so, and then it went back to normal. Everything seems ok. But why for some many hours?

On Thu., Feb. 6, 2020, 11:20 a.m. Ilya Kasnacheev, <[hidden email]> wrote:
Hello!

I suspect interference from different clusters, or too long timeouts on client preventing it from understanding that it's dropped from cluster already.

Regards,
--
Ilya Kasnacheev


чт, 6 февр. 2020 г. в 00:44, John Smith <[hidden email]>:
Hi getting allot of these messages. It seems to be coming from a single client. The client was restarted briefly because of updates...

The client is an apache flink streaming job.

[21:38:12,775][INFO][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918]
[21:38:12,776][WARNING][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Close incoming connection, unknown node [nodeId=3f5f544a-957d-460a-bf62-3bf5141afa7e, ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=5063179, bytesSent=3190214, bytesRcvd0=84, bytesSent0=36, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=xxxxxx, finished=false, heartbeatTs=1580938692770, hashCode=1501311954, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#25%xxxxxx%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=42 lim=42 cap=32768], inRecovery=null, outRecovery=null, super=GridNioSessionImpl [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918, createTime=1580938692770, closeTime=0, bytesSent=18, bytesRcvd=42, bytesSent0=18, bytesRcvd0=42, sndSchedTime=1580938692770, lastSndTime=1580938692770, lastRcvTime=1580938692770, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@1778eda4, directMode=true], GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Getting lots of Accepted "incoming communication connection"

Hello!

Maybe you have such long timeout somewhere (like, 10000 seconds)?

Maybe there were some network disruption, such as partial connectivity degradation? Maybe it's just random and you should expect a recurrence, unless you fix something in TCP stack configuration.

It's hard to say.

Regards,
--
Ilya Kasnacheev


пт, 7 февр. 2020 г. в 00:32, John Smith <[hidden email]>:
Ok it happened for a few hours, 4 or so, and then it went back to normal. Everything seems ok. But why for some many hours?

On Thu., Feb. 6, 2020, 11:20 a.m. Ilya Kasnacheev, <[hidden email]> wrote:
Hello!

I suspect interference from different clusters, or too long timeouts on client preventing it from understanding that it's dropped from cluster already.

Regards,
--
Ilya Kasnacheev


чт, 6 февр. 2020 г. в 00:44, John Smith <[hidden email]>:
Hi getting allot of these messages. It seems to be coming from a single client. The client was restarted briefly because of updates...

The client is an apache flink streaming job.

[21:38:12,775][INFO][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918]
[21:38:12,776][WARNING][grid-nio-worker-tcp-comm-1-#25%xxxxxx%][TcpCommunicationSpi] Close incoming connection, unknown node [nodeId=3f5f544a-957d-460a-bf62-3bf5141afa7e, ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=5063179, bytesSent=3190214, bytesRcvd0=84, bytesSent0=36, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=xxxxxx, finished=false, heartbeatTs=1580938692770, hashCode=1501311954, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#25%xxxxxx%]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=42 lim=42 cap=32768], inRecovery=null, outRecovery=null, super=GridNioSessionImpl [locAddr=/xxx.xxx.xxx.79:47100, rmtAddr=/xxx.xxx.xxx.65:42918, createTime=1580938692770, closeTime=0, bytesSent=18, bytesRcvd=42, bytesSent0=18, bytesRcvd0=42, sndSchedTime=1580938692770, lastSndTime=1580938692770, lastRcvTime=1580938692770, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@1778eda4, directMode=true], GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]