We are using ignite to share in memory data across the spark jobs. When we are running the spark job we see following warnings in ignite logs. We have done garbage collection tuning as suggested in apache ignite performance tips but still we see below warning.
[00:25:14,432][WARNING][jvm-pause-detector-worker] Possible too long JVM pause: 1430 milliseconds.
[14:43:44,960][WARNING][tcp-comm-worker-#1][TcpCommunicationSpi] Connect timed out (consider increasing 'failureDetectionTimeout' configuration property) [addr=/127.0.0
[14:43:44,960][WARNING][tcp-comm-worker-#1][TcpCommunicationSpi] Failed to connect to a remote node (make sure that destination node is alive and operating system firew