Network Segmentation Behaviour

classic Classic list List threaded Threaded
1 message Options
sue sue
Reply | Threaded
Open this post in threaded view
|

Network Segmentation Behaviour

Hi

I've been testing Ignite (2.8.1) and it's behaviour under network
segmentation.
According to the docs, Ignite nodes should be able to detect network
segmentation and apply the configured SegmentationPolicy.

However the segmentation handling didn't trigger as I would have expected it
to do.
For my tests, I setup three cluster nodes c1, c2 and c3 running in docker
containers, all competing for a shared IgniteLock instance in a loop.
Then I used iptables in container c2 to drop all incoming and outgoing
packages on that node.
After a few seconds I got the following events:

c1:
- EVT_NODE_FAILED for c2

c2:
- EVT_NODE_FAILED for c1
- EVT_NODE_FAILED for c3

c3:
- EVT_NODE_FAILED for c2

Then I reset the iptables rules expecting that c2 would rejoin the cluster
and detect segmentation.
However this didn't happen, c2 just keeps running as a second standalone
cluster instance.
Only after restarting c2 it rejoined the cluster.

Eventyally I was able to trigger the EVT_NODE_SEGMENTED event by pausing the
c2 container for 1minute. After resuming, c2 detects the segmentation and
runs the segmentation policy as excepcted.

Is this behaviour correct? Shouldn't the Ignite cluster be able to recover
from the first scenario?
During a network segmentation no packages would be able to move between
nodes, so the iptables approach should be realistic in my oppinion.

Maybe I have some wrong assumptions about network segmentation so any
feedback would be greatly appreciated.

Cheers Sam





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/