IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache

classic Classic list List threaded Threaded
5 messages Options
userx userx
Reply | Threaded
Open this post in threaded view
|

IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache

Hi all,

I am using IgniteDataStreamer to write to a cache. As a part of my testing,
I started 2 servers on a local node and 1 client locally. I put everything
in debug mode in eclipse, and put a debug point where I am calling
IgniteDataStreamer.addData(). After that i let 2-3 entries to be written to
the cache, I then stop at the same debug point before I let it write the
next entry. Just at that time, I kill one of the two servers (I see some
java.net.SocketException in the client log) and then let the client continue
to write rest of the entries, since one of the servers is still running. In
spite of the fact that my cache is in 'REPLICATED' mode and the
cachesyncwrite mode is FULL_SYNC, addData method did not throw an exception
and completed successfully.

Why should it be possible with the given cache modes ? FULL_SYNC shouldn't
let it complete if one of the servers go down, isn't it ?





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
mcherkasov mcherkasov
Reply | Threaded
Open this post in threaded view
|

Re: IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache

Hi,

it works as expected, with REPLICATED cache you can't lose you data while 
you have at least 1 server node alive.

Why do you think it should throw an exception?

Thanks,
Mikhail.

On Tue, Sep 5, 2017 at 5:05 PM, userx <[hidden email]> wrote:
Hi all,

I am using IgniteDataStreamer to write to a cache. As a part of my testing,
I started 2 servers on a local node and 1 client locally. I put everything
in debug mode in eclipse, and put a debug point where I am calling
IgniteDataStreamer.addData(). After that i let 2-3 entries to be written to
the cache, I then stop at the same debug point before I let it write the
next entry. Just at that time, I kill one of the two servers (I see some
java.net.SocketException in the client log) and then let the client continue
to write rest of the entries, since one of the servers is still running. In
spite of the fact that my cache is in 'REPLICATED' mode and the
cachesyncwrite mode is FULL_SYNC, addData method did not throw an exception
and completed successfully.

Why should it be possible with the given cache modes ? FULL_SYNC shouldn't
let it complete if one of the servers go down, isn't it ?





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/



--
Thanks,
Mikhail.
userx userx
Reply | Threaded
Open this post in threaded view
|

Re: IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache

Hi Mikhail,

I am just trying to understand the behavior of addData method in conjunction
with FULL_SYNC.

If we have just one server node left, then we are not really replicating,
isn't it ? So let's say we have to persist 2 entries, and after 1 write and
replication, one of the server goes down, then eventually the second write
is just written to one server and replication does not come into the
picture.

Again, I am just trying to understand whats the responsibility of addData
keeping in mind it throws exception in its declaration.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
mcherkasov mcherkasov
Reply | Threaded
Open this post in threaded view
|

Re: IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache

I think javadoc is the best source for this:

 /**
     * Flag indicating that Ignite should wait for write or commit replies
from all nodes.
     * This behavior guarantees that whenever any of the atomic or
transactional writes
     * complete, all other participating nodes which cache the written data
have been updated.
     */

so with FULL_SYNC client node will wait for data will be saved on the
primary node and backup node.
if you have REPLICATED cache that means you have 1 primary node and all
other nodes in the cluster
store backups, so in your case, you lost one backup and that's it. Data was
saved.

You right that now you have cluster consists of only 1 node, but you can
start a new node or even hundred nodes,
and data will be replicated to all new nodes.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: IgniteDataStreamer.addData - behavior for a FULL_SYNC Cache



On Tue, Sep 5, 2017 at 10:51 AM, mcherkasov <[hidden email]> wrote:
I think javadoc is the best source for this:

 /**
     * Flag indicating that Ignite should wait for write or commit replies
from all nodes.
     * This behavior guarantees that whenever any of the atomic or
transactional writes
     * complete, all other participating nodes which cache the written data
have been updated.
     */

so with FULL_SYNC client node will wait for data will be saved on the
primary node and backup node.
if you have REPLICATED cache that means you have 1 primary node and all
other nodes in the cluster
store backups, so in your case, you lost one backup and that's it. Data was
saved.

I don't think this is exactly true. The client node is calling addData(...) method and will not wait for anything, IgniteDataStreamer is completely asynchronous. 

However, the primary-key server node will wait for the backup server nodes to be updated before responding back to the client. 

In case of REPLICATED cache, the primary node will wait until all other nodes are updated, so essentially, all nodes are guaranteed to have the latest state. If one of the nodes crashes, then other nodes will still have the state.
 

You right that now you have cluster consists of only 1 node, but you can
start a new node or even hundred nodes,
and data will be replicated to all new nodes.