Partition Loss Policy options

classic Classic list List threaded Threaded
7 messages Options
novicr novicr
Reply | Threaded
Open this post in threaded view
|

Partition Loss Policy options

PartitionLossPolicy controls access level to the cache in case any partitions for this cache are not available on the cluster.  As far as I can see this policy is only consulted for cache put/get operations.  Is there a way to prevent queries from returning results (force an exception) when cache data is partially unavailable?

thanks,
Roman


akurbanov akurbanov
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy options

Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
novicr novicr
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy options

I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. 


On Tue, Oct 2, 2018 at 1:37 PM akurbanov <[hidden email]> wrote:
Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
novicr novicr
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy options

Anton,
thanks for quick response.  Not sure if I'm setting wrong expectations.  Just tried sql query and that exhibited the same behavior.  Created a pull request with a test: https://github.com/novicr/ignite/pull/3.  

The test goes through the following steps:
1. creates a 4 node cluster with persistence enabled.  
2. creates 2 caches - with setBackups(1)
3. populates caches with 1000 elements
4. runs a sql query and prints result size: 1000
5. stops 2 nodes 
6. runs sql query from step 4 and prints result size: (something less than 1000).

Thanks,
Roman


On Tue, Oct 2, 2018 at 2:07 PM Roman Novichenok <[hidden email]> wrote:

I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. 


On Tue, Oct 2, 2018 at 1:37 PM akurbanov <[hidden email]> wrote:
Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

RE: Partition Loss Policy options

Hi,

 

I’ve tried your test and it works as expected, with some partitions lost and the final size being ~850 (~150 less than on the start).

Am I missing something?

 

Thanks,

Stan

 

From: [hidden email]
Sent: 2 октября 2018 г. 22:21
To: [hidden email]
Subject: Re: Partition Loss Policy options

 

Anton,

thanks for quick response.  Not sure if I'm setting wrong expectations.  Just tried sql query and that exhibited the same behavior.  Created a pull request with a test: https://github.com/novicr/ignite/pull/3.  

 

The test goes through the following steps:

1. creates a 4 node cluster with persistence enabled.  

2. creates 2 caches - with setBackups(1)

3. populates caches with 1000 elements

4. runs a sql query and prints result size: 1000

5. stops 2 nodes 

6. runs sql query from step 4 and prints result size: (something less than 1000).

 

Thanks,

Roman

 

 

On Tue, Oct 2, 2018 at 2:07 PM Roman Novichenok <[hidden email]> wrote:

I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. 

 

On Tue, Oct 2, 2018 at 1:37 PM akurbanov <[hidden email]> wrote:

Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

novicr novicr
Reply | Threaded
Open this post in threaded view
|

Re: Partition Loss Policy options

Stan,
thanks for looking into it.  I agree with you that this is the observed behaviour, but it is not what I would expect.

My expectation would be to get an exception when I attempt to query unavailable partitions.  Ideally this behavior would be dependent on the PartitionLossPolicy.  When READ_ONLY_SAFE or READ_WRITE_SAFE policy is selected, and the query condition does not explicitly specify which partitions it is interested in, then query should fail. Query could implicitly specify partitions by including indexed value in the where clause.  Simpler implementation could just raise exceptions on queries when policy is ..._SAFE and some partitions are unavailable.

Thanks again,
Roman

On Tue, Oct 9, 2018 at 2:54 PM Stanislav Lukyanov <[hidden email]> wrote:

Hi,

 

I’ve tried your test and it works as expected, with some partitions lost and the final size being ~850 (~150 less than on the start).

Am I missing something?

 

Thanks,

Stan

 

From: [hidden email]
Sent: 2 октября 2018 г. 22:21
To: [hidden email]
Subject: Re: Partition Loss Policy options

 

Anton,

thanks for quick response.  Not sure if I'm setting wrong expectations.  Just tried sql query and that exhibited the same behavior.  Created a pull request with a test: https://github.com/novicr/ignite/pull/3.  

 

The test goes through the following steps:

1. creates a 4 node cluster with persistence enabled.  

2. creates 2 caches - with setBackups(1)

3. populates caches with 1000 elements

4. runs a sql query and prints result size: 1000

5. stops 2 nodes 

6. runs sql query from step 4 and prints result size: (something less than 1000).

 

Thanks,

Roman

 

 

On Tue, Oct 2, 2018 at 2:07 PM Roman Novichenok <[hidden email]> wrote:

I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. 

 

On Tue, Oct 2, 2018 at 1:37 PM akurbanov <[hidden email]> wrote:

Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Stanislav Lukyanov Stanislav Lukyanov
Reply | Threaded
Open this post in threaded view
|

RE: Partition Loss Policy options

Oh, sure, you’re right.

Turns out it only works this way for in-memory caches, need to fix it for persistence.

Filed https://issues.apache.org/jira/browse/IGNITE-9841 for this.

 

Thanks for reporting this!

 

Stan

 

From: [hidden email]
Sent: 9 октября 2018 г. 22:46
To: [hidden email]
Subject: Re: Partition Loss Policy options

 

Stan,

thanks for looking into it.  I agree with you that this is the observed behaviour, but it is not what I would expect.

 

My expectation would be to get an exception when I attempt to query unavailable partitions.  Ideally this behavior would be dependent on the PartitionLossPolicy.  When READ_ONLY_SAFE or READ_WRITE_SAFE policy is selected, and the query condition does not explicitly specify which partitions it is interested in, then query should fail. Query could implicitly specify partitions by including indexed value in the where clause.  Simpler implementation could just raise exceptions on queries when policy is ..._SAFE and some partitions are unavailable.

 

Thanks again,

Roman

 

On Tue, Oct 9, 2018 at 2:54 PM Stanislav Lukyanov <[hidden email]> wrote:

Hi,

 

I’ve tried your test and it works as expected, with some partitions lost and the final size being ~850 (~150 less than on the start).

Am I missing something?

 

Thanks,

Stan

 

From: [hidden email]
Sent: 2 октября 2018 г. 22:21
To: [hidden email]
Subject: Re: Partition Loss Policy options

 

Anton,

thanks for quick response.  Not sure if I'm setting wrong expectations.  Just tried sql query and that exhibited the same behavior.  Created a pull request with a test: https://github.com/novicr/ignite/pull/3.  

 

The test goes through the following steps:

1. creates a 4 node cluster with persistence enabled.  

2. creates 2 caches - with setBackups(1)

3. populates caches with 1000 elements

4. runs a sql query and prints result size: 1000

5. stops 2 nodes 

6. runs sql query from step 4 and prints result size: (something less than 1000).

 

Thanks,

Roman

 

 

On Tue, Oct 2, 2018 at 2:07 PM Roman Novichenok <[hidden email]> wrote:

I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. 

 

On Tue, Oct 2, 2018 at 1:37 PM akurbanov <[hidden email]> wrote:

Hello Roman,

Correct me if I'm mistaken, you are talking about SQL queries. That was
fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary
ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be
delivered in 2.7 release.

Regards,
Anton



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/