How to retrieve data from Collocated Cache with Simple Key

classic Classic list List threaded Threaded
8 messages Options
Kamal Kamal
Reply | Threaded
Open this post in threaded view
|

How to retrieve data from Collocated Cache with Simple Key

Hi,

    I've gone through the affinity collocation[1] example to understand how data gets collocated across caches. In my example, I found that I'm not able to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey<String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
Company.java
Person.java
Vladislav Pyatkov Vladislav Pyatkov
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

Hello Jamal,

I think so, It not possible, because data store in particular partition, which determine only by affinityKey.

More useful case, when you are not know fullKey, use ScanQuery(1) or use cache iterator.

(1): https://apacheignite.readme.io/docs/cache-queries

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Vladislav Pyatkov Vladislav Pyatkov
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

In reply to this post by Kamal

I am sorry for mistake Kamal...

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Kamal Kamal
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

Thanks for your response Vladislav.

Both ScanQuery and Iterator traverses the whole cache to find the value.
It may not be suitable in my environment as there can be huge number of
hits.

I understand that for fast retrieval `key-to-partition` mapping is done.
But, In AffinityKey documentation, it's specified that hashcode and equals
methods are implemented based on simple key.

* <p>
 * Note that the {@link #equals(Object)} and {@link #hashCode()} methods
 * delegate directly to the wrapped cache key provided by {@link #key()}
 * method.
 * <p>

On Mon, Jun 6, 2016 at 10:44 PM, Vladislav Pyatkov <[hidden email]> wrote:

I am sorry for mistake Kamal...

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

Hi Kamal,

There is no need to use any workaround like ScanQueries or iterators. You just need to use a valid key to retrieve the data.
The valid key in your example is new AffinityKey<>(key, affray). It means that every time you need to put or get a Person from the cache you need to use this kind of key where “key” and “affray” will vary.

Also it’s not required to use AffinityKey instance all the time. You are free to create your own implementation of a key

class PersonKey {

private int id;

@AffinityKeyMaped 
private int orgId;

//hash code and equals implementations are below
}

The ticket is created for NPE you got when were using a wrong key
Denis

On Jun 7, 2016, at 7:40 AM, Kamal C <[hidden email]> wrote:

Thanks for your response Vladislav.

Both ScanQuery and Iterator traverses the whole cache to find the value.
It may not be suitable in my environment as there can be huge number of
hits.

I understand that for fast retrieval `key-to-partition` mapping is done.
But, In AffinityKey documentation, it's specified that hashcode and equals
methods are implemented based on simple key.

* <p>
 * Note that the {@link #equals(Object)} and {@link #hashCode()} methods
 * delegate directly to the wrapped cache key provided by {@link #key()}
 * method.
 * <p>

On Mon, Jun 6, 2016 at 10:44 PM, Vladislav Pyatkov <[hidden email]> wrote:

I am sorry for mistake Kamal...

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.


Kamal Kamal
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

In my app, I'm trying to replace the existing ehcache implementation to
Ignite caches.

Using Affinity Collocation, I've collocated the data in the caches which are
accessed together. In many places, I've only SIMPLE key to retrieve the
data from the caches. (e.g. Retrieval from Person cache with only name)

In `put` operation, we have full object to insert so construction of PersonKey /
 AffinityKey is not a problem. But, during `get` operation, we may have only
the simple key to retrieve data.

I understand that Ignite looks data only in the partition specified by the
AffinityFunction computed on AffinityKey. Can I force Ignite to look for data
on all the local cache partitions in the node ?



On Tue, Jun 7, 2016 at 10:38 AM, Denis Magda <[hidden email]> wrote:
Hi Kamal,

There is no need to use any workaround like ScanQueries or iterators. You just need to use a valid key to retrieve the data.
The valid key in your example is new AffinityKey<>(key, affray). It means that every time you need to put or get a Person from the cache you need to use this kind of key where “key” and “affray” will vary.

Also it’s not required to use AffinityKey instance all the time. You are free to create your own implementation of a key

class PersonKey {

private int id;

@AffinityKeyMaped 
private int orgId;

//hash code and equals implementations are below
}

The ticket is created for NPE you got when were using a wrong key
Denis

On Jun 7, 2016, at 7:40 AM, Kamal C <[hidden email]> wrote:

Thanks for your response Vladislav.

Both ScanQuery and Iterator traverses the whole cache to find the value.
It may not be suitable in my environment as there can be huge number of
hits.

I understand that for fast retrieval `key-to-partition` mapping is done.
But, In AffinityKey documentation, it's specified that hashcode and equals
methods are implemented based on simple key.

* <p>
 * Note that the {@link #equals(Object)} and {@link #hashCode()} methods
 * delegate directly to the wrapped cache key provided by {@link #key()}
 * method.
 * <p>

On Mon, Jun 6, 2016 at 10:44 PM, Vladislav Pyatkov <[hidden email]> wrote:

I am sorry for mistake Kamal...

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



vdpyatkov vdpyatkov
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

Hello Kamal,

I will repeat, but you can use the ScanQuery.

org.apache.ignite.cache.query.ScanQuery#setPartition

You have only to find a local partition:
Affinity affinity = ignite.affinity(cacheName);

List<Integer> partitions = new ArrayList<>(affinity.partitions());

for (int i = 0; i < affinity.partitions(); i++)
partitions .add(i);

Map<Integer, ClusterNode> partPerNodes = affinity.mapPartitionsToNodes(randmPartitions);

and take only the local partition by node ID

On Wed, Jun 8, 2016 at 2:36 PM, Kamal C <[hidden email]> wrote:
In my app, I'm trying to replace the existing ehcache implementation to
Ignite caches.

Using Affinity Collocation, I've collocated the data in the caches which are
accessed together. In many places, I've only SIMPLE key to retrieve the
data from the caches. (e.g. Retrieval from Person cache with only name)

In `put` operation, we have full object to insert so construction of PersonKey /
 AffinityKey is not a problem. But, during `get` operation, we may have only
the simple key to retrieve data.

I understand that Ignite looks data only in the partition specified by the
AffinityFunction computed on AffinityKey. Can I force Ignite to look for data
on all the local cache partitions in the node ?



On Tue, Jun 7, 2016 at 10:38 AM, Denis Magda <[hidden email]> wrote:
Hi Kamal,

There is no need to use any workaround like ScanQueries or iterators. You just need to use a valid key to retrieve the data.
The valid key in your example is new AffinityKey<>(key, affray). It means that every time you need to put or get a Person from the cache you need to use this kind of key where “key” and “affray” will vary.

Also it’s not required to use AffinityKey instance all the time. You are free to create your own implementation of a key

class PersonKey {

private int id;

@AffinityKeyMaped 
private int orgId;

//hash code and equals implementations are below
}

The ticket is created for NPE you got when were using a wrong key
Denis

On Jun 7, 2016, at 7:40 AM, Kamal C <[hidden email]> wrote:

Thanks for your response Vladislav.

Both ScanQuery and Iterator traverses the whole cache to find the value.
It may not be suitable in my environment as there can be huge number of
hits.

I understand that for fast retrieval `key-to-partition` mapping is done.
But, In AffinityKey documentation, it's specified that hashcode and equals
methods are implemented based on simple key.

* <p>
 * Note that the {@link #equals(Object)} and {@link #hashCode()} methods
 * delegate directly to the wrapped cache key provided by {@link #key()}
 * method.
 * <p>

On Mon, Jun 6, 2016 at 10:44 PM, Vladislav Pyatkov <[hidden email]> wrote:

I am sorry for mistake Kamal...

On Jun 6, 2016 3:34 PM, "Kamal" <[hidden email]> wrote:
Hi,

    I've gone through the affinity collocation[1] example to understand how
data gets collocated across caches. In my example, I found that I'm not able
to retrieve data from collocated cache with simple key.

I mean.

Cache<AffinityKey&lt;String>, Person> personCache = ..;
personCache.get(new AffinityKey<>(key, affKey)); // returns value
personCache.get(new AffinityKey<>(key)); // throws NPE

Exception in thread "main" java.lang.NullPointerException
        at
org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction.partition(RendezvousAffinityFunction.java:428)
        at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.partition(GridCacheAffinityManager.java:206)
        at
org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheKeyObject(GridCacheContext.java:1801)
        at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.get(GridDhtAtomicCache.java:339)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
        at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
        at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:907)
        at
my.apache.ignite.examples.collocation.CacheCollocationExample.main(CacheCollocationExample.java:69)

In some scenarios, I have to fetch data from cache by simple key.

[1]: https://apacheignite.readme.io/docs/affinity-collocation
CacheCollocationExample.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/CacheCollocationExample.java>
Company.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Company.java>
Person.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n5452/Person.java>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-retrieve-data-from-Collocated-Cache-with-Simple-Key-tp5452.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.






--
Vladislav Pyatkov
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: How to retrieve data from Collocated Cache with Simple Key

In reply to this post by Kamal
Hi,

Please see below

On Jun 8, 2016, at 2:36 PM, Kamal C <[hidden email]> wrote:

In `put` operation, we have full object to insert so construction of PersonKey /
 AffinityKey is not a problem. But, during `get` operation, we may have only 
the simple key to retrieve data.

Introduce a special cache that will contain mapping of “personId” to “orgId”.

During a ‘get’ operation you can find in constant time “orgId” using this cache and after that create a valid AffinityKey and get a Person in constant time using this valid key.

Denis