AffinityKey Configuration in order to achieve multiple joins across caches

classic Classic list List threaded Threaded
4 messages Options
StartCoding StartCoding
Reply | Threaded
Open this post in threaded view
|

AffinityKey Configuration in order to achieve multiple joins across caches

Hi Team,


Below are my java templates whose object I want to store in the Ignite
Caches

Class A
{

  field1
  field2
  field3


}

Class B
{

  field1
  field4
  field5


}

Class C

{

  field5
  field6
  field7


}





I wanted to colocate data in such a way that all instances of class A and
Class B should stay together in the partition node where ClassAObj.field1 =
ClassBObj.field1
Silimarly I want to make sure that all instances of class B and Class C
should stay together in the node where ClassBObj.field5 = ClassCObj.field5.
How Can we achive using cache Key configuration?


I tried like this

Defined Key classes

Class AKey

{

  field1
 


}
Class BKey

{

  field1
  field5
 


}

Class CKey

{

  field5
 


}


used below property  in the IgniteConfiguration.



<property name="cacheKeyConfiguration">
            <list>
                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
                    <property name="typeName"
value="com.report.model.AKey"/>
                    <property name="affinityKeyFieldName" value="field1" />
                </bean>
                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
                    <property name="typeName"
value="com.report.model.BKey"/>
                    <property name="affinityKeyFieldName" value="field1" />
                </bean>
               

                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
                    <property name="typeName"
value="com.report.model.BKey"/>
                    <property name="affinityKeyFieldName" value="field5" />
                </bean>
                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
                    <property name="typeName"
value="com.report.model.CKey"/>
                    <property name="affinityKeyFieldName" value="field5" />
                </bean>


            </list>
 </property>

Is this the way I should map or there is some other way to do it. Any help
to this will be appreciated.

Thanks
Saji



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Mikhail Mikhail
Reply | Threaded
Open this post in threaded view
|

Re: AffinityKey Configuration in order to achieve multiple joins across caches

Hi,

You can have only one affinity key in a class, so in your case, you need to
choose the smallest table and made it replicated to avoid distributed joins.
Another option is to use denormalization of you data, for example, to store
Class B Class C in one class as one row.

Thanks,
Mike.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
StartCoding StartCoding
Reply | Threaded
Open this post in threaded view
|

Re: AffinityKey Configuration in order to achieve multiple joins across caches

Hi Mike,

Thanks for your quick response.

I am afraid denormalizing will work for me because I have just given a
simple example. There are 16 tables which in that case needs to be joined
into single entity. Replication was an approach I thought about and we have
already considered smaller tables for that. But there are 7 huge tables
which consists of 6M+ records and will degrade the performance using
replicated caches.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: AffinityKey Configuration in order to achieve multiple joins across caches


On Thu, Mar 15, 2018 at 9:51 PM, StartCoding <[hidden email]> wrote:
Hi Mike,

Thanks for your quick response.

I am afraid denormalizing will work for me because I have just given a
simple example. There are 16 tables which in that case needs to be joined
into single entity. Replication was an approach I thought about and we have
already considered smaller tables for that. But there are 7 huge tables
which consists of 6M+ records and will degrade the performance using
replicated caches.

Saji, in this case, when you cannot select one affinity key for all your tables, you can try the following:
  1. Think about having multiple caches for the same data with different affinity keys.
  2. Instead of doing JOINs all the time, for cases where you could not use the affinity key for collocation, you can try using distribute compute API and process data locally within caches on individual nodes. You can then aggregate your results on the client side manually.