How to update QueryEntities property of CacheConfiguration for existing cache in .net

classic Classic list List threaded Threaded
13 messages Options
siva siva
Reply | Threaded
Open this post in threaded view
|

How to update QueryEntities property of CacheConfiguration for existing cache in .net

This post was updated on .
Hi,
we have implemented .net core server with 3rd party and native persistence
enabled & .net core thick client to create caches and thinclient to perform
crud/sql operations to the grid from multiple applications

we are creating single cache for multiple models classes  and working
fine.But problem is if we want to add new model class which should be
queriable as old model classes to the existing cache since we are
registering QueryEntities at the time of create cache.

so, how to update QueryEntities property of CacheConfiguration with New
QueryEntity  Model class for existing/already created cache?



Created Cache by using bellow obejct:

   
 var queryList = _queryEntities.CreateQueryEntitiesListAsync();//contain list of QueryEntity classes for sql query Let say intial 2 QueryEntity classes
      cacheCfg = new CacheConfiguration
                {
                    Name = "CacheName",
                    CacheStoreFactory = new CacheStoreFactory(connectionString),
                    KeepBinaryInStore = false,  // Cache store works with deserialized data.
                    ReadThrough = true,
                    WriteThrough = true,
                    WriteBehindEnabled = true,
                    QueryEntities = queryList,   
                    CacheMode = CacheMode.Partitioned,
                    DataRegionName = "IgniteDataRegion",
                    EvictionPolicy = new LruEvictionPolicy
                    {
                        MaxSize = 100000
                    }
                };


I have tried like

     
 var queryList = _queryEntities.CreateQueryEntitiesListAsync();//contain list of QueryEntity classes for sql query Let say intial 2 model classes and added one more QueryEntity class (Updated query entity list)

     var cache = Ignite.GetCache<string, Object>(cacheName);
	 var cacheConfig = cache.GetConfiguration();
	 cacheConfig.QueryEntities = queryList;
	 var newCacheConfig = new CacheConfiguration(cacheConfig);
     Ignite.AddCacheConfiguration(newCacheConfig);
	 
         But still not effected ,QueryEntities property still list of two
QueryEntity classes only



Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Hi siva, modifying QueryEntity configuration of existing cache is not possible.
Also, having multiple model classes per cache is not recommended.
The recommended way is to create a new cache for every new model/entity, and have one QueryEntity per cache.
Note that you can achieve all the same queries with multiple caches, like joins across many entities and so on.

You can manipulate SQL tables for existing caches with DML, though, which is almost what you ask for:

Thanks,
Pavel 

On Mon, Sep 23, 2019 at 6:27 PM siva <[hidden email]> wrote:
Hi,
we have implemented .net core server with 3rd party and native persistence
enabled & .net core thick client to create caches and thinclient to perform
crud/sql operations to the grid from multiple applications

we are creating single cache for multiple models classes  and working
fine.But problem is if we want to add new model class which should be
queriable as old model classes to the existing cache since we are
registering QueryEntities at the time of create cache.

*so, how to update QueryEntities property of CacheConfiguration with New
QueryEntity  Model class for existing/already created cache?*


Created Cache by using bellow obejct:



I have tried like



         But still not effected ,QueryEntities property still list of two
QueryEntity classes only



Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

This post was updated on .
Thanks Pavel  for the reply

our application is  multi company application ,so each company instance have
more than 100 tables.so that we are creating single cache for each company
instance containing lot  entity classes as well as it thirdparty enable with
native persistence.


You can manipulate SQL tables for existing caches with DML,
yes,we are using DDL for if any new field added/removed to the entity/model
class by using sqlline.

For Updating New Model/Entity class is  visible to sql query only problem to
us now,
i have tried like this also
,newly entity class record inserted into ignite cache(name company1),working
for crud operations with key,for visible to sql api , tried to create table
with same entity/model class name  with specified cachename as company1
which is already created.but exception occured  as table already exists


CREATE TABLE CARSALES (
  ID INT(11),
  Name CHAR(35),
  CountryCode CHAR(3),
  District CHAR(20),
  Population INT(11),
  PRIMARY KEY (ID, CountryCode)
) WITH "backups=1, CACHE_NAME=company1"; //company cache already created

java.sql.SQLException: Table already exists: CARSALES


The recommended way is to create a new cache for every new model/entity, and have one QueryEntity per cache.

Let say we have 30 companies
30*200 tables = 6k caches  instead of 30 caches what is the performance impact(while restarting  and for curd operations)


--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Can you please attach a reproducer for that exception?

On Tue, Sep 24, 2019 at 8:46 AM siva <[hidden email]> wrote:
Thanks *Pavel *for the reply

our application is  multi company application ,so each company instance have
more than 100 tables.so that we are creating single cache for each company
instance containing lot  entity classes as well as it thirdparty enable with
native persistence.


*You can manipulate SQL tables for existing caches with DML,
*yes,we are using DML for if any new field added/removed to the entity/model
class by using sqlline.

For Updating New Model/Entity class is  visible to sql query only problem to
us now,
i have tried like this also
,newly entity class record inserted into ignite cache(name company1),working
for crud operations with key,for visible to sql api , tried to create table
with same entity/model class name  with specified cachename as company1
which is already created.but exception occured  as table already exists

java.sql.SQLException: Table already exists: CARSALES



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Hi Pavel,

I have attached github link for the reproducer of above exception


https://github.com/cvakarna/ignitereproducer
<https://github.com/cvakarna/ignitereproducer>  





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Ok, so Lecturer table already exists when you try to create it again.
Just run DROP TABLE IF EXISTS Lecturer before trying to create it.

On Tue, Sep 24, 2019 at 12:26 PM siva <[hidden email]> wrote:
Hi Pavel,

I have attached github link for the reproducer of above exception


https://github.com/cvakarna/ignitereproducer
<https://github.com/cvakarna/ignitereproducer





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Hi Pavel,
'

actually Lecturer table not exists since the cache created with Student
query entity only ,if we try to create another table (lets say HODepartment)
with already created cache also same exception is throwing .




Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Yes, you are correct, now I see the problem too. Let me investigate it a bit and get back to you.

On Wed, Sep 25, 2019 at 10:37 AM siva <[hidden email]> wrote:
Hi Pavel,
'

actually Lecturer table not exists since the cache created with Student
query entity only ,if we try to create another table (lets say HODepartment)
with already created cache also same exception is throwing .




Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Turns out we can't create multiple tables per cache with SQL DDL, this is a design limitation.
The error message is misleading, I'll file a ticket to fix it.

So again, I would advise to use one table per cache approach.
To reduce the overhead of creating 1000s of caches, you can leverage Cache Groups feature:

So instead of one cache per company, use one cache group per company.

Does this work for you?
 

On Wed, Sep 25, 2019 at 10:42 AM Pavel Tupitsyn <[hidden email]> wrote:
Yes, you are correct, now I see the problem too. Let me investigate it a bit and get back to you.

On Wed, Sep 25, 2019 at 10:37 AM siva <[hidden email]> wrote:
Hi Pavel,
'

actually Lecturer table not exists since the cache created with Student
query entity only ,if we try to create another table (lets say HODepartment)
with already created cache also same exception is throwing .




Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Thanks Pavel for helping,Ok i do go through  documentation for cache groups
and will check  once







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

This post was updated on .
Hi Pavel,

I have some queries related to cache groups ,if you help it will be useful
for us.


instead of one cache per company, use one cache group per company.

1.What is the performance impact
 =>Is there any number of cache group limitations?
    here cachegroups proportional to tenants

2.What If we have only cluster of 3 nodes(each node 16cpu and 56GB RAM with
ssd) handling  20 cachegroups with each cache group contains 120 caches(i.e
2.4k caches aprox)


That said, they might impact the performance of read operations and indexes
lookups

is it reasonable or drastically decrease? Since Our application is for both
read and create/update intensive (before create or update we should
query/read data from different  tables)




Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

Too many things come into play here, the only way to know is to test both approaches on your use case.
By the way, do you have the same table set for every tenant?
It may be beneficial to group caches not by tenant, but by data size and type.



On Thu, Sep 26, 2019 at 12:56 PM siva <[hidden email]> wrote:
Hi Pavel,

I have some queries related to cache groups ,if you help it will be useful
for us.


*instead of one cache per company, use one cache group per company.
*
1.What is the performance impact
 =>Is there any number of cache group limitations?
    here cachegroups proportional to tenants

2.What If we have only cluster of 3 nodes(each node 16cpu and 56GB RAM with
ssd) handling  20 cachegroups with each cache group contains 120 caches(i.e
2.4k caches aprox)


*That said, they might impact the performance of read operations and indexes
lookups
*is it reasonable or drastically decrease? Since Our application is for both
read and create/update intensive (before create or update we should
query/read data from different  tables)




Thanks
siva



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
siva siva
Reply | Threaded
Open this post in threaded view
|

Re: How to update QueryEntities property of CacheConfiguration for existing/already created cache

This post was updated on .
do you have the same table set for every tenant?
Yes ,we have  same tables set for each tenant







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/