What are minimal thread pools for grid clients?

classic Classic list List threaded Threaded
4 messages Options
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

What are minimal thread pools for grid clients?

Hi,

I am on 2.7.5 version and using the same Spring configuration for both
servers and thick clients.  I do not want to use thin client as I want my
client to be part of the grid.  What I have come to learn is that even if
Ignition.setClientMode(true); or conf.setClientMode(true); some thread pools
are created that really shouldn't be.  For example, if a cache is configured
for write-behind, with <property name="writeBehindFlushThreadCount"
value="32" />, 32 flusher threads will be created on the client, where the
flushing will only ever happen on the server if I am not mistaken.  So
thread pool creation seems to disregard whether or not I am a client.

In order to trim my client threads, I now load the Spring configuration, and
do the following below.  I have also noticed that if you don't give at least
one thread to the pool, for example IFGS, Ignite will not start and
complains that cnt must be > 0.  Several thread pools in the configuration
aren't documented here (https://apacheignite.readme.io/docs/thread-pools),
so what does:

1) UtilityCachePoolSize do and do I care about it if I am a client?
2) ManagementPoolSize do and do I care about it if I am a client?
3) QueryTheadPool (documented) but do I need it on a client?

It would be helpful if documentation could be added for the missing pools
above and also what the minimal client configuration should be as to
minimize client memory footprint and resource usage.

Thanks!
Rick

                    conf.setClientMode(true);
                    // For clients disable as many threads as we can.
                    Boolean disableConnectorConfig =
Boolean.parseBoolean(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_DISABLE_CONNECTOR_CONFIG,
CacheConstants.DEFAULT_IGNITE_CLIENT_DISABLE_CONNECTOR_CONFIG));
                    if(disableConnectorConfig) {
                        conf.setConnectorConfiguration(null);
                    }
                    // Public Thread pool used for compute grid.
                   
conf.setPublicThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_PUBLIC_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_PUBLIC_TP_SIZE)));
                    // Minimum must be one.
                   
conf.setDataStreamerThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_DATA_STREAMER_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_DATA_STREAMER_TP_SIZE)));
                    // Minimum must be one.
                   
conf.setIgfsThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_IGFS_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_IGFS_TP_SIZE)));
                   
conf.setServiceThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_SERVICE_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_SERVICE_TP_SIZE)));
                    conf.setDataStorageConfiguration(null);
                    List<CacheConfiguration> configs =
Arrays.asList(conf.getCacheConfiguration());
                    for(CacheConfiguration config : configs) {
                        if(config.isWriteBehindEnabled()) {
                            config.setWriteBehindFlushThreadCount(1);
                        }
                    }



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: What are minimal thread pools for grid clients?

Hello!

I recommend keeping all pools at something like 4.

Ignite clients can run a lot of operations (they can do compute, they use cache store when load is transactional, they run visor tasks, etc) so it's better to not zero out all pools outright.

Regards,
--
Ilya Kasnacheev


вт, 24 сент. 2019 г. в 11:38, rick_tem <[hidden email]>:
Hi,

I am on 2.7.5 version and using the same Spring configuration for both
servers and thick clients.  I do not want to use thin client as I want my
client to be part of the grid.  What I have come to learn is that even if
Ignition.setClientMode(true); or conf.setClientMode(true); some thread pools
are created that really shouldn't be.  For example, if a cache is configured
for write-behind, with <property name="writeBehindFlushThreadCount"
value="32" />, 32 flusher threads will be created on the client, where the
flushing will only ever happen on the server if I am not mistaken.  So
thread pool creation seems to disregard whether or not I am a client.

In order to trim my client threads, I now load the Spring configuration, and
do the following below.  I have also noticed that if you don't give at least
one thread to the pool, for example IFGS, Ignite will not start and
complains that cnt must be > 0.  Several thread pools in the configuration
aren't documented here (https://apacheignite.readme.io/docs/thread-pools),
so what does:

1) UtilityCachePoolSize do and do I care about it if I am a client?
2) ManagementPoolSize do and do I care about it if I am a client?
3) QueryTheadPool (documented) but do I need it on a client?

It would be helpful if documentation could be added for the missing pools
above and also what the minimal client configuration should be as to
minimize client memory footprint and resource usage.

Thanks!
Rick

                    conf.setClientMode(true);
                    // For clients disable as many threads as we can.
                    Boolean disableConnectorConfig =
Boolean.parseBoolean(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_DISABLE_CONNECTOR_CONFIG,
CacheConstants.DEFAULT_IGNITE_CLIENT_DISABLE_CONNECTOR_CONFIG));
                    if(disableConnectorConfig) {
                        conf.setConnectorConfiguration(null);
                    }
                    // Public Thread pool used for compute grid.

conf.setPublicThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_PUBLIC_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_PUBLIC_TP_SIZE)));
                    // Minimum must be one.

conf.setDataStreamerThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_DATA_STREAMER_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_DATA_STREAMER_TP_SIZE)));
                    // Minimum must be one.

conf.setIgfsThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_IGFS_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_IGFS_TP_SIZE)));

conf.setServiceThreadPoolSize(Integer.parseInt(TAFJCacheFactory.getCacheProperties().getProperty(CacheConstants.KEY_PROP_IGNITE_CLIENT_SERVICE_TP_SIZE,
CacheConstants.DEFAULT_IGNITE_CLIENT_SERVICE_TP_SIZE)));
                    conf.setDataStorageConfiguration(null);
                    List<CacheConfiguration> configs =
Arrays.asList(conf.getCacheConfiguration());
                    for(CacheConfiguration config : configs) {
                        if(config.isWriteBehindEnabled()) {
                            config.setWriteBehindFlushThreadCount(1);
                        }
                    }



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
rick_tem rick_tem
Reply | Threaded
Open this post in threaded view
|

Re: What are minimal thread pools for grid clients?

Thanks for your reply.  Can you answer the 2nd part of the question?

1) UtilityCachePoolSize do and do I care about it if I am a client?
2) ManagementPoolSize do and do I care about it if I am a client?
3) QueryTheadPool (documented) but do I need it on a client?

Would be helpful if these could be documented on the threadpools page
previously mentioned.

Thanks,
Rick



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: What are minimal thread pools for grid clients?

Hello!

1) Yes.
2) Yes.
3) I'm not sure.

Regards,
--
Ilya Kasnacheev


ср, 2 окт. 2019 г. в 18:04, rick_tem <[hidden email]>:
Thanks for your reply.  Can you answer the 2nd part of the question?

1) UtilityCachePoolSize do and do I care about it if I am a client?
2) ManagementPoolSize do and do I care about it if I am a client?
3) QueryTheadPool (documented) but do I need it on a client?

Would be helpful if these could be documented on the threadpools page
previously mentioned.

Thanks,
Rick



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/