Tuning client performance.

classic Classic list List threaded Threaded
7 messages Options
javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Tuning client performance.

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Tuning client performance.

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?

 

Alex Shapkin
javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: Tuning client performance.

Sorry the regular full mesh client. Maybe some threadPoolSizes

On Fri, 30 Aug 2019 at 11:23, Alexandr Shapkin <[hidden email]> wrote:

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?

 

javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: Tuning client performance.

Actually some more details.

I have a partitioned cache with about 4Million records over 3 nodes. When I do a get from the REST API I can hit about 4000 Get per second.

From my application I can only achieve about 200 Get per second. My application is single threaded consumer consuming from single partition topic and then does a GET per kafka record. I know as a single consumer thread the application without cache can handle give or take 2000. So if figure with a bit of tuning I can get up to 1000 Gets.

Also using async GET cache.

On Fri, 30 Aug 2019 at 12:00, John Smith <[hidden email]> wrote:
Sorry the regular full mesh client. Maybe some threadPoolSizes

On Fri, 30 Aug 2019 at 11:23, Alexandr Shapkin <[hidden email]> wrote:

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?

 

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Tuning client performance.

Hello!

Can you switch to getAll()? Alterntively, try to use more than 1 thread.

Regards,
--
Ilya Kasnacheev


пт, 30 авг. 2019 г. в 19:11, John Smith <[hidden email]>:
Actually some more details.

I have a partitioned cache with about 4Million records over 3 nodes. When I do a get from the REST API I can hit about 4000 Get per second.

From my application I can only achieve about 200 Get per second. My application is single threaded consumer consuming from single partition topic and then does a GET per kafka record. I know as a single consumer thread the application without cache can handle give or take 2000. So if figure with a bit of tuning I can get up to 1000 Gets.

Also using async GET cache.

On Fri, 30 Aug 2019 at 12:00, John Smith <[hidden email]> wrote:
Sorry the regular full mesh client. Maybe some threadPoolSizes

On Fri, 30 Aug 2019 at 11:23, Alexandr Shapkin <[hidden email]> wrote:

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?

 

javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: Tuning client performance.

Actually, I looked closer at my code. Cannot use getAll() and the Queue is single partition, so can't use multiple threads, the application is per event and we have tested it up to 3K events per second not including the cache.

But also the application a fully async using vertx.io. Looking closer at the code cache.get() is a blocking call which vertx.io doesn't like to much.

So I will switch to IgniteFuture. I'm pretty confident that will help allot.



On Mon, 2 Sep 2019 at 07:13, Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Can you switch to getAll()? Alterntively, try to use more than 1 thread.

Regards,
--
Ilya Kasnacheev


пт, 30 авг. 2019 г. в 19:11, John Smith <[hidden email]>:
Actually some more details.

I have a partitioned cache with about 4Million records over 3 nodes. When I do a get from the REST API I can hit about 4000 Get per second.

From my application I can only achieve about 200 Get per second. My application is single threaded consumer consuming from single partition topic and then does a GET per kafka record. I know as a single consumer thread the application without cache can handle give or take 2000. So if figure with a bit of tuning I can get up to 1000 Gets.

Also using async GET cache.

On Fri, 30 Aug 2019 at 12:00, John Smith <[hidden email]> wrote:
Sorry the regular full mesh client. Maybe some threadPoolSizes

On Fri, 30 Aug 2019 at 11:23, Alexandr Shapkin <[hidden email]> wrote:

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?

 

javadevmtl javadevmtl
Reply | Threaded
Open this post in threaded view
|

Re: Tuning client performance.

Yeah, initial tests show improvements just by switching to async get.

On Tue, 3 Sep 2019 at 11:06, John Smith <[hidden email]> wrote:
Actually, I looked closer at my code. Cannot use getAll() and the Queue is single partition, so can't use multiple threads, the application is per event and we have tested it up to 3K events per second not including the cache.

But also the application a fully async using vertx.io. Looking closer at the code cache.get() is a blocking call which vertx.io doesn't like to much.

So I will switch to IgniteFuture. I'm pretty confident that will help allot.



On Mon, 2 Sep 2019 at 07:13, Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Can you switch to getAll()? Alterntively, try to use more than 1 thread.

Regards,
--
Ilya Kasnacheev


пт, 30 авг. 2019 г. в 19:11, John Smith <[hidden email]>:
Actually some more details.

I have a partitioned cache with about 4Million records over 3 nodes. When I do a get from the REST API I can hit about 4000 Get per second.

From my application I can only achieve about 200 Get per second. My application is single threaded consumer consuming from single partition topic and then does a GET per kafka record. I know as a single consumer thread the application without cache can handle give or take 2000. So if figure with a bit of tuning I can get up to 1000 Gets.

Also using async GET cache.

On Fri, 30 Aug 2019 at 12:00, John Smith <[hidden email]> wrote:
Sorry the regular full mesh client. Maybe some threadPoolSizes

On Fri, 30 Aug 2019 at 11:23, Alexandr Shapkin <[hidden email]> wrote:

Hello,

 

Not a thin-client tuning in general, but you can check a serialization settings in order to make sure you do not use a default one.

 

From: [hidden email]
Sent: Friday, August 30, 2019 6:04 PM
To: [hidden email]
Subject: Tuning client performance.

 

Hi, is there any specific client settings we can set to tune the client performance, maybe some thread pools or any stuff like that?