Optimise replicated caches in multinodes

classic Classic list List threaded Threaded
2 messages Options
steve.hostettler steve.hostettler
Reply | Threaded
Open this post in threaded view
|

Optimise replicated caches in multinodes

Hello,

in order to provide "easy" multinode, we have set all of our caches to
REPLICATED and FULL_ASYNC. We get an improvment (Around 20%) with a second
node, but it is very clear that somehow the grid is waiting (95% cpu
utilization with 1 node, 75% with 2 nodes).

I understand that I should rather go for partitioned caches, but I am trying
here to provide an "easy" first step towards multinode by avoiding rewriting
the logic.

Looking at the statistics I get the following

GridNearAtomicSingleUpdateRequest=766,197
GridDhtAtomicDeferredUpdateResponse=8,634
GridDhtAtomicSingleUpdateRequest=1,681,590
GridNearAtomicUpdateResponse=836,598

I assume that these are linked to the put in the replicated caches that are
asynchronously sent to the other node.

I would like to understand a bit better these numbers and the difference
between them.
Another question: is there a way to increase the batching of messages on the
grid. Especially when it comes to replicated caches?

Thanks a lot for your help



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
akorensh akorensh
Reply | Threaded
Open this post in threaded view
|

Re: Optimise replicated caches in multinodes

Hi,
   You are correct, these are Ignite request/response messages generated
when you are putting data into
   a cache. The numbers roughly correspond to different multiples of the
number of put/putAll you are
   performing. These numbers will depend on how large the objects you are
putting into the cache are, the  
   network throughput, cpu config, and other hardware/OS  parameters.
 
   It is best to use system provided metrics to measure performance.  
  https://apacheignite.readme.io/docs/new-metrics
  https://apacheignite.readme.io/docs/cache-metrics
  https://apacheignite.readme.io/docs/memory-metrics
  More information is provided in system views:
https://apacheignite.readme.io/docs/system-views


  To measure performance/fine-tune Ignite to your use case
  determine the # of caches, size of objects, number of puts/gets you are
trying to perform in a given
  interval.  Set the cache(s) to different parameters to see   what
throughput you achieve. Try using
  putAll/datastreamers when possible.  Use the information in the above
provided links to measure each
  scenario. https://apacheignite.readme.io/docs/cache-modes
  Use the above provided links to measure each scenario.
  Use data in this link to prepare your use case for prod:
https://apacheignite.readme.io/docs/preparing-for-production


  Data Streamers provide the ability to batch messages. See
https://apacheignite.readme.io/docs/data-streamers

Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/