AS we already know, after the cluster restart, every GET call gets data from
DISK for the first time and loads into RAM and subsequent calls data will
read from RAM only..
First time GET calls are 10 times slower than read from RAM, which we wanted
to avoid by pre-loading the entire data into RAM after the cluster restart.
So here am exploring efficient ways to read entire data once so that it will
pre-load the data into RAM, so GET calls from client will be much faster.
Running ScanQuery on all the partitions of the cache would be good way to
read data very fast in very less time ? OR any other better ways of
achieving the same
However, this seems to be quite slow. Taking more than 3 hours to read one
cache which has 400 M records. We have 30 such caches to load data, so not
fining this so efficient.
Can we improve this, we do have very powerful machines with 128 CPU, 2 TB
RAM, HDD, our CPU utilization is also not so high when we are preloading the
Changing thread pool size will have any impact this read ???