Slow initial Cache Insertion

classic Classic list List threaded Threaded
7 messages Options
mrinalkamboj mrinalkamboj
Reply | Threaded
Open this post in threaded view
|

Slow initial Cache Insertion

Following is my Cache configuration, aim is to insert 1 million records, I am setting the initial size to avoid delay due to resizing, as suggested in the Apache ignite documentation, but still the first insertion takes around 14 - 15 seconds, when second time onward it is reduced to 3 - 4 sec and then it remains consistent. What can I do to avoid the first time delay.

<cacheConfiguration>
      <cacheConfiguration name="OrderCache" cacheMode="Replicated" startSize="104857600">
        <queryEntities>
          <queryEntity keyType="System.Int32" valueType="OrderEntity,ApacheIgnite">
          </queryEntity>
        </queryEntities>
      </cacheConfiguration>
    </cacheConfiguration>
Pavel Tupitsyn Pavel Tupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

Hi,

Initial slowdown can be caused by JIT and JVM warmup, this is normal.
What are your JvmInitialMemory settings?

Pavel.

On Wed, Jun 22, 2016 at 10:18 AM, mrinalkamboj <[hidden email]> wrote:
Following is my Cache configuration, aim is to insert 1 million records, I am
setting the initial size to avoid delay due to resizing, as suggested in the
Apache ignite documentation, but still the first insertion takes around 14 -
15 seconds, when second time onward it is reduced to 3 - 4 sec and then it
remains consistent. What can I do to avoid the first time delay.

<cacheConfiguration>
      <cacheConfiguration name="OrderCache" cacheMode="Replicated"
startSize="104857600">
        <queryEntities>
          <queryEntity keyType="System.Int32"
valueType="OrderEntity,ApacheIgnite">
          </queryEntity>
        </queryEntities>
      </cacheConfiguration>
    </cacheConfiguration>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

mrinalkamboj mrinalkamboj
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

Hello Pavel,

No explicit setting for JVM, it would be default

thanks,

Mrinal
Pavel Tupitsyn Pavel Tupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

You can try setting initial memory to a higher value.

Also, what do you mean by first and second time? When you load the data second time, is the cache empty or not? 
Can you attach the code which does both first and second insert?

Pavel.

On Wed, Jun 22, 2016 at 2:55 PM, mrinalkamboj <[hidden email]> wrote:
Hello Pavel,

No explicit setting for JVM, it would be default

thanks,

Mrinal



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781p5795.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

mrinalkamboj mrinalkamboj
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

This post was updated on .
Second time means repeating the same insertion operation as the first time, Cache is not empty, it contains the data of the first time, so it would replace, I don't call Cache.Clear() explicitly.

Also I am trying to bump initial heap size - "java -Xms1024m"
but its not working, any idea the correct command
Pavel Tupitsyn Pavel Tupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

If you use DataStreamer, and AllowOverwrite if false (default), then the streamer would not actually update cache entries.
That's why it runs faster on the second run.

Pavel.

On Wed, Jun 22, 2016 at 3:25 PM, mrinalkamboj <[hidden email]> wrote:
Second time means repeating the same insertion operation as the first time,
Cache is not empty, it contains the data of the first time, so it would
replace, I don't call Cache.Clear() explicitly



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781p5798.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

mrinalkamboj mrinalkamboj
Reply | Threaded
Open this post in threaded view
|

Re: Slow initial Cache Insertion

Alright then my testing was not completely fair, I need to rerun post setting the AllowOverwrite and this would make Ignite's performance much more realistic when compared with parallel products.
Thanks for the insight.