Slow initial Cache Insertion

[mrinalkamboj]

Following is my Cache configuration, aim is to insert 1 million records, I am setting the initial size to avoid delay due to resizing, as suggested in the Apache ignite documentation, but still the first insertion takes around 14 - 15 seconds, when second time onward it is reduced to 3 - 4 sec and then it remains consistent. What can I do to avoid the first time delay.

<cacheConfiguration>
      <cacheConfiguration name="OrderCache" cacheMode="Replicated" startSize="104857600">
        <queryEntities>
          <queryEntity keyType="System.Int32" valueType="OrderEntity,ApacheIgnite">
          </queryEntity>
        </queryEntities>
      </cacheConfiguration>
    </cacheConfiguration>

[Pavel Tupitsyn]

Hi,

Initial slowdown can be caused by JIT and JVM warmup, this is normal.

What are your JvmInitialMemory settings?

Pavel.

On Wed, Jun 22, 2016 at 10:18 AM, mrinalkamboj <[hidden email]> wrote:

Following is my Cache configuration, aim is to insert 1 million records, I am
setting the initial size to avoid delay due to resizing, as suggested in the
Apache ignite documentation, but still the first insertion takes around 14 -
15 seconds, when second time onward it is reduced to 3 - 4 sec and then it
remains consistent. What can I do to avoid the first time delay.

<cacheConfiguration>
      <cacheConfiguration name="OrderCache" cacheMode="Replicated"
startSize="104857600">
        <queryEntities>
          <queryEntity keyType="System.Int32"
valueType="OrderEntity,ApacheIgnite">
          </queryEntity>
        </queryEntities>
      </cacheConfiguration>
    </cacheConfiguration>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

[mrinalkamboj]

Hello Pavel,

No explicit setting for JVM, it would be default

thanks,

Mrinal

[Pavel Tupitsyn]

You can try setting initial memory to a higher value.

Also, what do you mean by first and second time? When you load the data second time, is the cache empty or not? 

Can you attach the code which does both first and second insert?

Pavel.

On Wed, Jun 22, 2016 at 2:55 PM, mrinalkamboj <[hidden email]> wrote:

Hello Pavel,

No explicit setting for JVM, it would be default

thanks,

Mrinal



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781p5795.html

Sent from the Apache Ignite Users mailing list archive at Nabble.com.

[mrinalkamboj]

Second time means repeating the same insertion operation as the first time, Cache is not empty, it contains the data of the first time, so it would replace, I don't call Cache.Clear() explicitly.

Also I am trying to bump initial heap size - "java -Xms1024m"
but its not working, any idea the correct command

[Pavel Tupitsyn]

If you use DataStreamer, and AllowOverwrite if false (default), then the streamer would not actually update cache entries.

That's why it runs faster on the second run.

Pavel.

On Wed, Jun 22, 2016 at 3:25 PM, mrinalkamboj <[hidden email]> wrote:

Second time means repeating the same insertion operation as the first time,
Cache is not empty, it contains the data of the first time, so it would
replace, I don't call Cache.Clear() explicitly



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Slow-initial-Cache-Insertion-tp5781p5798.html

Sent from the Apache Ignite Users mailing list archive at Nabble.com.

[mrinalkamboj]

Alright then my testing was not completely fair, I need to rerun post setting the AllowOverwrite and this would make Ignite's performance much more realistic when compared with parallel products.
Thanks for the insight.