Performance Question

classic Classic list List threaded Threaded
5 messages Options
Simin Eftekhari Simin Eftekhari
Reply | Threaded
Open this post in threaded view
|

Performance Question

Hello Everyone,

I am new to apache ignite. I'd appreciate your help with the following question.

I modified the CacheApiExample, to insert 2,000,000 entries into the cache.


If I run the program as is, the insert all the entries takes less 10 seconds (which I think is still too long). If I start an additional ignite node (on the same vm), the inserts take over 3 minutes. This is a huge discrepancy. I'd have thought that an additional instance would actually speed up the inserts.

Can someone please explain what is going?

thanks

SK
Mikael Mikael
Reply | Threaded
Open this post in threaded view
|

Re: Performance Question

Hi!

Well, it depends on what you are doing I guess ;)

The example use put(), this is the slowest way of putting anything in
the cache compared to streamers for example, anyway, if you run your
code on one node in a big loop that does put 2M times it will be slower,
with two nodes it will have to send half of the entries to the other
node, so 1M entries are being sent over the network to the other node,
with one node you don't need to send anything over the network.

You say 10 sec is slow, not sure what you compare with, too me that
sounds like a good time, Ignite will by default put your objects off
heap so the objects have to be serialized  and moved off heap and there
is a lot of things going on behind the scenes that takes time in case
you compare with a HashMap or something.

Ignite likes to do things in parallel, having one thread doing all the
puts is the worst way to put a lot of entries in a cache, try to create
a streamer and run your example with two nodes on that and see what you
get, I am not sure what you intend to use it for ? you can feed Ignite a
huge amount of entries per second but trying to get all of them through
one single thread on one node is not going to do that.

How did you configure the cache ? number of backups ? persistence ?
partitioned/replicated ?

Mikael


Den 2019-10-20 kl. 19:02, skrev Simin Eftekhari:

> Hello Everyone,
>
> I am new to apache ignite. I'd appreciate your help with the following
> question.
>
> I modified the CacheApiExample, to insert 2,000,000 entries into the
> cache.
>
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheApiExample.java
>
> If I run the program as is, the insert all the entries takes less 10
> seconds (which I think is still too long). If I start an additional
> ignite node (on the same vm), the inserts take over 3 minutes. This is
> a huge discrepancy. I'd have thought that an additional instance would
> actually speed up the inserts.
>
> Can someone please explain what is going?
>
> thanks
>
> SK
Simin Eftekhari Simin Eftekhari
Reply | Threaded
Open this post in threaded view
|

Re: Performance Question

Hello Micheal,
Thanks for the reply.
I did not make any changes to the configuration (example-ignite.xml), in either scenario. There is no backup. 


Also, in both cases, the ignite nodes are running on the same vm, meaning, there should not be any network hops.
I am executing the same program, in one case as is, in another case, as is with an additional ignite node started in the background, using this script:


I am just doing some basic benchmarking using the existing ignite examples.

SK

On Sun, Oct 20, 2019 at 1:31 PM Mikael <[hidden email]> wrote:
Hi!

Well, it depends on what you are doing I guess ;)

The example use put(), this is the slowest way of putting anything in
the cache compared to streamers for example, anyway, if you run your
code on one node in a big loop that does put 2M times it will be slower,
with two nodes it will have to send half of the entries to the other
node, so 1M entries are being sent over the network to the other node,
with one node you don't need to send anything over the network.

You say 10 sec is slow, not sure what you compare with, too me that
sounds like a good time, Ignite will by default put your objects off
heap so the objects have to be serialized  and moved off heap and there
is a lot of things going on behind the scenes that takes time in case
you compare with a HashMap or something.

Ignite likes to do things in parallel, having one thread doing all the
puts is the worst way to put a lot of entries in a cache, try to create
a streamer and run your example with two nodes on that and see what you
get, I am not sure what you intend to use it for ? you can feed Ignite a
huge amount of entries per second but trying to get all of them through
one single thread on one node is not going to do that.

How did you configure the cache ? number of backups ? persistence ?
partitioned/replicated ?

Mikael


Den 2019-10-20 kl. 19:02, skrev Simin Eftekhari:
> Hello Everyone,
>
> I am new to apache ignite. I'd appreciate your help with the following
> question.
>
> I modified the CacheApiExample, to insert 2,000,000 entries into the
> cache.
>
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheApiExample.java
>
> If I run the program as is, the insert all the entries takes less 10
> seconds (which I think is still too long). If I start an additional
> ignite node (on the same vm), the inserts take over 3 minutes. This is
> a huge discrepancy. I'd have thought that an additional instance would
> actually speed up the inserts.
>
> Can someone please explain what is going?
>
> thanks
>
> SK
kimec.ethome.sk kimec.ethome.sk
Reply | Threaded
Open this post in threaded view
|

Re: Performance Question

Hi,

I would suggest you to deploy ignite server nodes on separate machines.
If you are using virtualized infrastructure, I would suggest you to
create two separate VMs. Ideally each VM should be running on a
different hypervisor with enough headroom so that both VMs don't compete
for resources (CPUs) at the same time.
In addition to that, you should probably have the client application
running on yet another dedicated machine.

That being said, I believe proper performance testing is possible on
physical hardware only.

Also, are the 2,000,000 entries you use in your test unique? How do you
generate them?

Kind regards,

Kamil Mišúth

On 2019-10-20 20:48, Simin Eftekhari wrote:

> Hello Micheal,
> Thanks for the reply.
>
> I did not make any changes to the configuration (example-ignite.xml),
> in either scenario. There is no backup.
>
> https://github.com/apache/ignite/blob/master/examples/config/example-ignite.xml
>
> Also, in both cases, the ignite nodes are running on the same vm,
> meaning, there should not be any network hops.
> I am executing the same program, in one case as is, in another case,
> as is with an additional ignite node started in the background, using
> this script:
>
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/ExampleNodeStartup.java
>
> I am just doing some basic benchmarking using the existing ignite
> examples.
>
> SK
>
> On Sun, Oct 20, 2019 at 1:31 PM Mikael <[hidden email]>
> wrote:
>
>> Hi!
>>
>> Well, it depends on what you are doing I guess ;)
>>
>> The example use put(), this is the slowest way of putting anything
>> in
>> the cache compared to streamers for example, anyway, if you run your
>>
>> code on one node in a big loop that does put 2M times it will be
>> slower,
>> with two nodes it will have to send half of the entries to the other
>>
>> node, so 1M entries are being sent over the network to the other
>> node,
>> with one node you don't need to send anything over the network.
>>
>> You say 10 sec is slow, not sure what you compare with, too me that
>> sounds like a good time, Ignite will by default put your objects off
>>
>> heap so the objects have to be serialized  and moved off heap and
>> there
>> is a lot of things going on behind the scenes that takes time in
>> case
>> you compare with a HashMap or something.
>>
>> Ignite likes to do things in parallel, having one thread doing all
>> the
>> puts is the worst way to put a lot of entries in a cache, try to
>> create
>> a streamer and run your example with two nodes on that and see what
>> you
>> get, I am not sure what you intend to use it for ? you can feed
>> Ignite a
>> huge amount of entries per second but trying to get all of them
>> through
>> one single thread on one node is not going to do that.
>>
>> How did you configure the cache ? number of backups ? persistence ?
>> partitioned/replicated ?
>>
>> Mikael
>>
>> Den 2019-10-20 kl. 19:02, skrev Simin Eftekhari:
>>> Hello Everyone,
>>>
>>> I am new to apache ignite. I'd appreciate your help with the
>> following
>>> question.
>>>
>>> I modified the CacheApiExample, to insert 2,000,000 entries into
>> the
>>> cache.
>>>
>>>
>>
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheApiExample.java
>>>
>>> If I run the program as is, the insert all the entries takes less
>> 10
>>> seconds (which I think is still too long). If I start an
>> additional
>>> ignite node (on the same vm), the inserts take over 3 minutes.
>> This is
>>> a huge discrepancy. I'd have thought that an additional instance
>> would
>>> actually speed up the inserts.
>>>
>>> Can someone please explain what is going?
>>>
>>> thanks
>>>
>>> SK
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Performance Question

In reply to this post by Simin Eftekhari
Hello!

I think that you should use DataStreamer to populate data. Should remove most of this discrepancy.

Regards,
--
Ilya Kasnacheev


вс, 20 окт. 2019 г. в 20:02, Simin Eftekhari <[hidden email]>:
Hello Everyone,

I am new to apache ignite. I'd appreciate your help with the following question.

I modified the CacheApiExample, to insert 2,000,000 entries into the cache.


If I run the program as is, the insert all the entries takes less 10 seconds (which I think is still too long). If I start an additional ignite node (on the same vm), the inserts take over 3 minutes. This is a huge discrepancy. I'd have thought that an additional instance would actually speed up the inserts.

Can someone please explain what is going?

thanks

SK