Expected serialization performance of Ignite .NET

classic Classic list List threaded Threaded
6 messages Options
camer314 camer314
Reply | Threaded
Open this post in threaded view
|

Expected serialization performance of Ignite .NET

I have a 21 property C# class (mix of int and string) and am using
IBinarizable interface as suggested in the documentation.

My cache is configured such that each cache entry is a collection of these
objects, lets say each cache item is a List.

I have 10 million instance of this class. For simplicity lets say each cache
entry holds 10 of these objects, amounting to 1 million cache entries.

Each object is roughly about 120 bytes long, so 10 million = ~1.2 gigabytes
of data stored.

I am using a LOCAL cache and a simple foreach loop over the cache takes in
the region of 25 seconds. This seems like an eternity. I understand there is
a lot of serialization happening, probably a lot of garbage collecting as
well, but it still seems like a large amount of time to effectively move
memory from one location to another.

Does that time seem exorbitant to you given the above specs or is it
expected?

What is the optimal way to lay out cache items locally for cache read
iteration (that is, compute needs to iterate the entire cache)?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Expected serialization performance of Ignite .NET

Can you please attach a working project to reproduce those numbers?

It is hard to say without the code: a class with 21 properties can vary in size a lot.
There are many other things at play - JVM options, RAM size, benchmark method, etc.

On Mon, Dec 2, 2019 at 2:08 PM camer314 <[hidden email]> wrote:
I have a 21 property C# class (mix of int and string) and am using
IBinarizable interface as suggested in the documentation.

My cache is configured such that each cache entry is a collection of these
objects, lets say each cache item is a List.

I have 10 million instance of this class. For simplicity lets say each cache
entry holds 10 of these objects, amounting to 1 million cache entries.

Each object is roughly about 120 bytes long, so 10 million = ~1.2 gigabytes
of data stored.

I am using a LOCAL cache and a simple foreach loop over the cache takes in
the region of 25 seconds. This seems like an eternity. I understand there is
a lot of serialization happening, probably a lot of garbage collecting as
well, but it still seems like a large amount of time to effectively move
memory from one location to another.

Does that time seem exorbitant to you given the above specs or is it
expected?

What is the optimal way to lay out cache items locally for cache read
iteration (that is, compute needs to iterate the entire cache)?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
camer314 camer314
Reply | Threaded
Open this post in threaded view
|

Re: Expected serialization performance of Ignite .NET

It seems a combination of a better spec machine and a parallel for loop has
improved performance, although it still takes 8 seconds to run through all
the cache items.

Here is some basic test code...would appreciate any tips on how to improve
access in this type of usage pattern:

https://wtwdeeplearning.blob.core.windows.net/temp/ignitetest.zip?st=2019-12-03T05%3A47%3A38Z&se=2019-12-12T05%3A47%3A00Z&sp=rl&sv=2018-03-28&sr=b&sig=t%2FXw4bpAFRo7aKdpIbwLfTFOB4Sv%2FeetSi%2FvVSRjg8w%3D

On my VM it takes 25 seconds to populate and 8 seconds to retrieve.

What is the most efficient way to iterate all items on a local cache? In a
real situation I would not know the keys i have in the cache and the only
way I could get decent throughput on the read was the parallel loop which
implies i know the keys already.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Expected serialization performance of Ignite .NET

I've tried your code and got similar numbers (11s to write, 3s to read).
Profiler shows that serialization takes 90% of the streaming time (40% is your WriteBinaryInternal method, the rest is Ignite internals)
Actual size of serialized object is 161 bytes, and DataRegionMetris.TotalAllocatedSize is ~2.3Gb, which is how much memory Ignite actually uses to store all that data

Your code is good, and timings seem good to me too (serializing 2 gigabytes in 10 seconds is not bad in my books).
I don't see anything to speed up here, tweaking JVM does not seem to do much.
You could split source data in some batches and run multiple streamers in parallel, that brings the time down to 6s on my machine.

> What is the most efficient way to iterate all items on a local cache
Either ScanQuery without filter, or simply foreach (var entry in cache) - ICache implements IEnumerable



On Tue, Dec 3, 2019 at 9:12 AM camer314 <[hidden email]> wrote:
It seems a combination of a better spec machine and a parallel for loop has
improved performance, although it still takes 8 seconds to run through all
the cache items.

Here is some basic test code...would appreciate any tips on how to improve
access in this type of usage pattern:

https://wtwdeeplearning.blob.core.windows.net/temp/ignitetest.zip?st=2019-12-03T05%3A47%3A38Z&se=2019-12-12T05%3A47%3A00Z&sp=rl&sv=2018-03-28&sr=b&sig=t%2FXw4bpAFRo7aKdpIbwLfTFOB4Sv%2FeetSi%2FvVSRjg8w%3D

On my VM it takes 25 seconds to populate and 8 seconds to retrieve.

What is the most efficient way to iterate all items on a local cache? In a
real situation I would not know the keys i have in the cache and the only
way I could get decent throughput on the read was the parallel loop which
implies i know the keys already.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
camer314 camer314
Reply | Threaded
Open this post in threaded view
|

Re: Expected serialization performance of Ignite .NET

Ok thanks.

I have managed to get the times down to 13 seconds write and 7.5 seconds
read, basically double your times, on my 12 processor Intel i7-8850 laptop
which is running 100% so you must have some good kit!

Does Java version have any bearing on performance?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ptupitsyn ptupitsyn
Reply | Threaded
Open this post in threaded view
|

Re: Expected serialization performance of Ignite .NET

> 12 processor Intel i7-8850 laptop which is running 100% so you must have some good kit
I got those numbers on desktop i7-9700k. Laptops are still no match for desktops (unfortunately).

> Does Java version have any bearing on performance?
Java version can certainly affect performance, but not in this case, where .NET part does most of the work

On Wed, Dec 4, 2019 at 2:53 AM camer314 <[hidden email]> wrote:
Ok thanks.

I have managed to get the times down to 13 seconds write and 7.5 seconds
read, basically double your times, on my 12 processor Intel i7-8850 laptop
which is running 100% so you must have some good kit!

Does Java version have any bearing on performance?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/