I have a question in regards to Apache Ignite performance tuning.
This is my config so far:
I plan to store 2 billion entries in the database. The machine that I have is not great, 32gb ram nothing more.
I wonder how I can tune the performance to speed up things. I know that if I leave out indexing that would speed things are but indices are necessary. Also an SQL access is necessary though not necessarily during inserts. As I'm using C++ I don't know if ignite supports the feature in creating an SQL access later on after the data has been inserted in a cache. Is this possible?
According to my logging I'm able to insert
42108964 entries in 66 minutes
with 610 Insertions per second:610
and an average of 1057 insertions per second (which is not great in my opinion)
According to ignites performance guidelines I segregated already to Wal stores from the persistence storage drive.
I'm also getting Apache ignite errors from time to time:
[21:24:15] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=db-checkpoint-thread, igniteInstanceName=null, finished=false, heartbeatTs=1605385441327]]]
Tried to find something online and in the Apache Ignite documentation but struggles so hopefully any geek out there can drop a comment...
How many nodes will you be running this on ?
What is your use case, what kind of performance is important to you, reading, writing ? do you need access to all data or will you just use a small subset ?
In terms of writing you can get good performance with a streamer
instead of using SQL insert if that would be possible.
On 2020-11-14 23:35, Wolfgang Meyerle wrote:
|Free forum by Nabble||Edit this page|