I found an expected behaviour on apache ignite.
If you do ( TABLE partitioned with 1 backup and ATOMIC mode ):
alter table .. nologging;
set stream on ;
.. loading a lot of data with jdbc ..
.. close connection to flush data ..
.. reopen connection..
alter table .. logging
The last step "alter table logging" writes a lot of data and it's "slow", if we load the table
with 26 threads in about 10 minute and every thread uses a dedicated partition, , the nologging phase is around 11 minutes.
I would like to understand what it is doing and how to speed up the process, I try to understand from SqlAlterTableCommand.java what it's doing, but it's not straightforward.
I guess it has to write all the data to disk. After 'alter table logging' returns, you are guaranteed consistency on this table, meaning all of its pages have to be persisted to disk. Obviously, it may take a lot of time if you have many gigabytes to flush.
Hello Ilya ,
thank you for your answer, it's make sense. I see the class called it's changewal and as you wrote, it's probably call a flush of the cache .
Is it possible to speed up the flush to disk with more threads ? I think they are controlled by system pool or is it another parameter?
Il giorno lun 26 ott 2020 alle ore 13:05 Ilya Kasnacheev <[hidden email]> ha scritto:
Usually it is I/O bound, meaning, you can only speed it up by having more, faster disks.
However, in some cases, tuning persistence settings such as checkpoint frequency, checkpoint page buffer size and checkpoint thread pool size may help. You can also try the direct-io module, but you may see mixed results.
пн, 26 окт. 2020 г. в 16:00, Matteo Durighetto <[hidden email]>:
|Free forum by Nabble||Edit this page|