How to make full use of network bandwidth?

classic Classic list List threaded Threaded
5 messages Options
Michael Jay Michael Jay
Reply | Threaded
Open this post in threaded view
|

How to make full use of network bandwidth?

Hi,
I have two physical hosts, each host has a sever node, node A and node B. I
created a table and inserted data into table via JDBCthin driver. But the
performance of inserting is poor. When the bandwidth of network is 100Mbps,
the throughput from node A to node B is only about 2Mbps, 1 million rows
need 800s to be finished. Then I only changed the bandwidth of network to
1Gbps, the throughput from node A to node B is up to 10Mbps, 1 million rows
need 250s to be finished. Moreover, the cacheMode is "partitioned".
How to explain this case? Is there some limitaiton or configuration on
network? What did I miss? How can How can we  use all network bandwidth?
Thanks!!
  default-config.xml
<http://apache-ignite-users.70518.x6.nabble.com/file/t1501/default-config.xml>  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexey Popov Alexey Popov
Reply | Threaded
Open this post in threaded view
|

Re: How to make full use of network bandwidth?

Hi Michael,

Did you try non-default parameters for
1) socketSendBuffer and socketReceiveBuffer [1] in JDBC connection string?
2) socketSendBufferSize and socketReceiveBufferSize [2] in Ignite server
node configuration for SqlConnectorConfiguration?

Please change them to 128k and make a try.

JDBC thin driver works in a single thread in one-by-one request-by-response
SQL statement execution. I am not sure you can fully utilize a network here.
Probably, you could use an IgniteDataStreamer to achieve better results.
Please have a look at [3].
Or you should create multiple JDBC connections and parallel your inserts in
multiple threads.

[1]
https://apacheignite-sql.readme.io/docs/jdbc-driver#section-jdbc-thin-driver
[2]
https://apacheignite-sql.readme.io/docs/jdbc-driver#section-cluster-configuration
[3] https://apacheignite.readme.io/docs/data-loading

Thank you,
Alexey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Michael Jay Michael Jay
Reply | Threaded
Open this post in threaded view
|

Re: How to make full use of network bandwidth?

Thank you, Alexey. I'll try you advice and let you know the result later.
Thanks again.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: How to make full use of network bandwidth?

Hi Michael, were you able to apply the suggestions. It would be nice if you would share your results with the community.

D.

On Mon, Jan 8, 2018 at 6:42 PM, Michael Jay <[hidden email]> wrote:
Thank you, Alexey. I'll try you advice and let you know the result later.
Thanks again.

Michael Jay Michael Jay
Reply | Threaded
Open this post in threaded view
|

Re: How to make full use of network bandwidth?

Sorry, I almost forgot it. I have  tried the suggestions above, but it didn't
work well. Finally, I changed the parameter of writeSynchronizationMode to
"FULL_AYSNC", it worked.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/