whether IgniteHadoopIgfsSecondaryFileSystem will cache HDFS filein IGFS?

classic Classic list List threaded Threaded
4 messages Options
Xun Zhai Xun Zhai
Reply | Threaded
Open this post in threaded view
|

whether IgniteHadoopIgfsSecondaryFileSystem will cache HDFS filein IGFS?

It is not obviously performance difference between hdfs and IgniteHadoopIgfsSecondaryFileSystem

this is my configuration
core-site.xml

mapred-site.xml

default-config.xml

Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: whether IgniteHadoopIgfsSecondaryFileSystem will cache HDFS filein IGFS?

Hi,

What numbers do you get? What Ignite version are you on?

In fact IGFS should give you performance boost but make sure that you’re using Ignite 1.6.0 that has many performance related optimizations in this area.

Denis

On Jun 3, 2016, at 9:56 AM, Jason <[hidden email]> wrote:

It is not obviously performance difference between hdfs and IgniteHadoopIgfsSecondaryFileSystem

this is my configuration
core-site.xml
<34FEF805@EB25BF62.1F2A5157>

mapred-site.xml
<3104F80D@EB25BF62.1F2A5157>

default-config.xml

<36FEFA05@EB25BF62.1F2A5157>

Xun Zhai Xun Zhai
Reply | Threaded
Open this post in threaded view
|

Re: whether IgniteHadoopIgfsSecondaryFileSystem will cache HDFS filein IGFS?

This post was updated on .
my job worked correctly, my ignite version is 1.6.0, i run the job twice, but when i watched my cache hits by cmd "cache -system", no cache is hited
Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: whether IgniteHadoopIgfsSecondaryFileSystem will cache HDFS filein IGFS?

Do you measure cache reads or cache puts? Cache reads should be faster after IGFS is warmed up.

> On Jun 12, 2016, at 5:40 AM, Xun Zhai <[hidden email]> wrote:
>
> my job work correctly, the ignite version is 1.6.0, i run the job twice, but
> when i watch my cache hits by cmd "cache -system", no cache is hited
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/whether-IgniteHadoopIgfsSecondaryFileSystem-will-cache-HDFS-filein-IGFS-tp5399p5586.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.