Apache MapReduce Job with Ignite

classic Classic list List threaded Threaded
2 messages Options
pgarg pgarg
Reply | Threaded
Open this post in threaded view
|

Apache MapReduce Job with Ignite

asked by shah

Hi,

I am trying to use IGFS with Apache Mapreduce Jobs but I am getting errors for failed connection with IGFS. Also, I tried to install ignite on Apache Hadoop, trying to use in-memory mapreduce along with IGFS, I still get error messages for failed connections, here are the messages that I get on console that is running ignite.sh,
Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- H/N/C [hosts=1, nodes=1, CPUs=1] ^-- CPU [cur=1%, avg=0.87%, GC=0%] ^-- Heap [used=182MB, free=81.52%, comm=989MB] ^-- Public thread pool [active=0, idle=16, qSize=0] ^-- System thread pool [active=0, idle=16, qSize=0] ^-- Outbound messages queue [size=0] [24-Apr-2015 13:04:46][ERROR][igfs-accept-worker-#76%null%][IpcSharedMemoryServerEndpoint] Failed to process incoming shared memory connection: invalid stream header: 47455420 [24-04-2015 13:05:16][INFO ][ignite-metrics-logger][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- H/N/C [hosts=1, nodes=1, CPUs=1]

^-- CPU [cur=2.33%, avg=0.88%, GC=0%]

I am using the example-igfs.xml as configuration file. Any easy way to run the ignite on Apache hadoop using igfs?

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss
pgarg pgarg
Reply | Threaded
Open this post in threaded view
|

Re: Apache MapReduce Job with Ignite

commented by vladimir ozerov

Shah, It appears to be a problem with communication between Hadoop and Ignite processes over shared memory. Could you please enable debug as described below, reproduce the problem and attach the logs here? 1) Copy files from /libs/optional/ignite-log4j to /libs 2) Add the following tag to the file /config/ignite-log4j.xml:

<category name="org.apache.ignite.internal.util.ipc"> <level value="DEBUG"/> </category>

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss