Error in running wordcount hadoop example in ignite

classic Classic list List threaded Threaded
3 messages Options
mehdi sey mehdi sey
Reply | Threaded
Open this post in threaded view
|

Error in running wordcount hadoop example in ignite

hi
i want to execute wordcount example of hadoop in apache ignite. i have used
apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my
default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just
as bellow:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/util
       http://www.springframework.org/schema/util/spring-util.xsd">

   
    <description>
        Spring file for Ignite node configuration with IGFS and Apache
Hadoop map-reduce support enabled.
        Ignite node will start with this configuration by default.
    </description>

   
    <bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="systemPropertiesModeName"
value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
        <property name="searchSystemEnvironment" value="true"/>
    </bean>

   
    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
       
        <property name="connectorConfiguration">
            <bean
class="org.apache.ignite.configuration.ConnectorConfiguration">
                <property name="port" value="11211"/>
            </bean>
        </property>
       
        <property name="includeEventTypes">
            <list>
               
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_JOB_MAPPED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>

               
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
            </list>
        </property>

i have run an ignite node with bellow command in command line in linux
ubuntu:
*/usr/local/apache-ignite-hadoop-2.6.0-bin/bin/ignite.sh
/usr/local/apache-ignite-hadoop-2.6.0-bin/config/default-config.xml*

after starting ignite node i execute a wordcount example in hadoop for
ruuning in ignite with bellow command:

*./hadoop jar
/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar
wordcount /input/hadoop output2*

but after executing above command i have encounter an error just as attached
image. please help about solving problem. i have seen also below link but it
could not help.
http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tc2146.html#a2183
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h2.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h1.png>





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Denis Mekhanikov Denis Mekhanikov
Reply | Threaded
Open this post in threaded view
|

Re: Error in running wordcount hadoop example in ignite

Hi!

As far as I can see, you tried running examples from Hadoop 2.7.7
But Ignite uses Hadoop version 2.4.1 internally.
So, I would start from checking the same examples for the matching version of Hadoop (2.4.1).

Denis

чт, 28 февр. 2019 г. в 08:58, mehdi sey <[hidden email]>:
hi
i want to execute wordcount example of hadoop in apache ignite. i have used
apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my
default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just
as bellow:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/util
       http://www.springframework.org/schema/util/spring-util.xsd">


    <description>
        Spring file for Ignite node configuration with IGFS and Apache
Hadoop map-reduce support enabled.
        Ignite node will start with this configuration by default.
    </description>


    <bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="systemPropertiesModeName"
value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
        <property name="searchSystemEnvironment" value="true"/>
    </bean>


    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="connectorConfiguration">
            <bean
class="org.apache.ignite.configuration.ConnectorConfiguration">
                <property name="port" value="11211"/>
            </bean>
        </property>

        <property name="includeEventTypes">
            <list>

                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_JOB_MAPPED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>


                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
            </list>
        </property>

i have run an ignite node with bellow command in command line in linux
ubuntu:
*/usr/local/apache-ignite-hadoop-2.6.0-bin/bin/ignite.sh
/usr/local/apache-ignite-hadoop-2.6.0-bin/config/default-config.xml*

after starting ignite node i execute a wordcount example in hadoop for
ruuning in ignite with bellow command:

*./hadoop jar
/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar
wordcount /input/hadoop output2*

but after executing above command i have encounter an error just as attached
image. please help about solving problem. i have seen also below link but it
could not help.
http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tc2146.html#a2183
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h2.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h1.png>





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
mehdi sey mehdi sey
Reply | Threaded
Open this post in threaded view
|

Re: Error in running wordcount hadoop example in ignite

thanks that works with 2.4

On Tue, Apr 2, 2019 at 7:32 AM Denis Mekhanikov <[hidden email]> wrote:
Hi!

As far as I can see, you tried running examples from Hadoop 2.7.7
But Ignite uses Hadoop version 2.4.1 internally.
So, I would start from checking the same examples for the matching version of Hadoop (2.4.1).

Denis

чт, 28 февр. 2019 г. в 08:58, mehdi sey <[hidden email]>:
hi
i want to execute wordcount example of hadoop in apache ignite. i have used
apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my
default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just
as bellow:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/util
       http://www.springframework.org/schema/util/spring-util.xsd">


    <description>
        Spring file for Ignite node configuration with IGFS and Apache
Hadoop map-reduce support enabled.
        Ignite node will start with this configuration by default.
    </description>


    <bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="systemPropertiesModeName"
value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
        <property name="searchSystemEnvironment" value="true"/>
    </bean>


    <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="connectorConfiguration">
            <bean
class="org.apache.ignite.configuration.ConnectorConfiguration">
                <property name="port" value="11211"/>
            </bean>
        </property>

        <property name="includeEventTypes">
            <list>

                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_JOB_MAPPED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>


                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
            </list>
        </property>

i have run an ignite node with bellow command in command line in linux
ubuntu:
*/usr/local/apache-ignite-hadoop-2.6.0-bin/bin/ignite.sh
/usr/local/apache-ignite-hadoop-2.6.0-bin/config/default-config.xml*

after starting ignite node i execute a wordcount example in hadoop for
ruuning in ignite with bellow command:

*./hadoop jar
/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar
wordcount /input/hadoop output2*

but after executing above command i have encounter an error just as attached
image. please help about solving problem. i have seen also below link but it
could not help.
http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tc2146.html#a2183
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h2.png>
<http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h1.png>





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/