Does any one have working Ignite cluster on AWS

classic Classic list List threaded Threaded
14 messages Options
sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Does any one have working Ignite cluster on AWS

Hi All, 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

--
Thanks & Regards
Sri Tummala

favas favas
Reply | Threaded
Open this post in threaded view
|

RE: Does any one have working Ignite cluster on AWS

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

thanks for replying but sorry not getting it please dumb it down, below are my questions.

Statically giving the public IP list in the IP configuration section in ignite config file.
Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.
Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

Thanks
Sri 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala

ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

Hello!

Please refer to the docs:


Regards,
--
Ilya Kasnacheev


пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:
thanks for replying but sorry not getting it please dumb it down, below are my questions.

Statically giving the public IP list in the IP configuration section in ignite config file.
Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.
Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

Thanks
Sri 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

documentation is not covering step by step it's not that helpful.

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Please refer to the docs:


Regards,
--
Ilya Kasnacheev


пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:
thanks for replying but sorry not getting it please dumb it down, below are my questions.

Statically giving the public IP list in the IP configuration section in ignite config file.
Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.
Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

Thanks
Sri 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS


Just swap GridGain with Ignite artifacts, the rest is identical.

-
Denis


On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:
documentation is not covering step by step it's not that helpful.

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:
Hello!

Please refer to the docs:


Regards,
--
Ilya Kasnacheev


пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:
thanks for replying but sorry not getting it please dumb it down, below are my questions.

Statically giving the public IP list in the IP configuration section in ignite config file.
Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.
Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

Thanks
Sri 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

favas favas
Reply | Threaded
Open this post in threaded view
|

RE: Does any one have working Ignite cluster on AWS

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

also can you send me full config file? so I dont have any typos etc....

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <valueIP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>


On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

In reply to this post by sri hari kali charan Tummala
Working Gridgain Ignite cluster on AWS, followed below steps and changed a bit with aws_static_ip file.

https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2

aws_static_ip:-

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

On Wed, Oct 16, 2019 at 1:14 PM sri hari kali charan Tummala <[hidden email]> wrote:
I am getting bean invalid exception can you help? values in my aws-static-ip-finder.xml file is below.

<bean class="org.apache.ignite.configuration.IgniteConfiguration" >

    <!-- other properties -->

    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->

    <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

            <property name="ipFinder">

                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <property name="addresses">

                        <list>

                            <value>172.31.81.211</value>

                            <value>172.31.82.21</value>

                        </list>

                    </property>

                </bean>

            </property>

        </bean>

    </property>

</bean>





class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1052)

at org.apache.ignite.Ignition.start(Ignition.java:350)

at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:103)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:97)

at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:750)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:860)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:730)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:699)

at org.apache.ignite.Ignition.start(Ignition.java:347)

... 1 more

Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:378)

... 9 more

Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)

at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1901)

at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:741)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:613)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3132)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:852)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)

at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)

at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)

at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:429)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:391)

... 12 more

Failed to start grid: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]


On Wed, Oct 16, 2019 at 1:01 PM sri hari kali charan Tummala <[hidden email]> wrote:
Hi Denis, 

Do I need to do below steps for both the ec2 instances? or just one ?


Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem [hidden email]

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.7.6.zip scp://[hidden email]

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.7.6.zip

If you are going to connect to the cluster via REST API or use Web Consoleenable the 'ignite-rest-http' module.

$ cp -r gridgain-community-8.7.6/libs/optional/ignite-rest-http/ gridgain-community-8.7.6/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.


Thanks

Sri 





On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

what are the next steps after creating a cluster, I want to run basic sql create some tables and load some data right now I dont want to code right away in Java will sqlline.sh works? I did launched it fails with no current connection.



On Wed, Oct 16, 2019 at 2:24 PM sri hari kali charan Tummala <[hidden email]> wrote:
Working Gridgain Ignite cluster on AWS, followed below steps and changed a bit with aws_static_ip file.

https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2

aws_static_ip:-

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

On Wed, Oct 16, 2019 at 1:14 PM sri hari kali charan Tummala <[hidden email]> wrote:
I am getting bean invalid exception can you help? values in my aws-static-ip-finder.xml file is below.

<bean class="org.apache.ignite.configuration.IgniteConfiguration" >

    <!-- other properties -->

    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->

    <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

            <property name="ipFinder">

                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                    <property name="addresses">

                        <list>

                            <value>172.31.81.211</value>

                            <value>172.31.82.21</value>

                        </list>

                    </property>

                </bean>

            </property>

        </bean>

    </property>

</bean>





class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1052)

at org.apache.ignite.Ignition.start(Ignition.java:350)

at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:103)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:97)

at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:750)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:860)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:730)

at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:699)

at org.apache.ignite.Ignition.start(Ignition.java:347)

... 1 more

Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)

at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:378)

... 9 more

Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)

at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)

at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)

at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1901)

at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:741)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:613)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3132)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:852)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)

at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)

at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)

at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)

at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)

at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)

at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)

at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:429)

at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:391)

... 12 more

Failed to start grid: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]


On Wed, Oct 16, 2019 at 1:01 PM sri hari kali charan Tummala <[hidden email]> wrote:
Hi Denis, 

Do I need to do below steps for both the ec2 instances? or just one ?


Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem [hidden email]

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.7.6.zip scp://[hidden email]

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.7.6.zip

If you are going to connect to the cluster via REST API or use Web Consoleenable the 'ignite-rest-http' module.

$ cp -r gridgain-community-8.7.6/libs/optional/ignite-rest-http/ gridgain-community-8.7.6/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.


Thanks

Sri 





On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala


Screen Shot 2019-10-16 at 2.38.31 PM.png (22K) Download Attachment
stephendarlington stephendarlington
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

You have to tell it where to connect:

./sqlline -u jdbc:ignite:<a href="thin://127.0.0.1/" class="">thin://127.0.0.1/

I also wrote this showing a few ways to load data without firing up an IDE:


My current favourite “no Java” method of playing around is a Scala REPL.

Regards,
Stephen

On 16 Oct 2019, at 19:38, sri hari kali charan Tummala <[hidden email]> wrote:

what are the next steps after creating a cluster, I want to run basic sql create some tables and load some data right now I dont want to code right away in Java will sqlline.sh works? I did launched it fails with no current connection.



On Wed, Oct 16, 2019 at 2:24 PM sri hari kali charan Tummala <[hidden email]> wrote:
Working Gridgain Ignite cluster on AWS, followed below steps and changed a bit with aws_static_ip file.

https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2

aws_static_ip:-

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

On Wed, Oct 16, 2019 at 1:14 PM sri hari kali charan Tummala <[hidden email]> wrote:
I am getting bean invalid exception can you help? values in my aws-static-ip-finder.xml file is below.

<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>




class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1052)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:103)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:97)
at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:750)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:860)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:730)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:699)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:378)
... 9 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1901)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:741)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:613)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3132)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:852)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:429)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:391)
... 12 more
Failed to start grid: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

On Wed, Oct 16, 2019 at 1:01 PM sri hari kali charan Tummala <[hidden email]> wrote:
Hi Denis, 

Do I need to do below steps for both the ec2 instances? or just one ?


Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem [hidden email]

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.7.6.zip scp://[hidden email]

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.7.6.zip

If you are going to connect to the cluster via REST API or use Web Consoleenable the 'ignite-rest-http' module.

$ cp -r gridgain-community-8.7.6/libs/optional/ignite-rest-http/ gridgain-community-8.7.6/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.


Thanks

Sri 





On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

<Screen Shot 2019-10-16 at 2.38.31 PM.png>


sri hari kali charan Tummala sri hari kali charan Tummala
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

Hi Stephen , 

in your example or any spark example I only see creating local ignite node how to connect spark to ignite client which I already have?

b.filter('href is not null') \
.drop('hash', 'meta') \
.write.format('ignite') \
.option('config','default-config.xml') \
.option('table','bookmarks') \
.option('primaryKeyFields','href') \
.mode('overwrite') \
.save()

My code:-

package com.ignite.examples.spark

import com.ignite.examples.model.Address
import org.apache.ignite.{Ignite, Ignition}
import org.apache.ignite.cache.query.SqlFieldsQuery
import org.apache.ignite.client.{ClientCache, IgniteClient}
import org.apache.ignite.configuration.{CacheConfiguration, ClientConfiguration}
import java.lang.{Long => JLong, String => JString}
import org.apache.ignite.cache.query.SqlFieldsQuery
import org.apache.ignite.spark.IgniteDataFrameSettings.{FORMAT_IGNITE, OPTION_CONFIG_FILE, OPTION_TABLE}
import org.apache.log4j.{Level, Logger}
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.col

object SparkClientConnectionTest {

private val CACHE_NAME = "SparkCache"

//private val CONFIG = "/Users/kalit_000/Downloads/designing-event-driven-applications-apache-kafka-ecosystem/05/demos/kafka-streams-after/ApacheIgnitePoc/src/main/scala/com/ignite/examples/config/example-ignite.xml"


def setupExampleData = {

val cfg2 = new ClientConfiguration().setAddresses("3.88.248.113:10800")
val igniteClient:IgniteClient = Ignition.startClient(cfg2)

System.out.format(">>> Created cache [%s].\n", CACHE_NAME)

val cache:ClientCache[Integer, Address] = igniteClient.getOrCreateCache(CACHE_NAME)

cache.query(new SqlFieldsQuery(String.format("DROP TABLE IF EXISTS Person"))
.setSchema("PUBLIC")).getAll

cache.query(new SqlFieldsQuery(String.format("CREATE TABLE IF NOT EXISTS Person (id LONG,street varchar, zip VARCHAR, PRIMARY KEY (id) ) WITH \"VALUE_TYPE=%s\"", classOf[Address].getName))
.setSchema("PUBLIC")).getAll

cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(1L.asInstanceOf[JLong],"Jameco", "04074").setSchema("PUBLIC")).getAll
cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(2L.asInstanceOf[JLong],"Bremar road", "520003").setSchema("PUBLIC")).getAll
cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(3L.asInstanceOf[JLong],"orange road", "1234").setSchema("PUBLIC")).getAll

System.out.format(">>> Data Inserted into Cache [%s].\n", CACHE_NAME)

val data=cache.query(new SqlFieldsQuery("select * from Person").setSchema("PUBLIC")).getAll

println(data.toString)
}

def sparkDSLExample(implicit spark: SparkSession): Unit = {
println("Querying using Spark DSL.")
println

val igniteDF = spark.read
.format(FORMAT_IGNITE) //Data source type.
.option(OPTION_TABLE, "person") //Table to read.
.option(OPTION_CONFIG_FILE, CONFIG) //Ignite config.
.load()
.filter(col("id") >= 2) //Filter clause.
.filter(col("name") like "%J%") //Another filter clause.

println("Data frame schema:")

igniteDF.printSchema() //Printing query schema to console.

println("Data frame content:")

igniteDF.show() //Printing query results to console.
}


def main(args: Array[String]): Unit = {

setupExampleData

//Creating spark session.
implicit val spark = SparkSession.builder()
.appName("Spark Ignite data sources example")
.master("local")
.config("spark.executor.instances", "2")
.getOrCreate()

// Adjust the logger to exclude the logs of no interest.
Logger.getRootLogger.setLevel(Level.ERROR)
Logger.getLogger("org.apache.ignite").setLevel(Level.INFO)

sparkDSLExample




}

}




On Thu, Oct 17, 2019 at 7:09 AM Stephen Darlington <[hidden email]> wrote:
You have to tell it where to connect:

./sqlline -u jdbc:ignite:thin://127.0.0.1/

I also wrote this showing a few ways to load data without firing up an IDE:


My current favourite “no Java” method of playing around is a Scala REPL.

Regards,
Stephen

On 16 Oct 2019, at 19:38, sri hari kali charan Tummala <[hidden email]> wrote:

what are the next steps after creating a cluster, I want to run basic sql create some tables and load some data right now I dont want to code right away in Java will sqlline.sh works? I did launched it fails with no current connection.



On Wed, Oct 16, 2019 at 2:24 PM sri hari kali charan Tummala <[hidden email]> wrote:
Working Gridgain Ignite cluster on AWS, followed below steps and changed a bit with aws_static_ip file.

https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2

aws_static_ip:-

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

On Wed, Oct 16, 2019 at 1:14 PM sri hari kali charan Tummala <[hidden email]> wrote:
I am getting bean invalid exception can you help? values in my aws-static-ip-finder.xml file is below.

<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>




class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1052)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:103)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:97)
at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:750)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:860)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:730)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:699)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:378)
... 9 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1901)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:741)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:613)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3132)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:852)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:429)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:391)
... 12 more
Failed to start grid: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

On Wed, Oct 16, 2019 at 1:01 PM sri hari kali charan Tummala <[hidden email]> wrote:
Hi Denis, 

Do I need to do below steps for both the ec2 instances? or just one ?


Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem [hidden email]

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.7.6.zip scp://[hidden email]

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.7.6.zip

If you are going to connect to the cluster via REST API or use Web Consoleenable the 'ignite-rest-http' module.

$ cp -r gridgain-community-8.7.6/libs/optional/ignite-rest-http/ gridgain-community-8.7.6/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.


Thanks

Sri 





On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

<Screen Shot 2019-10-16 at 2.38.31 PM.png>




--
Thanks & Regards
Sri Tummala

stephendarlington stephendarlington
Reply | Threaded
Open this post in threaded view
|

Re: Does any one have working Ignite cluster on AWS

Spark starts a thick-client and connects to the cluster you specify in the discoverySpi section of the configuration file.

On 17 Oct 2019, at 17:05, sri hari kali charan Tummala <[hidden email]> wrote:

Hi Stephen , 

in your example or any spark example I only see creating local ignite node how to connect spark to ignite client which I already have?

b.filter('href is not null') \
.drop('hash', 'meta') \
.write.format('ignite') \
.option('config','default-config.xml') \
.option('table','bookmarks') \
.option('primaryKeyFields','href') \
.mode('overwrite') \
.save()

My code:-

package com.ignite.examples.spark

import com.ignite.examples.model.Address
import org.apache.ignite.{Ignite, Ignition}
import org.apache.ignite.cache.query.SqlFieldsQuery
import org.apache.ignite.client.{ClientCache, IgniteClient}
import org.apache.ignite.configuration.{CacheConfiguration, ClientConfiguration}
import java.lang.{Long => JLong, String => JString}
import org.apache.ignite.cache.query.SqlFieldsQuery
import org.apache.ignite.spark.IgniteDataFrameSettings.{FORMAT_IGNITE, OPTION_CONFIG_FILE, OPTION_TABLE}
import org.apache.log4j.{Level, Logger}
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.col

object SparkClientConnectionTest {

private val CACHE_NAME = "SparkCache"

//private val CONFIG = "/Users/kalit_000/Downloads/designing-event-driven-applications-apache-kafka-ecosystem/05/demos/kafka-streams-after/ApacheIgnitePoc/src/main/scala/com/ignite/examples/config/example-ignite.xml"


def setupExampleData = {

val cfg2 = new ClientConfiguration().setAddresses("3.88.248.113:10800")
val igniteClient:IgniteClient = Ignition.startClient(cfg2)

System.out.format(">>> Created cache [%s].\n", CACHE_NAME)

val cache:ClientCache[Integer, Address] = igniteClient.getOrCreateCache(CACHE_NAME)

cache.query(new SqlFieldsQuery(String.format("DROP TABLE IF EXISTS Person"))
.setSchema("PUBLIC")).getAll

cache.query(new SqlFieldsQuery(String.format("CREATE TABLE IF NOT EXISTS Person (id LONG,street varchar, zip VARCHAR, PRIMARY KEY (id) ) WITH \"VALUE_TYPE=%s\"", classOf[Address].getName))
.setSchema("PUBLIC")).getAll

cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(1L.asInstanceOf[JLong],"Jameco", "04074").setSchema("PUBLIC")).getAll
cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(2L.asInstanceOf[JLong],"Bremar road", "520003").setSchema("PUBLIC")).getAll
cache.query(new SqlFieldsQuery("INSERT INTO Person(id,street, zip) VALUES(?,?, ?)").setArgs(3L.asInstanceOf[JLong],"orange road", "1234").setSchema("PUBLIC")).getAll

System.out.format(">>> Data Inserted into Cache [%s].\n", CACHE_NAME)

val data=cache.query(new SqlFieldsQuery("select * from Person").setSchema("PUBLIC")).getAll

println(data.toString)
}

def sparkDSLExample(implicit spark: SparkSession): Unit = {
println("Querying using Spark DSL.")
println

val igniteDF = spark.read
.format(FORMAT_IGNITE) //Data source type.
.option(OPTION_TABLE, "person") //Table to read.
.option(OPTION_CONFIG_FILE, CONFIG) //Ignite config.
.load()
.filter(col("id") >= 2) //Filter clause.
.filter(col("name") like "%J%") //Another filter clause.

println("Data frame schema:")

igniteDF.printSchema() //Printing query schema to console.

println("Data frame content:")

igniteDF.show() //Printing query results to console.
}


def main(args: Array[String]): Unit = {

setupExampleData

//Creating spark session.
implicit val spark = SparkSession.builder()
.appName("Spark Ignite data sources example")
.master("local")
.config("spark.executor.instances", "2")
.getOrCreate()

// Adjust the logger to exclude the logs of no interest.
Logger.getRootLogger.setLevel(Level.ERROR)
Logger.getLogger("org.apache.ignite").setLevel(Level.INFO)

sparkDSLExample




}

}




On Thu, Oct 17, 2019 at 7:09 AM Stephen Darlington <[hidden email]> wrote:
You have to tell it where to connect:

./sqlline -u jdbc:ignite:thin://127.0.0.1/

I also wrote this showing a few ways to load data without firing up an IDE:


My current favourite “no Java” method of playing around is a Scala REPL.

Regards,
Stephen

On 16 Oct 2019, at 19:38, sri hari kali charan Tummala <[hidden email]> wrote:

what are the next steps after creating a cluster, I want to run basic sql create some tables and load some data right now I dont want to code right away in Java will sqlline.sh works? I did launched it fails with no current connection.



On Wed, Oct 16, 2019 at 2:24 PM sri hari kali charan Tummala <[hidden email]> wrote:
Working Gridgain Ignite cluster on AWS, followed below steps and changed a bit with aws_static_ip file.

https://www.gridgain.com/docs/8.7.6//installation-guide/manual-install-on-ec2

aws_static_ip:-

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

On Wed, Oct 16, 2019 at 1:14 PM sri hari kali charan Tummala <[hidden email]> wrote:
I am getting bean invalid exception can you help? values in my aws-static-ip-finder.xml file is below.

<bean class="org.apache.ignite.configuration.IgniteConfiguration" >
    <!-- other properties -->
    <!-- Explicitly configure TCP discovery SPI to provide a list of nodes. -->
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>172.31.81.211</value>
                            <value>172.31.82.21</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>




class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1052)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:391)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:103)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:97)
at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:750)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:860)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:730)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:699)
at org.apache.ignite.Ignition.start(Ignition.java:347)
... 1 more
Caused by: org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:399)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304)
at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:378)
... 9 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1901)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:741)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:613)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3132)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:852)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:76)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadDocument(XmlBeanDefinitionReader.java:429)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:391)
... 12 more
Failed to start grid: Failed to instantiate Spring XML application context [springUrl=file:/home/ec2-user/aws-static-ip-finder.xml, err=Line 1 in XML document from URL [file:/home/ec2-user/aws-static-ip-finder.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 68; cvc-elt.1: Cannot find the declaration of element 'bean'.]

On Wed, Oct 16, 2019 at 1:01 PM sri hari kali charan Tummala <[hidden email]> wrote:
Hi Denis, 

Do I need to do below steps for both the ec2 instances? or just one ?


Setting Up the Environment

Connect to the instance via ssh:

$ ssh -i privatekey.pem [hidden email]

If java is not already installed, install it using the package manager of the instance.

$ sudo yum install java-1.8.0-openjdk.x86_64

Upload a GridGain distribution package into the instance. Run the following command from you local machine:

$ scp -i privatekey.pem gridgain-community-8.7.6.zip scp://[hidden email]

Login to the instance again and unpack the package:

$ unzip gridgain-community-8.7.6.zip

If you are going to connect to the cluster via REST API or use Web Consoleenable the 'ignite-rest-http' module.

$ cp -r gridgain-community-8.7.6/libs/optional/ignite-rest-http/ gridgain-community-8.7.6/libs/

Repeat the above steps for the second instance. Now we are ready to configure the cluster nodes.


Thanks

Sri 





On Wed, Oct 16, 2019 at 9:18 AM sri hari kali charan Tummala <[hidden email]> wrote:
Hi , 

someone sent me this I will give a try.


Thanks
Sri 

On Wed, Oct 16, 2019 at 12:26 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

For Question 1

                In the ignite config file you use to launch ignite, you can give list of IP’s like below. The same config file should be placed in all the Ec2 servers. Once it is done, you can start ignite by using command $IGNITE_HOME/bin.ignite.sh.

 

<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->

        <property name="discoverySpi">

              

            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

                

                <property name="ipFinder">

                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">

                            <property name="addresses">

                                <list>

                                    <value> IP1</value>  

                                    <value> IP2 </value>

                                    <value> IP3 </value>

                                </list>

                            </property>

                    </bean>

 

                </property>

            </bean>

        </property>

 

For Question 2

                In this approach, you don’t need to give the list of IP’s in ignite config file, instead you just have to create an s3 bucket and mention the s3 bucket name and access key in the ignite config. See below.

 

<!-- Discover IP using Amazone s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="<your s3 bucket name>"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="Your access key id"/>

                        <constructor-arg value="you secret access key"/>

                    </bean>

                </property>

            </bean>

        </property>

        </bean>

    </property>

                               

Hope it will help you

 

Regards,

Favas 

 

From: Denis Magda <[hidden email]>
Sent: Tuesday, October 15, 2019 11:21 PM
To: [hidden email]
Subject: Re: Does any one have working Ignite cluster on AWS

 

 

Just swap GridGain with Ignite artifacts, the rest is identical.


-

Denis

 

 

On Tue, Oct 15, 2019 at 8:23 AM sri hari kali charan Tummala <[hidden email]> wrote:

documentation is not covering step by step it's not that helpful.

 

On Tue, Oct 15, 2019 at 10:35 AM Ilya Kasnacheev <[hidden email]> wrote:

Hello!

 

Please refer to the docs:

 

 

Regards,

--

Ilya Kasnacheev

 

 

пн, 30 сент. 2019 г. в 17:57, sri hari kali charan Tummala <[hidden email]>:

thanks for replying but sorry not getting it please dumb it down, below are my questions.

 

Statically giving the public IP list in the IP configuration section in ignite config file.

Question:- wherein the config file should I give the IP address, imagine I launch 3 ec2 instances with public IP should I create a file in S3 bucket with file containing IP address of ec2 instance?

 

Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

Question:- ok I will give the s3 bucket name does the bucket need to have a file containing IP address or just empty bucket?

 

Thanks

Sri 

 

On Mon, Sep 30, 2019 at 1:28 AM Muhammed Favas <[hidden email]> wrote:

Hi Sri,

 

If your nodes created in EC2, it is very simple to start the ignite cluster. You have two option to configure the IP for all nodes auto discover the IP.

  1. Statically giving the public IP list in the IP configuration section in ignite config file.
  2. Use s3 bucket to configure the IP, and it will automatically discover the related nodes from s3 bucket.

 

I am using the 2nd method, and below is the configuration I have given for my cluster

 

<!-- Discover IP using Amazon s3. -->

        <property name="discoverySpi">

        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">

        <property name="ipFinder">

            <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">

                <property name="bucketName" value="yours3bucketname"/>

                <property name="awsCredentials">

                    <bean class="com.amazonaws.auth.BasicAWSCredentials">

                        <constructor-arg value="YOUR_ACCESS_KEY_ID"/>
                        <constructor-arg value="YOUR_SECRET_ACCESS_KEY"/>

                   </bean>

                </property>

            </bean>

        </property>

        </bean>

        </property>

 

 

Regards,

Favas 

 

From: sri hari kali charan Tummala <[hidden email]>
Sent: Friday, September 27, 2019 10:02 PM
To: [hidden email]
Subject: Does any one have working Ignite cluster on AWS

 

Hi All, 

 

can someone help me run a working ignite cuter on AWS , did anyone able to figure out steps required for setting up working ignite cluster on AWS.

 

Ignite documentation doesnt make sense it's just launching a docker instance on Ec2.

 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 


 

--

Thanks & Regards

Sri Tummala

 



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala



--
Thanks & Regards
Sri Tummala

<Screen Shot 2019-10-16 at 2.38.31 PM.png>




--
Thanks & Regards
Sri Tummala