Running Ignite Cluster using Docker containers

classic Classic list List threaded Threaded
10 messages Options
vitalys vitalys
Reply | Threaded
Open this post in threaded view
|

Running Ignite Cluster using Docker containers

Hi,

I am trying to run Ignite Cluster using Docker containers. I have Docker for
Desktop running on my Windows box and I downloaded Apache Ignite 2.7.0 image
from Docker Hub. I am able to start Ignite server using docker command :

<source>docker run --rm --name myignite -p 47500:47500 -p 47501:47501 -p
10800:10800 -it apacheignite/ignite</source>

I am trying to connect to the Server Node from my client application and
here I hit a snag. How do I connect to the server? I opened an image and
went to the "config" folder and I found "default-config.xml" there with no
"IpFinder" section. What ports am I suppose to connect to from the client?
What hostname do I need to specify on the client side, "localhost"?  

I looked for the documentation in Apache Ignite tutorial but there are very
few things there designate to this topic. Is there any example or some kind
of tutorial on how to run Ignite Cluster using Docker containers?

Thank you in advance.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Hello,

 

With default configuration Ignite uses MulticastIpFinder that will try to scan for available Ignite instances within your network automatically.

The default required ports are: 47500 and 47100 for discovery and communications and localhost(127.0.0.1) is the default alias for docker machine.

 

I would suggest you to set port forwarding for 47100 as well.

With that default configuration you can connect from you local machine to Docker at 127.0.0.1:47500 for thick and 127.0.0.1:10800 for thin clients.

 

You can change default configuration with CONFIG_URL parameter [1] that could point to file:///my-config.xml as well. [2]

 

You can also try to replace the whole “config” folder with your own configuration files:

docker run -v /myconfigFolder:/opt/ignite/apache-ignite/config ….

 

[1] - https://apacheignite.readme.io/docs/docker-deployment

[2] - https://stackoverflow.com/questions/54714911/run-ignite-docker-with-custom-config-file

 

From: [hidden email]
Sent: Wednesday, July 31, 2019 5:12 PM
To: [hidden email]
Subject: Running Ignite Cluster using Docker containers

 

Hi,

 

I am trying to run Ignite Cluster using Docker containers. I have Docker for

Desktop running on my Windows box and I downloaded Apache Ignite 2.7.0 image

from Docker Hub. I am able to start Ignite server using docker command :

 

<source>docker run --rm --name myignite -p 47500:47500 -p 47501:47501 -p

10800:10800 -it apacheignite/ignite</source>

 

I am trying to connect to the Server Node from my client application and

here I hit a snag. How do I connect to the server? I opened an image and

went to the "config" folder and I found "default-config.xml" there with no

"IpFinder" section. What ports am I suppose to connect to from the client?

What hostname do I need to specify on the client side, "localhost"? 

 

I looked for the documentation in Apache Ignite tutorial but there are very

few things there designate to this topic. Is there any example or some kind

of tutorial on how to run Ignite Cluster using Docker containers?

 

Thank you in advance.

 

 

 

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Alex Shapkin
vitalys vitalys
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

This post was updated on .
I am getting different issue now. I configured my Ignite Server to use Unicast :

<property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">                 
   <property name="ipFinder">                         
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">                                   
                                 
                                  <property name="addresses">
                                        <list>
                                          <value>localhost:47500..47504</value>                                       
                                        </list>
                                  </property>
                </bean>
          </property>
        </bean>
</property>       

When I start my first Docker Container it looks good and I am able to connect to the Ignite Server using my client code.

 docker run --rm --name myignite1 -p 47500:47500 -p 47501:47501 -p 10800:10800 -it myignite/ignite:1.0


However when I am trying to bring up another server Node :

 docker run --rm --name myignite2 -p 47502:47502 -p 47503:47503 -p 10801:10800 -it myignite/ignite:1.0


it's not forming a Cluster with the previously started Node.

If I run my Ignite Docker image like that :

 docker run --rm --net=host --name myignite2 -p 47502:47502 -p 47503:47503 -p 10801:10800 -it myignite/ignite:1.0

 then Cluster is created and I can see it if I open "ignitevisor" console. However Client application is unable to join the Cluster then
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Hello,

 

Please, refer to the docker port forwarding documentation [1]

In other words, instead of  -p 47503:47503 you need to write -p 47503:47500,

unless you have not configured the default discovery spi port to 47503 explicitly.

 

Please, recall that Ignite requires a minimum set of open ports for a general communication: TcpDiscoverySpi port - 47500 and TcpCommunicationSpi port - 47100.

 

If you want to enable a thin clients communication, then 10800 default port is required as well.

 

To summarize, your docker cmd should look like:

docker run --rm --name myignite1 -p 47503:47500 -p 47103:47100 -p 10803:10800 -it ignite-image

 

Now your host machine should be able to connect to the container with 47103, 47503, 10803

 

[1] - https://docs.docker.com/config/containers/container-networking/#published-ports

 

 

From: [hidden email]
Sent: Wednesday, August 7, 2019 6:11 PM
To: [hidden email]
Subject: RE: Running Ignite Cluster using Docker containers

 

I am getting different issue now. I configured my Ignite Server to use Unicast : 

<property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">                  
   <property name="ipFinder">                          
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">                                    
                                  
                                  <property name="addresses">
                                        <list>
                                          <value>localhost:47500..47504</value>                                        
                                        </list>
                                  </property>
                </bean>
          </property>
        </bean>
</property>        

When I start my first Docker Container it looks good and I am able to connect to the Ignite Server using my client code. 

 docker run --rm --name myignite1 -p 47500:47500 -p 47501:47501 -p 10800:10800 -it myignite/ignite:1.0 


However when I am trying to bring up another server Node : 

 docker run --rm --name myignite2 -p 47502:47502 -p 47503:47503 -p 10801:10800 -it myignite/ignite:1.0 


it's not forming a Cluster with the previously started Node. 

If I run my Ignite Docker image like that : 

 docker run --rm --net=host --name myignite2 -p 47502:47502 -p 47503:47503 -p 10801:10800 -it myignite/ignite:1.0 

 then Cluster is created and I can see it if I open "ignitevisor" console. However Client application is unable to join the Cluster then 
-- 
Sent from: 
http://apache-ignite-users.70518.x6.nabble.com/

Alex Shapkin
vitalys vitalys
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Here is the problem. I can start one Server Node in the Docker container and
I can connect to that Node from my client application. However When I run
another Docker Containers they are not joining the Cluster. Here is an
example :

Node 1 :

docker run --rm --name myignite1 -p 47501:47500 -p 47101:47100 -p
10801:10800 -it myignite/ignite:1.0

[17:50:48] Topology snapshot [ver=1, locNode=d290d944, servers=1, clients=0,
state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]

Node 2 :
docker run --rm --name myignite3 -p 47503:47500 -p 47103:47100 -p
10803:10800 -it myignite/ignite:1.0

[17:54:22] Topology snapshot [ver=1, locNode=fe47c5c7, servers=1, clients=0,
state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]

________________________________________________________________________________

They Nodes can't discover one another. When I am starting my Nodes there is
no such issue since they are using the same Configuration where
TcpDiscoveryIpVmFimder is defined :

<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">  
                                 
  <property name="addresses">
        <list>
          <value>localhost:47500..47504</value>                                        
      </list>
  </property>
</bean>

So my question is : how do I make multiple nodes running in different Docker
Containers discover each other?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Alexandr Shapkin Alexandr Shapkin
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Most likely you need to get them aware of each other.

I.e. configure DiscoverySpi addresses range to 49500..49509.

 

Please, share you Ignite config for containerized nodes.

 

Alexandr

 

From: [hidden email]
Sent: Thursday, August 8, 2019 9:00 PM
To: [hidden email]
Subject: RE: Running Ignite Cluster using Docker containers

 

Here is the problem. I can start one Server Node in the Docker container and

I can connect to that Node from my client application. However When I run

another Docker Containers they are not joining the Cluster. Here is an

example :

 

Node 1 :

 

docker run --rm --name myignite1 -p 47501:47500 -p 47101:47100 -p

10801:10800 -it myignite/ignite:1.0

 

[17:50:48] Topology snapshot [ver=1, locNode=d290d944, servers=1, clients=0,

state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]

 

Node 2 :

docker run --rm --name myignite3 -p 47503:47500 -p 47103:47100 -p

10803:10800 -it myignite/ignite:1.0

 

[17:54:22] Topology snapshot [ver=1, locNode=fe47c5c7, servers=1, clients=0,

state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]

 

________________________________________________________________________________

 

They Nodes can't discover one another. When I am starting my Nodes there is

no such issue since they are using the same Configuration where

TcpDiscoveryIpVmFimder is defined :

 

<bean

class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">                                                               

                                                                  

  <property name="addresses">

                <list>

                  <value>localhost:47500..47504</value>                                       

      </list>

  </property>

</bean>

 

So my question is : how do I make multiple nodes running in different Docker

Containers discover each other?

 

 

 

--

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

 

Alex Shapkin
vitalys vitalys
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

<property name="discoverySpi">
   <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
     <property name="ipFinder">  
        <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">  
                                 
          <property name="addresses">
                <list>
                  <value>localhost:47500..47504</value>                                        
                </list>
          </property>
       </bean>
    </property>
  </bean>
</property>



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Niels Ejrnæs Niels Ejrnæs
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Hello

We're testing out Ignite and are running them in Docker as well. It's a bit tricky to get to work.
Keep in mind that referencing localhost inside a docker container is pointing them against their internal address. Not the host address of your machine. So that's why they aren't discovering eachother.

There are some options I can suggest:
  1. Create a docker network and attach your containers to this network. Then you can add them to your addresses list by their container name. This is what we are currently doing.
  2. Run them with --network host. Then you can use "localhost" as this will now be your host machine. In my opinion it kinda defeats the point of running it on Docker, but this is where I have had the best results.

Best regards
Niels Elkjær Ejrnæs
Software Developer R&D






-----Original Message-----
From: vitalys <[hidden email]>
Sent: 8. august 2019 20:22
To: [hidden email]
Subject: RE: Running Ignite Cluster using Docker containers

<property name="discoverySpi">
   <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
     <property name="ipFinder">  
        <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">  
                                 
          <property name="addresses">
                <list>
                  <value>localhost:47500..47504</value>                                        
                </list>
          </property>
       </bean>
    </property>
  </bean>
</property>



--
Sent from: https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&amp;data=02%7C01%7CNiels.Ejrnaes%40enghouse.com%7C30bb112adffb4afedc3a08d71c2d558e%7C427e40023c0240489e280eba58b331f4%7C1%7C1%7C637008853347767016&amp;sdata=42KDcTU9%2FS%2ButBA7qRkPgeW%2BKo1zfP1N1zo4bmXgzBc%3D&amp;reserved=0
vitalys vitalys
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

Thank you.

Actually I switched from Unicast to Multicast and it seems like it's working
fine. I was able to connect to the Cluster from my Client application,
however still having issues using JDBC connection.

It's a bit frustrating since Ignite recommends to run the Nodes in the
containers but they don't provide examples or quality documentation.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Niels Ejrnæs Niels Ejrnæs
Reply | Threaded
Open this post in threaded view
|

RE: Running Ignite Cluster using Docker containers

I ran Multicast in the beginning as well, but the overlay network in Docker Swarm does not support Multicast. So I had to drop it and use the TcpDiscoveryVmIpFinder. Our containers register themselves in a shared key-value database (etcd) upon startup and remove themselves on shutdown. We then populate the addresses from this registry runtime. Works so far.

Best regards
Niels Elkjær Ejrnæs
Software Developer R&D