Managing apache-ignite under Kubernetes

classic Classic list List threaded Threaded
3 messages Options
Miguel Díaz Miguel Díaz
Reply | Threaded
Open this post in threaded view
|

Managing apache-ignite under Kubernetes

Hi all

I'm installing a persistence apache-ignite in Kubernetes using the Chart available in the official web. After some troubleshooting 2 nodes of ignite are running and some SQL DBs were created with their caches.

The method used to discover nodes is based in auto-discovery of Kubernetes.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="peerClassLoadingEnabled" value="false"/>
        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                        <property name="namespace" value="default"/>
                        <property name="serviceName" value="my-ignite-service"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>

The problem I see is when I try to load the Visor and open the configuration. If I use the same config file that is used by ignite.sh the Visor can not connect to the Grid (topology is empty). I have to create a different configuration file specifying the IP (fortunately my kubernetes has only a node: 127.0.0.1). Next configuration is ok for a Kuebernes with only a node and visor connects to the grid.

<property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                                <value>127.0.0.1:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

More info: I see the Kubernets libraries are loaded in both process (inginte.sh and visor) but visor doesnt apply the kubernetes configuration. No error appears. The Kurbernetes auto-discovery is desirable to install Ignite in Kubernetes production environments.

Did you experience this issue? any workaround?
Humphrey Humphrey
Reply | Threaded
Open this post in threaded view
|

Re: Managing apache-ignite under Kubernetes

This post was updated on .
We did the same, creating a pod with the following configuration:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                       
                        <property name="namespace" value="#{systemEnvironment['PROJECT_NAME']}"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>


And then we point to the visor command (startvisor.sh) where we pass the
config file:

cd <IGNITE_HOME>/bin
./ignitevisorcmd.sh -cfg=config/visor-config.xml

In the environment, we are passing the project name, I think in your case
you're using default namespace, so you can put there default. We have
packaged all the dependend jars and put it in a lib folder.


When the container starts it needs to keep running and then we enter the
console and start up visor from there. When we done we can close visor and
stop the pod.

Humphrey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Miguel Díaz Miguel Díaz
Reply | Threaded
Open this post in threaded view
|

Re: Managing apache-ignite under Kubernetes

Thanks Humphrey, yes, finally I migrate the test to a multinode kubernetes with several pods and the same configuration worked fine. If the visor is started on a pod (inside the grid) the configuration adobe works as the Visor only needs to be connected to one component inside the grid. What I dont have clear is why TcpDiscoveryKubernetesIpFinder doesnt work with Visor ... but anyway, the cluster is running.

El jue., 14 nov. 2019 a las 21:47, Humphrey (<[hidden email]>) escribió:
We did the same, creating a pod with the following configuration:


And then we point to the visor command (startvisor.sh) where we pass the
config file:


In the environment, we are passing the project name, I think in your case
you're using default namespace, so you can put there default. We have
packaged all the dependend jars and put it in a lib folder.

When the container starts it needs to keep running and then we enter the
console and start up visor from there. When we done we can close visor and
stop the pod.

Humphrey



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/