Trying to get MariaDB data loaded into Ignite through a web console created config

classic Classic list List threaded Threaded
9 messages Options
Steven Castano Steven Castano
Reply | Threaded
Open this post in threaded view
|

Trying to get MariaDB data loaded into Ignite through a web console created config

So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]



dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]



Steven Castano Steven Castano
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]




Denis Magda Denis Magda
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]




Steven Castano Steven Castano
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Create a sample app how? In java or something? Is there no way to get the instance to load the caches when it starts?

I’m not a big java coder at all, I do most of my stuff in Python… is there a simple way in python to connect to the cluster and issue the load caches command? Or maybe an already made Java app that someone has out there that I could use as an example?

*SMC

Steven Castano | [hidden email]



On Jun 19, 2019, at 12:43 AM, Denis Magda <[hidden email]> wrote:

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]





dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Cache loading can be triggered from the application side only at the moment. It's unsupported for Python yet.

The easiest way here is to make LoadCaches.java workable in your Docker environment. Share an exception generated by the class. You said it failing there.

-
Denis


On Tue, Jun 18, 2019 at 9:56 PM Steven Castano <[hidden email]> wrote:
Create a sample app how? In java or something? Is there no way to get the instance to load the caches when it starts?

I’m not a big java coder at all, I do most of my stuff in Python… is there a simple way in python to connect to the cluster and issue the load caches command? Or maybe an already made Java app that someone has out there that I could use as an example?

*SMC

Steven Castano | [hidden email]



On Jun 19, 2019, at 12:43 AM, Denis Magda <[hidden email]> wrote:

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]





Steven Castano Steven Castano
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

It’s not failing I guess... I thought it was supposed to auto populate when the cluster started up. Then I was just trying to run the class file through java, but that wasn’t working. I was hoping there was some sort of command line option to put the data in there. 

I’m trying to research how to quickly write a java app that will load them, but I can’t seem to find any instructions or examples. 

Sent from my iPhone

On Jun 19, 2019, at 1:05 AM, Denis Magda <[hidden email]> wrote:

Cache loading can be triggered from the application side only at the moment. It's unsupported for Python yet.

The easiest way here is to make LoadCaches.java workable in your Docker environment. Share an exception generated by the class. You said it failing there.

-
Denis


On Tue, Jun 18, 2019 at 9:56 PM Steven Castano <[hidden email]> wrote:
Create a sample app how? In java or something? Is there no way to get the instance to load the caches when it starts?

I’m not a big java coder at all, I do most of my stuff in Python… is there a simple way in python to connect to the cluster and issue the load caches command? Or maybe an already made Java app that someone has out there that I could use as an example?

*SMC

Steven Castano | [hidden email]



On Jun 19, 2019, at 12:43 AM, Denis Magda <[hidden email]> wrote:

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]





ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Hello!

Unfortunately we don't seem to have any support to do this from any tooling.

You will have to execute something like this in your Docker container:
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;

public class Preloader {
public static void main(String[] args) {
Ignition.setClientMode(true);
try (Ignite ignite = Ignition.start("/path/to/ignite.xml")) {
ignite.cache(args[0]).loadCache(null);
}
}
}
you should compile it against ignite libs and execute on your docker.

Regards,
--
Ilya Kasnacheev


ср, 19 июн. 2019 г. в 08:20, Steven Castano <[hidden email]>:
It’s not failing I guess... I thought it was supposed to auto populate when the cluster started up. Then I was just trying to run the class file through java, but that wasn’t working. I was hoping there was some sort of command line option to put the data in there. 

I’m trying to research how to quickly write a java app that will load them, but I can’t seem to find any instructions or examples. 

Sent from my iPhone

On Jun 19, 2019, at 1:05 AM, Denis Magda <[hidden email]> wrote:

Cache loading can be triggered from the application side only at the moment. It's unsupported for Python yet.

The easiest way here is to make LoadCaches.java workable in your Docker environment. Share an exception generated by the class. You said it failing there.

-
Denis


On Tue, Jun 18, 2019 at 9:56 PM Steven Castano <[hidden email]> wrote:
Create a sample app how? In java or something? Is there no way to get the instance to load the caches when it starts?

I’m not a big java coder at all, I do most of my stuff in Python… is there a simple way in python to connect to the cluster and issue the load caches command? Or maybe an already made Java app that someone has out there that I could use as an example?

*SMC

Steven Castano | [hidden email]



On Jun 19, 2019, at 12:43 AM, Denis Magda <[hidden email]> wrote:

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]





ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Trying to get MariaDB data loaded into Ignite through a web console created config

Hello again!

Forgot to say, it takes cache name as 1st arg.

Regards,
--
Ilya Kasnacheev


вт, 2 июл. 2019 г. в 13:38, Ilya Kasnacheev <[hidden email]>:
Hello!

Unfortunately we don't seem to have any support to do this from any tooling.

You will have to execute something like this in your Docker container:
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;

public class Preloader {
public static void main(String[] args) {
Ignition.setClientMode(true);
try (Ignite ignite = Ignition.start("/path/to/ignite.xml")) {
ignite.cache(args[0]).loadCache(null);
}
}
}
you should compile it against ignite libs and execute on your docker.

Regards,
--
Ilya Kasnacheev


ср, 19 июн. 2019 г. в 08:20, Steven Castano <[hidden email]>:
It’s not failing I guess... I thought it was supposed to auto populate when the cluster started up. Then I was just trying to run the class file through java, but that wasn’t working. I was hoping there was some sort of command line option to put the data in there. 

I’m trying to research how to quickly write a java app that will load them, but I can’t seem to find any instructions or examples. 

Sent from my iPhone

On Jun 19, 2019, at 1:05 AM, Denis Magda <[hidden email]> wrote:

Cache loading can be triggered from the application side only at the moment. It's unsupported for Python yet.

The easiest way here is to make LoadCaches.java workable in your Docker environment. Share an exception generated by the class. You said it failing there.

-
Denis


On Tue, Jun 18, 2019 at 9:56 PM Steven Castano <[hidden email]> wrote:
Create a sample app how? In java or something? Is there no way to get the instance to load the caches when it starts?

I’m not a big java coder at all, I do most of my stuff in Python… is there a simple way in python to connect to the cluster and issue the load caches command? Or maybe an already made Java app that someone has out there that I could use as an example?

*SMC

Steven Castano | [hidden email]



On Jun 19, 2019, at 12:43 AM, Denis Magda <[hidden email]> wrote:

Steven,

Create a sample app based on LoadCaches and deploy it in Docker. It should work out.

--
Denis Magda


On Fri, Jun 14, 2019 at 5:17 PM Steven Castano <[hidden email]> wrote:
Thanks… that got me much farther… I was able to generate a config, get the docker image built with the Dockerfile and deploy a running container which now connects back to the web console. It shows all of my caches created for each of the tables I wanted to import, but I can’t seem to figure out how to load the data. the LoadCaches.java file doesn’t want to run inside the docker container. So I appear to have all my caches, but they’re all empty. The screencast only shows how to run it in an IDE, not on the command line once it’s built.

*SMC

Steven Castano | [hidden email]



On Jun 14, 2019, at 6:38 PM, Denis Magda <[hidden email]> wrote:

Hi Steven,

Please check out a series of screencast "Ignite Web Console - Automatic RDBMS Integration" that shows how to use to complete this process:

It might be the case that with docker you need to pass the settings differently. Please, check docker configuration settings:

-
Denis


On Thu, Jun 13, 2019 at 1:44 PM Steven Castano <[hidden email]> wrote:
So I was able to get a 2 node cluster started, v2.7.5 and got the web-console running through a docker container. I’m even able to load the MariaDB JDBC driver and connect to my database.

The problem I’m having is that when I’m done configuring things and I save and download the .zip file for the cluster configuration… I don’t know where to put it. No matter where I store it on my server I keep getting a “file not found” type error when looking for the properties file.

Even when creating a cluster with all the defaults not even connected to another database with no persistence, I still can’t get the downloaded project files working. Is there something I’m missing, or maybe a good tutorial online I can follow??

*SMC

Steven Castano | [hidden email]