Migrating from Oracle to Apache Ignite.

classic Classic list List threaded Threaded
7 messages Options
rizal123 rizal123
Reply | Threaded
Open this post in threaded view
|

Migrating from Oracle to Apache Ignite.

Hi,

I have a project/poc, about migrating database oracle into in memory apache
ignite.

First of all, this is my topology.
<http://apache-ignite-users.70518.x6.nabble.com/file/t1530/Topology_Draft_1.png>

in case image not showing: https://ibb.co/cbi5cR

I have done this thing:
1. Create node server cluster. And import schema oracle into it.
2. Load data from Oracle into server cluster using LoadCache.
3. From my application, change datasource into ignite cluster. (just only
one IP address). Currently i am using Jdbc Thin.
4. Start my application, and its Up. It`s running well.

I have the following problems:
1. JDBC Thin, does not support Transactional SQL. I really need this ticket
to be fixed.
2. Multi connection IP Address for JDBC Thin, or Load Balancer for JDBC
Thin.
3. Automatic fail over. I have tested 1 machine, with 3 node cluster server.
If the first node (That was first turn on) down, the connection will down
too. Though there are still 2 clusters that live.

Please let me know if there any solution...



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Andrew Mashenkov Andrew Mashenkov
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

Hi,

1. SQL will support transactions after next ticket will be resolved [1]. 
You can track it's state via "start watching this issue" link on issue page.

2. JDBC pooling, failover and load banacling is not supported for JDBC. 
More over JDBC connection is not thread safe. 
Feel free to create a ticket for features you need.

Most likely, c3pi0 can be used for connection polling once transaction support will be added [2].


On Wed, Jan 10, 2018 at 6:05 AM, rizal123 <[hidden email]> wrote:
Hi,

I have a project/poc, about migrating database oracle into in memory apache
ignite.

First of all, this is my topology.
<http://apache-ignite-users.70518.x6.nabble.com/file/t1530/Topology_Draft_1.png>

in case image not showing: https://ibb.co/cbi5cR

I have done this thing:
1. Create node server cluster. And import schema oracle into it.
2. Load data from Oracle into server cluster using LoadCache.
3. From my application, change datasource into ignite cluster. (just only
one IP address). Currently i am using Jdbc Thin.
4. Start my application, and its Up. It`s running well.

I have the following problems:
1. JDBC Thin, does not support Transactional SQL. I really need this ticket
to be fixed.
2. Multi connection IP Address for JDBC Thin, or Load Balancer for JDBC
Thin.
3. Automatic fail over. I have tested 1 machine, with 3 node cluster server.
If the first node (That was first turn on) down, the connection will down
too. Though there are still 2 clusters that live.

Please let me know if there any solution...



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/



--
Best regards,
Andrey V. Mashenkov
Regards,
Andrew.
rizal123 rizal123
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

Hi Andrew,

Thanks for your reply.

Hope the ticket will be on 2.4

Next question about replication. I have 3 node server with different
machine/ip. How ignite replicate/distribution data between them? Whereas my
application only access into one node.

Please let me know if there something I miss...

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Denis Magda-2 Denis Magda-2
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

The data will be distributed evenly among the node. You can read more on this here [1] or watch this video [2]

[2] https://www.youtube.com/watch?v=G22L2KW9gEQ


Denis

On Jan 10, 2018, at 5:18 PM, rizal123 <[hidden email]> wrote:

Hi Andrew,

Thanks for your reply.

Hope the ticket will be on 2.4

Next question about replication. I have 3 node server with different
machine/ip. How ignite replicate/distribution data between them? Whereas my
application only access into one node.

Please let me know if there something I miss...

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

rizal123 rizal123
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

Hi Denis,

My cluster node still cannot distribute the data.

Here is my experiment:
*My Client Apps*:
                    Client using JDBC Thin Ignite.
                    Connection conn =
DriverManager.getConnection("jdbc:ignite:thin://10.1.12.67");
                    And
                    Create PrepareStatement for Execute query "Insert ABC".

*Check 3 Node*:
                    Using Application DBeaver to connect to 3 Cluster Node
(10.1.12.67, 10.1.12.68, 10.1.12.69).
                    Data 'ABC' only stored at Cluster Node (10.1.12.67).
                    The other node didn`t get the data.

*Here is my Ignite Configuration, for 3 machine/IP Address (10.1.12.67,
10.1.12.68, 10.1.12.69)* :

                /IgniteConfiguration cfg = new IgniteConfiguration();
                cfg.setIgniteInstanceName("BrandCluster");
                cfg.setClientMode(false);

                TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
                ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..47510"));
               
                TcpDiscoverySpi discovery = new TcpDiscoverySpi();
                discovery.setLocalAddress("127.0.0.1");
                discovery.setLocalPort(47500);
                discovery.setIpFinder(ipFinder);
               
                cfg.setDiscoverySpi(discovery);/

Would you mind to show me IgniteConfiguration for multi cluster (different
ip address)?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ilya.kasnacheev ilya.kasnacheev
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

Hello!

> ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..47510"));
> discovery.setLocalAddress("127.0.0.1");

It seems that you don't have a cluster of 3 nodes, but rather 3 different localhost-only clusters. No wonder data doesn't get distributed.

I would expect something between the lines of:

                ipFinder.setAddresses(Arrays.asList("10.1.12.67:47500..47510", "10.1.12.68:47500..47510", "10.1.12.69:47500..47510"));

And I'd remove setLocalAddress() assignment.

Make sure that nodes report topology snapshot of 3 servers after they start up.

Regards,

--
Ilya Kasnacheev

2018-01-12 11:14 GMT+03:00 rizal123 <[hidden email]>:
Hi Denis,

My cluster node still cannot distribute the data.

Here is my experiment:
*My Client Apps*:
                    Client using JDBC Thin Ignite.
                    Connection conn =
DriverManager.getConnection("jdbc:ignite:thin://10.1.12.67");
                    And
                    Create PrepareStatement for Execute query "Insert ABC".

*Check 3 Node*:
                    Using Application DBeaver to connect to 3 Cluster Node
(10.1.12.67, 10.1.12.68, 10.1.12.69).
                    Data 'ABC' only stored at Cluster Node (10.1.12.67).
                    The other node didn`t get the data.

*Here is my Ignite Configuration, for 3 machine/IP Address (10.1.12.67,
10.1.12.68, 10.1.12.69)* :

                /IgniteConfiguration cfg = new IgniteConfiguration();
                cfg.setIgniteInstanceName("BrandCluster");
                cfg.setClientMode(false);

                TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
                ipFinder.setAddresses(Arrays.asList("127.0.0.1:47500..47510"));

                TcpDiscoverySpi discovery = new TcpDiscoverySpi();
                discovery.setLocalAddress("127.0.0.1");
                discovery.setLocalPort(47500);
                discovery.setIpFinder(ipFinder);

                cfg.setDiscoverySpi(discovery);/

Would you mind to show me IgniteConfiguration for multi cluster (different
ip address)?

rizal123 rizal123
Reply | Threaded
Open this post in threaded view
|

Re: Migrating from Oracle to Apache Ignite.

Hi Ilya,

Thanks for your reply.

It is solved now, our cluster has distributed the data.
And from Log Node, it is show 3 server.

"[09:45:33] Topology snapshot [ver=9, servers=1, clients=0, CPUs=8,
heap=0.77GB]
[09:47:15] Topology snapshot [ver=10, servers=2, clients=0, CPUs=12,
heap=1.8GB]
[09:47:44] Topology snapshot [ver=11, servers=3, clients=0, CPUs=12,
heap=2.8GB]"

Thanks Ilya.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/