Future of IGFS.

classic Classic list List threaded Threaded
3 messages Options
Chris Software Chris Software
Reply | Threaded
Open this post in threaded view
|

Future of IGFS.

Hello, 

I have heard a rumor that IGFS will be dropped in the future.  Can anyone confirm or deny, and tell me why or why not?  If there is a discussion thread somewhere I would love a link.  

Source:  alamar on StackOverflow

Thank you. 
Chris Software Chris Software
Reply | Threaded
Open this post in threaded view
|

Re: Future of IGFS.

Hello! 

Would this question be better suited to the contributor's distro?  For one thing, I have a couple of patches I would like to submit if IGFS has a future.

Thank you.

On Sun, May 12, 2019 at 2:54 PM Chris Software <[hidden email]> wrote:
Hello, 

I have heard a rumor that IGFS will be dropped in the future.  Can anyone confirm or deny, and tell me why or why not?  If there is a discussion thread somewhere I would love a link.  

Source:  alamar on StackOverflow

Thank you. 
dmagda dmagda
Reply | Threaded
Open this post in threaded view
|

Re: Future of IGFS.

Hello Chris,

IGFS was developed with the thinking that it would be a solution for Hadoop acceleration. However, it was discovered in practice that the performance benefits it gives are insignificant for production deployments. For instance, based on my experience IGFS combined with Hadoop Accelerator might show 20%, 30% sometimes 2x or even zero performance acceleration. Plus, it required notable integration efforts.

IGFS failed to show *consistent* order of magnitude performance gains since it's not enough just to put data in RAM. Your RAM-based storage has to be tightly coupled with APIs used by the applications. With IGFS the storage was Ignite while the APIs were developed by Hive, Impala, Pig, MapReduce, etc.

That's why for Hadoop offloading use cases and real-time analytics deploy Ignite in one of its standard configurations - Ignite with native persistence enabled. Use Ignite SQL, compute grid, ML for the data located in Ignite and Hadoop frameworks for HDFS data sets. Consider Spark as a generic API that can go and merge data stored in both databases. More details are in this discussion:

-
Denis


On Tue, May 14, 2019 at 12:13 PM Chris Software <[hidden email]> wrote:
Hello! 

Would this question be better suited to the contributor's distro?  For one thing, I have a couple of patches I would like to submit if IGFS has a future.

Thank you.

On Sun, May 12, 2019 at 2:54 PM Chris Software <[hidden email]> wrote:
Hello, 

I have heard a rumor that IGFS will be dropped in the future.  Can anyone confirm or deny, and tell me why or why not?  If there is a discussion thread somewhere I would love a link.  

Source:  alamar on StackOverflow

Thank you.