Trouble Enabling Query Indexing

classic Classic list List threaded Threaded
14 messages Options
leroyJr leroyJr
Reply | Threaded
Open this post in threaded view
|

Trouble Enabling Query Indexing

This post has NOT been accepted by the mailing list yet.
Hi, I'm using 1.0.0-incubating and I'm trying to set cache configuration to allow indexing so that I can run cache queries.

I'm starting ignite using the following:

val ign = Ignition.start("example-ignite.xml")

With the following property in the "ignite.cfg" bean:
       <property name="cacheConfiguration">
            <list>
               
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="1"/>
                    <property name="queryIndexEnabled" value="true"/>
                </bean>
            </list>
        </property>

and receive the following error:

org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context [springUrl=file:/Users/mike.matney/sandbox/cacheloader/example-ignite.xml, err=Error creating bean with name 'ignite.cfg' defined in URL [file:/Users/mike.matney/sandbox/cacheloader/example-ignite.xml]: Cannot create inner bean 'org.apache.ignite.configuration.CacheConfiguration#788c8575' of type [org.apache.ignite.configuration.CacheConfiguration] while setting bean property 'cacheConfiguration' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.configuration.CacheConfiguration#788c8575' defined in URL [file:/Users/mike.matney/sandbox/cacheloader/example-ignite.xml]: Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'queryIndexEnabled' of bean class [org.apache.ignite.configuration.CacheConfiguration]: Bean property 'queryIndexEnabled' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?]
  at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:660)
  at org.apache.ignite.Ignition.start(Ignition.java:334)
  ... 43 elided

Is this still a valid method for enabling indexing in the cache configuration?  If not can you please point me in the right direction for enabling query indexing.

Thanks and Cheers,
Mike
Abhishek M Abhishek M
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

This post has NOT been accepted by the mailing list yet.
Hi,

Dont need to set <property name="queryIndexEnabled" value="true"/>. Remove it from config.

When you are creating cache. just put the Index type. like as below:

CacheConfiguration<Long, MyClass> cfg = new CacheConfiguration<Long, MyClass>();
cfg.setIndexedTypes(Long.class, MyClass.class);
cache = ignite.getOrCreateCache(cfg);
leroyJr leroyJr
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

Thanks Abhishek,  I was hoping that would do the case too.  Despite setting the index types like you suggested, and then creating the cache, I still get the following error when attempting to query (this is the same error I received before setting the index types too.)

javax.cache.CacheException: Indexing is disabled for cache: TestCache
  at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.validate(IgniteCacheProxy.java:480)
  at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:428)
  ... 43 elided

Below is the scala code I've used to demonstrate the indexing issue I've encountered.  Thanks again for the response.

import org.apache.ignite.configuration._
import org.apache.ignite.cache._
import org.apache.ignite._

import scala.collection.JavaConversions._

val ign = Ignition.start("example-ignite.xml")

// Query Annotation Fields
import org.apache.ignite.cache.query._
import org.apache.ignite.cache.query.annotations._

case class Foo(
    @QuerySqlField(index=true)
    id: Int,

    @QuerySqlField(index=true)
    value: Int,

    @QuerySqlField
    name: String
    )

val fooCacheCfg = new CacheConfiguration[Long,Foo]()
fooCacheCfg.setName("TestCache")
fooCacheCfg.setCacheMode(CacheMode.PARTITIONED)
fooCacheCfg.setIndexedTypes(Long.getClass, Foo.getClass)

val fCache = ign.getOrCreateCache[Long, Foo](fooCacheCfg)

fCache.put(123, Foo(id=1, value=1000, name="bob"))

val q1 = fCache.query(new SqlQuery(Foo.getClass.getName, "id > 0")).getAll()

On Mon, May 25, 2015 at 1:14 AM, Abhishek M [via Apache Ignite Users] <[hidden email]> wrote:
Hi,

Dont need to set <property name="queryIndexEnabled" value="true"/>. Remove it from config.

When you are creating cache. just put the Index type. like as below:

CacheConfiguration<Long, MyClass> cfg = new CacheConfiguration<Long, MyClass>();
cfg.setIndexedTypes(Long.class, MyClass.class);
cache = ignite.getOrCreateCache(cfg);


If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/Trouble-Enabling-Query-Indexing-tp377p389.html
To unsubscribe from Trouble Enabling Query Indexing, click here.
NAML



--
Mike Matney
Verona, WI

"Welcome to the 21st century, where each passing day becomes more like a Philip K. Dick novel."

sergi sergi
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by leroyJr
Please look at CacheConfiguration.setIndexedTypes method.
sergi sergi
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

By the way in Scala you should use @ScalarCacheQuerySqlField annotation. Ignite has ScalarCacheQueryExample in examples to show how it works.
fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by leroyJr
Hey Leroy,

I'm having the same issue with the latest release. Did you ever find a solution or this?

Thanks,
Florian
dsetrakyan dsetrakyan
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

fluffy wrote
I'm having the same issue with the latest release. Did you ever find a solution or this?
Forian, does the solution provided in this thread with @ScacheCacheQuerySqlField annotation work for you?

D.
leroyJr leroyJr
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by fluffy
Fluffy,

We ended up using the @ScalarCacheQuerySqlField annotation that dsetrakyan suggested. Additionally we had limited success depending on the key type being indexed.

Take a look at http://apache-ignite-users.70518.x6.nabble.com/SQL-query-question-tc426.html

When we used Scala's Int as a key it would fail. If we used Java Integer it worked.  I believe the Ignite folks will be addressing this in an upcoming release.
fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by dsetrakyan
D.,

our entire source is in java and I am no Scala expert. Is there a Java equivalent?

One other thing to note, this problem seemed to start occuring after I changed my maven compile to not package with all dependencies. What I find funny is that the 'ignite-indexing' package doesn't actually provide any source files when you are just using the '@QuerySqlField' and basic Indexing functions.

I saw another thread that said this type of problem may be solved by including the 'ignite-indexing' dependency?

Unfortunately I can't package everything in a single jar due to some other issues...
fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by dsetrakyan
I have a very very simple example. And based on everything I know and see in the documentation, indexing should work on this with no problems. Here's the code. I'm also attaching the Spring Config File.

Any help would be much appreciated....

----------------------

Source Code

----------------------

packagegrid.test;

import org.apache.ignite.*;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.logger.log4j.Log4JLogger;

import java.io.Serializable;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.UUID;

public class TestCacheConfig {

    public static void main(String[] args) throws IgniteException, InterruptedException {

        Ignite ignite;
        IgniteLogger log = new Log4JLogger();

        boolean rungrid=true;

        try {

            log.debug("STARTING GRID....");
            ignite = Ignition.start("<grid-cache.xml");
            ignite.configuration().setNetworkTimeout(5000);
            ignite.configuration().setClientMode(false);

           CacheConfiguration<UUID, BasicGridPoint> pointCfg = new CacheConfiguration<>("Points");

            pointCfg.setName("Points");
            pointCfg.setIndexedTypes(UUID.class, BasicGridPoint.class);

            IgniteCache<String, BasicGridPoint> pointCache = ignite.cache("Points");
            pointCache.put("dsadsa-dsadsa-dsasd",new BasicGridPoint());

            while (rungrid) {

                try {
                    log.debug("-------------------------------");
                    log.debug(" # of Nodes: " + ignite.cluster().nodes().size());//+" | Grid Size: "+ g.cache(CACHE_NAME).keySet().size());
                    log.debug(" Local Node ID: " + ignite.cluster().localNode().id());
                    log.debug(" Global Point Cache Keys: " + pointCache.size());
                    log.debug("-------------------------------");


                    SqlFieldsQuery PointTypeQuery = new SqlFieldsQuery("select distinct Type from BasicGridPoint");
                    try (QueryCursor<List<?>> cursor = pointCache.query(PointTypeQuery)) {
                        Collection<List<?>> PointTypesRaw = cursor.getAll();
                        Iterator PointCountIter = PointTypesRaw.iterator();
                        while (PointCountIter.hasNext()) {
                            List tmpList = (List) PointCountIter.next();
                            Iterator ListIter = tmpList.iterator();
                            while (ListIter.hasNext()) {
                                String type = (String) ListIter.next();
                                System.out.print("Type: " + type);
                            }
                        }
                    }

                    Thread.sleep(10000);
                } catch (Exception e) {
                    log.debug("Loop - Caught:" + e.toString());
                    for (int i=0; i<e.getStackTrace().length; i++)
                        log.debug(e.getStackTrace()[i].toString());
                    Thread.sleep(10000);
                }
            }

            //Stopping Grid and associated subprocesses
            log.debug("Shutting things down...");

              Ignition.stop(true);


        } catch (Exception e) {
            log.debug("Main - Caught:" + e.toString());
        }

    }
    public static class BasicGridPoint implements Serializable {

        @QuerySqlField
        private String Type;
        @QuerySqlField
        private String Ident;


        public String getType() {
            return Type;
        }

        public void setType(String pointType) {
            Type = pointType;
        }

        public String getIdent() {
            return Ident;
        }

        public void setIdent(String ident) {
            Ident = ident;
        }
    }

}

----------------------------

Spring config

---------------------------

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

         
        <property name="peerClassLoadingEnabled" value="false"/>

        <property name="marshaller">
            <bean class="org.apache.ignite.marshaller.optimized.OptimizedMarshaller">
               
                <property name="requireSerializable" value="false"/>
            </bean>
        </property>

       
        <property name="cacheConfiguration">
               
                <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="Points"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="0"/>
                    <property name="offHeapMaxMemory" value="0"/>
                    <property name="startSize" value="3000000"/>
                </bean>
  </list>
        </property>
       
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                     <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                 <value>127.0.0.1:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

    </bean>

</beans>

vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

Fluffy,

You have a cache configuration in both code and XML. The one in XML is actually used, but it doesn't provide indexed types. Your code configuration looks correct, so I think you should simply remove it from XML.

-Val

On Fri, Jun 12, 2015 at 1:39 PM, fluffy <[hidden email]> wrote:
I have a very very simple example. And based on everything I know and see in
the documentation, indexing should work on this with no problems. Here's the
code. I'm also attaching the Spring Config File.

Any help would be much appreciated....

----------------------

Source Code

----------------------
/packagegrid.test;

import org.apache.ignite.*;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.logger.log4j.Log4JLogger;

import java.io.Serializable;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.UUID;

public class TestCacheConfig {

    public static void main(String[] args) throws IgniteException,
InterruptedException {

        Ignite ignite;
        IgniteLogger log = new Log4JLogger();

        boolean rungrid=true;

        try {

            log.debug("STARTING GRID....");
            ignite = Ignition.start("<grid-cache.xml&quot;);
            ignite.configuration().setNetworkTimeout(5000);
            ignite.configuration().setClientMode(false);

           CacheConfiguration&lt;UUID, BasicGridPoint> pointCfg = new
CacheConfiguration<>("Points");

            pointCfg.setName("Points");
            pointCfg.setIndexedTypes(UUID.class, BasicGridPoint.class);

            IgniteCache<String, BasicGridPoint> pointCache =
ignite.cache("Points");
            pointCache.put("dsadsa-dsadsa-dsasd",new BasicGridPoint());

            while (rungrid) {

                try {
                    log.debug("-------------------------------");
                    log.debug(" # of Nodes: " +
ignite.cluster().nodes().size());//+" | Grid Size: "+
g.cache(CACHE_NAME).keySet().size());
                    log.debug(" Local Node ID: " +
ignite.cluster().localNode().id());
                    log.debug(" Global Point Cache Keys: " +
pointCache.size());
                    log.debug("-------------------------------");


                    SqlFieldsQuery PointTypeQuery = new
SqlFieldsQuery("select distinct Type from BasicGridPoint");
                    try (QueryCursor<List&lt;?>> cursor =
pointCache.query(PointTypeQuery)) {
                        Collection<List&lt;?>> PointTypesRaw =
cursor.getAll();
                        Iterator PointCountIter = PointTypesRaw.iterator();
                        while (PointCountIter.hasNext()) {
                            List tmpList = (List) PointCountIter.next();
                            Iterator ListIter = tmpList.iterator();
                            while (ListIter.hasNext()) {
                                String type = (String) ListIter.next();
                                System.out.print("Type: " + type);
                            }
                        }
                    }

                    Thread.sleep(10000);
                } catch (Exception e) {
                    log.debug("Loop - Caught:" + e.toString());
                    for (int i=0; i<e.getStackTrace().length; i++)
                        log.debug(e.getStackTrace()[i].toString());
                    Thread.sleep(10000);
                }
            }

            //Stopping Grid and associated subprocesses
            log.debug(&quot;Shutting things down...&quot;);

              Ignition.stop(true);


        } catch (Exception e) {
            log.debug(&quot;Main - Caught:&quot; + e.toString());
        }

    }
    public static class BasicGridPoint implements Serializable {

        @QuerySqlField
        private String Type;
        @QuerySqlField
        private String Ident;


        public String getType() {
            return Type;
        }

        public void setType(String pointType) {
            Type = pointType;
        }

        public String getIdent() {
            return Ident;
        }

        public void setIdent(String ident) {
            Ident = ident;
        }
    }

}

----------------------------

Spring config

---------------------------

&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">


        <property name="peerClassLoadingEnabled" value="false"/>

        <property name="marshaller">
            <bean
class="org.apache.ignite.marshaller.optimized.OptimizedMarshaller">

                <property name="requireSerializable" value="false"/>
            </bean>
        </property>


        <property name="cacheConfiguration">

                <list>
                <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="Points"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="atomicityMode" value="ATOMIC"/>
                    <property name="backups" value="0"/>
                    <property name="offHeapMaxMemory" value="0"/>
                    <property name="startSize" value="3000000"/>
                </bean>
                </list>
        </property>

        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                     <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                 <value>127.0.0.1:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

    </bean>

</beans>
/




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Trouble-Enabling-Query-Indexing-tp377p481.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

Val,

I removed the Cache Config in the XML file and still don't get indexing. Here's the error I see int he startup logs:

[10:33:26,574][WARN ][main][GridIndexingManager] Indexing is disabled (to enable please configure GridIndexingSpi).
...
[10:33:31,121][WARN ][main][root] Logging at DEBUG level without checking if DEBUG level is enabled: Loop - Caught:javax.cache.CacheException: Indexing is disabled for cache: Points
[10:33:31,121][WARN ][main][root] Logging at DEBUG level without checking if DEBUG level is enabled: org.apache.ignite.internal.processors.cache.IgniteCacheProxy.validate(IgniteCacheProxy.java:545)
[10:33:31,121][WARN ][main][root] Logging at DEBUG level without checking if DEBUG level is enabled: org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:493)
[10:33:31,121][WARN ][main][root] Logging at DEBUG level without checking if DEBUG level is enabled: com.gometrics.grid.test.TestCacheConfig.main(TestCacheConfig.java:53)



and here's my spring config now... Almost nothing in it:


<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">

         
        <property name="peerClassLoadingEnabled" value="false"/>

              <property name="marshaller">
                   <bean class="org.apache.ignite.marshaller.optimized.OptimizedMarshaller">
                        <property name="requireSerializable" value="false"/>
                   </bean>
               </property>   

   
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                     
                   
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                               
                                <value>127.0.0.1:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

    </bean>

</beans>



Any ideas?

Thanks much,
Florian
fluffy fluffy
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

In reply to this post by vkulichenko
Val,

never mind, I think that did it. I had to make a couple of other changes in my environment, but that I think allowed indexing to work again.

I never realized that the Spring XML definition and the java DSL of a Cache are mutually exclusive? From GridGain I was used to specifying the basics in Spring XML and then just modifying it in Java when necessary....

The documentation says that XMl and Java are mutually exclusive for the Index MetaData stuff, but in terms of CacheConfiguration, does it pick a Spring XML configuration over a Java configuration and then ignore Java?

Thanks much
vkulichenko vkulichenko
Reply | Threaded
Open this post in threaded view
|

Re: Trouble Enabling Query Indexing

Fluffy,

Cache is created only once and with only one version of config (it can be taken declared in XML or in code - doesn't matter). And in case there is a cache configuration in initial Ignite config, it will be created on node startup, so all other attempts to create a cache with the same name dynamically after node is already started will fail. In case you use getOrCreateCache method with modified configuration, it will get existing cache instead of creating a new one, therefore this configuration will be ignored.

-Val

On Sat, Jun 13, 2015 at 4:59 AM, fluffy <[hidden email]> wrote:
Val,

never mind, I think that did it. I had to make a couple of other changes in
my environment, but that I think allowed indexing to work again.

I never realized that the Spring XML definition and the java DSL of a Cache
are mutually exclusive? From GridGain I was used to specifying the basics in
Spring XML and then just modifying it in Java when necessary....

The documentation says that XMl and Java are mutually exclusive for the
Index MetaData stuff, but in terms of CacheConfiguration, does it pick a
Spring XML configuration over a Java configuration and then ignore Java?

Thanks much




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Trouble-Enabling-Query-Indexing-tp377p490.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.