Kevin,

I had similar issue see this post http://our.umbraco.org/forum/developers/extending-umbraco/11819-Examine-range-query-numeric you may need to pad out the data when inserting into the index.

Regards

Ismail

Thanks Ismail, I did see this post but it wasn't apparently obvious where / how the padding takes places. Is there any chance you could give me an example in relation to the code I have posted above.

All the best 

Kevin

You will need to implement gatheringnode index and create another field in the index with the padded value and then search on that value my code looks like

public ExamineEvents()

        {

            //Add event handler for 'GatheringNodeData' on our 'WOI'

            ExamineManager.Instance.IndexProviderCollection[WoiIndexer].GatheringNodeData

                += ExamineEvents_GatheringNodeData;

 

        }

 

        /// <summary>

        /// Event handler for GatheringNodeIndex.

        /// This will fire everytime Examine is creating/updating an index for an item

        /// </summary>

        /// <param name="sender"></param>

        /// <param name="e"></param>

        void ExamineEvents_GatheringNodeData(object sender, IndexingNodeDataEventArgs e)

        {

            //check if this is 'Content' (as opposed to media, etc...)

            if (e.IndexType == IndexTypes.Content)

            {

                if (e.Node.UmbNodeTypeAlias() == ProductAlias)

                {

                    //if this is a 'product' page look at ultimate picker params 

                    var node = new Node(e.NodeId);

                    foreach (var ultimateAlias in _ultimatePickerAliases) {

                        var propertyValue = node.GetPropertyValue(ultimateAlias);

                        if(propertyValue!=string.Empty)

                        {

                            if(propertyValue.Contains(","))

                            {

                                string[] propValues = propertyValue.Split(new[] {','});

                                var sbConcatValue=new StringBuilder();

                                foreach (var propValue in propValues)

                                {

                                    sbConcatValue.Append(GetFieldValue(e, propValue, ultimateAlias));

                                    sbConcatValue.Append(",");

                                }

 

                                e.Fields.Add("__" + ultimateAlias, sbConcatValue.ToString().TrimEnd(','));

 

                            }

                            else

                            {

                                e.Fields.Add("__" + ultimateAlias,GetFieldValue(e, propertyValue, ultimateAlias));

                            }

                        }

                    }

 

                }

                AddToContentsField(e);

            }

 

        }

 

        private void AddToContentsField(IndexingNodeDataEventArgs e)

        {

            Dictionary<string, string> fields = e.Fields;

            var combinedFields = new StringBuilder();

            foreach (KeyValuePair<string, string> keyValuePair in fields)

            {              

                combinedFields.AppendLine(keyValuePair.Value);              

            }

            e.Fields.Add("contents",combinedFields.ToString());

        }

 

        /// <summary>

        /// get ultimate picker field acutal value not the id of target 

        /// </summary>

        /// <param name="e"></param>

        /// <param name="propertyValue"></param>

        /// <param name="luceneFieldAlias"></param>

        /// <returns></returns>

        private string GetFieldValue(IndexingNodeDataEventArgs e, string propertyValue,string luceneFieldAlias)

        {

            int nodeId = 0;

            int.TryParse(propertyValue, out nodeId);

 

                var n= new Node(nodeId);

                //node does not exist but we have numeric value

                if(n.Id!=0){

                    if (NumericProperties.Contains(luceneFieldAlias))

                    {

                        //have to pad out to get lucene range queries to work

                        int i = 0;

                        int.TryParse(n.Name, out i);

 

                        return i.ToString("D6");

                    }

                    else{

                        return n.Name;

                    }

                }

                return nodeId.ToString("D6");         

 

        }

 

the padding happens in GetFieldValue i have a list of properties already set which contain numeric values that need to be padded out. Dont forget to update your search to search on the padded properties.

Regards

Ismail

 

Hi Ismail,

Many thanks once again for your reply,

This solution appears to work in returning properties with values between £100,000 and £200,000 and for Properties with values between £200,000 and £300,000, however for properties with values of £0 to £100,000 I am getting no results at all when I have two properties at values of £80,000 and £76,000 in this range and fully indexed?

Do you have any ideas why this might be? 

Thanks

Kevin

I would test your index using luke and try querying also write out the query that examine is generating and test that in luke. Also what analyser are you using in your examine config if none is provided it defaults to standard if you use whitespace analyser that can give different results on my index i use standard.

Regards

Ismail

 

I am using the Standard Analyser in the Indexer settings. The following screen grabs highlight in yellow a stripped to the bones query showing it working with 100,00+ values but not working with sub £100,000 values. I'm slightly confused by this.

The sub value 80000 should be in the index as 080000 what format are you using in the GatheringNodeEvent I am using ToString("D6");         

Regards

Ismail

My code looks as follows, unsure if If using best practice in places but kind of doing this by trial and error at the moment, I have added a screen grab also to show that when I use '080000' I still get no results:

 

var query = sc.NodeTypeAlias("PropertyDetail");

            if (Location != String.Empty)
            {
                query.And().Field("location", Location);
            }
            if (Type != String.Empty)
            {
                query.And().Field("type", Type);
            }
            if (Parking != String.Empty)
            {
                query.And().Field("parking", Parking);
            }
            if (Ensuite != String.Empty)
            {
                query.And().Field("ensuite", Ensuite);
            }

            if (PriceLower != String.Empty && PriceHigher != String.Empty)
            {
                var paddedLower = int.Parse(PriceLower).ToString("D6");
                var paddedHigher = int.Parse(PriceHigher).ToString("D6");

                query = query.And().Range("price", paddedLower, paddedHigher, true, true);
            }

            var fields = new List<string>();
            var values = new List<string>();

            //Build up number of bedrooms
            if (!String.IsNullOrEmpty(Studio))
            {
                fields.Add("numberOfBedrooms");
                values.Add("0");
            }

            if (!String.IsNullOrEmpty(OneBedroom))
            {
                fields.Add("numberOfBedrooms");
                values.Add("1");
            }

            if (!String.IsNullOrEmpty(TwoBedroom))
            {
                fields.Add("numberOfBedrooms");
                values.Add("2");
            }

            if (!String.IsNullOrEmpty(ThreeBedroom))
            {
                fields.Add("numberOfBedrooms");
                values.Add("3");
            }

            if (!String.IsNullOrEmpty(FourBedroom))
            {
                fields.Add("numberOfBedrooms");
                fields.Add("numberOfBedrooms");
                fields.Add("numberOfBedrooms");
                fields.Add("numberOfBedrooms");
                values.Add("4");
                values.Add("5");
                values.Add("6");
                values.Add("7");
            }

            if (fields.Count != 0 && values.Count != 0)
            {
                var theFields = (fields.ToArray());
                var theValues = (values.ToArray());
                query.And().GroupedOr(theFields, theValues);
            }

            query.And().OrderBy(orderByFields);

            var compliedQuery = query.Compile();
            SearchResults = ExamineManager.Instance.Search(compliedQuery);

            SearchResultListing.DataSource = SearchResults;
            SearchResultListing.DataBind()
 

Kevin,

You are missing the GatheringNode code you need to get the value in as D6 as well as querying if you look at my code i have implemented GatheringNode event so that when an indexing event occurs i am injecting in new field which has the value as D6 then i am querying on that.  So in my index i have field called participants which has values like 10,11 ,12 etc however i also have a field called __participants this has the padded value e.g. 000010 and I am querying on that. So in your case you need to get values  for price in new field call it __price with the padded value then query on that.

Regards

Ismail

Ok Ismail, sorry I have completely missed that. Where abouts in my project do I add the event to inject the new padded field? Can this just be in my SearchResults User Control?

In your project create a class 

using Examine;

using UmbracoExamine;

public class ExamineEvents:ApplicationBase

   {

     public ExamineEvents()

        {

            //Add event handler for 'GatheringNodeData' 

            ExamineManager.Instance.IndexProviderCollection["yourindexhere"].GatheringNodeData

                += ExamineEvents_GatheringNodeData;

 

        }

   }

then add code to inject the fields see my original code posted earlier. The class inherits from ApplicationBase so its hooked up on site load. Also take a look at demo project code that also shows you how to do it http://www.farmcode.org/post/2010/07/01/Examine-demo-site-source-code-from-CodeGarden-2010.aspx

Regards

Ismail

Thanks Ismail, one thing and then I think I'll nearly be there, In terms of the code you sent I'm having issues with e.node.UmbNodeTypeAlias() and node.GetPropertyValue("price") resolving in my class and therefore cannot build and test etc. What other namespaces do I need?

The GetPropertyValue is an extension in a static class looks like

 public static string GetPropertyValue(this Node node, string alias)

        {

            return getPropertyValue(node,alias);

        }

        private static string getPropertyValue(Node node,string alias)

        {

            Property property = node.GetProperty(alias);

            return property == null ? string.Empty : property.Value;

        }

 

The other method is extension method and found in UmbracoExamine.LinqXmlExtensions

Ok cannot find UmbracoExamine.LinqXmlExtensions how do I go about using this? Is this not part of the standard UmbracoExamine.dll

Apologies for the number of questions but I guess thats what this place is for and I need to get this working ASAP. I'm sure its not far off now.

Cheers again

Kevin

Kevin,

In my class i have the following using statements 

 

using System.Collections.Generic;
using umbraco.BusinessLogic;
using Examine;
using UmbracoExamine;
using umbraco.presentation.nodeFactory;
using System.Text;

also i am using the latest version of examine from source. What version are you using?

Regards

Ismail

 

Ok I think the latest version of Examine has sorted that now. You mention that if my ExamineEvents() Class inherits from application base it should be triggered as soon as umbraco is launched however it doesnt seem to be kicking in, so no padding of values can take place and inserted into the index. Do I have to do anything else for this or call this explicitly?  

Thanks again.

 

UPDATE: Dont' worry about this, it is firing now so I will update shortly on progress. Cheers

did you rebuild the index after deploying your code? As you will need to rebuild so that the values get injected in. you can use http://our.umbraco.org/projects/backoffice-extensions/examine-index-admin to rebuild.

Regards

Ismail

Hi Ismail

I have regenerated the Index and I have stepped through the code and I can see the code is adding the padding as required at the following point

e.Fields.Add("__" + "price", GetFieldValue(e, propertyValue, "price"));

However, this new padded value __price doesnt appear in the index when I query it using LUKE after this code has run. Am I missing a final step?

Screenshot below shows the padded value:

kevin,

In luke on the first page do you see under fields the field __price ? 

Regards

Ismail

No, the field is not being added - See Image below: 

The code I have in my ExamineEvents Class is now as follows:

using System;
using System.Collections.Generic;
using umbraco.BusinessLogic;
using Examine;
using UmbracoExamine;
using umbraco.presentation.nodeFactory;
using System.Text;


namespace Client.classes
{
    public class ExamineEvents : ApplicationBase
    {
        public ExamineEvents()
        {

            //Add event handler for 'GatheringNodeData' 
            ExamineManager.Instance.IndexProviderCollection["DemoIndexer"].GatheringNodeData += new EventHandler<IndexingNodeDataEventArgs>(ExamineEvents_GatheringNodeData);

        }

        void ExamineEvents_GatheringNodeData(object sender, IndexingNodeDataEventArgs e)
        {
            umbraco.NodeFactory.Node theNode = new umbraco.NodeFactory.Node(e.NodeId);

            //check if this is 'Content' (as opposed to media, etc...)
            if (e.IndexType == IndexTypes.Content)
            {

                if (theNode.NodeTypeAlias == "PropertyDetail")
                {
                    var node = new Node(e.NodeId);

                    var propertyValue = ExamineEventsHelper.GetPropertyValue(theNode, "price");

                    if (propertyValue != string.Empty)
                    {

                        e.Fields.Add("__" + "price", GetFieldValue(e, propertyValue, "price"));

                    }

                }

                AddToContentsField(e);

            }


        }

        private void AddToContentsField(IndexingNodeDataEventArgs e)
        {

            Dictionary<string, string> fields = e.Fields;

            var combinedFields = new StringBuilder();

            foreach (KeyValuePair<string, string> keyValuePair in fields)
            {

                combinedFields.AppendLine(keyValuePair.Value);

            }

            e.Fields.Add("contents", combinedFields.ToString());

        }



        private string GetFieldValue(IndexingNodeDataEventArgs e, string propertyValue, string luceneFieldAlias)
        {

            int nodeId = 0;

            int.TryParse(propertyValue, out nodeId);

            var n = new Node(nodeId);

            //node does not exist but we have numeric value

            if (n.Id != 0)
            {

                //have to pad out to get lucene range queries to work

                int i = 0;

                int.TryParse(n.Name, out i);

                return i.ToString("D6");

            }

            return nodeId.ToString("D6");

        }

    }
}

The following helper class contains the static methods:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using umbraco.presentation.nodeFactory;

namespace Client.classes
{
    public static class ExamineEventsHelper
    {
        public static string GetPropertyValue(umbraco.NodeFactory.Node node, string alias)
        {
            return getPropertyValue(node, alias);
        }

        private static string getPropertyValue(umbraco.NodeFactory.Node node, string alias)
        {
            var property = (umbraco.NodeFactory.Property)node.GetProperty(alias);
            return property == null ? string.Empty : property.Value;
        }

    }
}

I'm hoping that when this is done it will proved a good reference point for other people, once again many thanks. Kevin

Can you paste your examineindex.config file contents. main point of interest is tag IndexUserFields need to see if that is empty which means all fields get indexed and is all good or you have set the fields manually therefore you also need to set your __price field.

Regards

Ismail

Hi Ismail, as requested. I believe this should be indexing all properties on the document type.

ExamineIndex.config as per tutorial videos:

  <?xml version="1.0"?>
<!-- 
Umbraco examine is an extensible indexer and search engine.
This configuration file can be extended to create your own index sets.
Index/Search providers can be defined in the UmbracoSettings.config

More information and documentation can be found on CodePlex: http://umbracoexamine.codeplex.com
-->
<ExamineLuceneIndexSets>
    <!-- The internal index set used by Umbraco back-office - DO NOT REMOVE -->
    <IndexSet SetName="InternalIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/Internal/">
      <IndexAttributeFields>
        <add Name="id" />
        <add Name="nodeName" />
        <add Name="updateDate" />
        <add Name="writerName" />
        <add Name="path" />
        <add Name="nodeTypeAlias" />
        <add Name="parentID" />
      </IndexAttributeFields>
      <IndexUserFields />
      <IncludeNodeTypes/>
      <ExcludeNodeTypes />
    </IndexSet>

    <!-- The internal index set used by Umbraco back-office for indexing members - DO NOT REMOVE -->
    <IndexSet SetName="InternalMemberIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/InternalMember/">
        <IndexAttributeFields>
            <add Name="id" />
            <add Name="nodeName"/>
            <add Name="updateDate" />
            <add Name="writerName" />
            <add Name="loginName" />
            <add Name="email" />
            <add Name="nodeTypeAlias" />
        </IndexAttributeFields>
        <IndexUserFields/>
        <IncludeNodeTypes/>
        <ExcludeNodeTypes />
    </IndexSet>

  <IndexSet SetName="DemoIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/DemoIndex/">
    <IndexAttributeFields>
      <add Name="id" />
      <add Name="nodeName"/>
      <add Name="nodeTypeAlias" />
      <add Name="parentID" />
    </IndexAttributeFields>
    <IndexUserFields/>
    <IncludeNodeTypes/>
    <ExcludeNodeTypes />
  </IndexSet>

</ExamineLuceneIndexSets>

ExamineSettings.config as per tutorial videos

<?xml version="1.0"?>
<!-- 
Umbraco examine is an extensible indexer and search engine.
This configuration file can be extended to add your own search/index providers.
Index sets can be defined in the ExamineIndex.config if you're using the standard provider model.

More information and documentation can be found on CodePlex: http://umbracoexamine.codeplex.com
-->
<Examine>
  <ExamineIndexProviders>
    <providers>
      <add name="InternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
           supportUnpublished="true"
           supportProtected="true"
           interval="10"
           analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"/>

      <add name="InternalMemberIndexer" type="UmbracoExamine.UmbracoMemberIndexer, UmbracoExamine"
           supportUnpublished="true"
           supportProtected="true"
           interval="10"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"/>

      <add name="DemoIndexer" type="UmbracoExamine.LuceneExamineIndexer, UmbracoExamine"
              runAsync="true"
              supportUnpublished="false"
              supportProtected="true"
              interval="10"
              analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
        indexSet="DemoIndexSet" />
    </providers>
  </ExamineIndexProviders>

  <ExamineSearchProviders defaultProvider="InternalSearcher">
    <providers>
      <add name="InternalSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
           analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"/>

      <add name="InternalMemberSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" enableLeadingWildcards="true"/>

      <add name="DemoSearcher" type="UmbracoExamine.LuceneExamineSearcher, UmbracoExamine"
            analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" 
              indexSet="DemoIndexSet"/>
    </providers>
  </ExamineSearchProviders>

</Examine>

 

ExamineEvents reference to above settings

ExamineManager.Instance.IndexProviderCollection["DemoIndexer"].GatheringNodeData += new EventHandler<IndexingNodeDataEventArgs>(ExamineEvents_GatheringNodeData);

 

Kevin,

I am now stumped as that field should now be in the index.  Can you try turning of your app pool. Then delete the index folder for DemoIndex then restart the app pool then rebuild the index it may be getting itself in a twist somewhere so this should give us complete new clean index.

Regards

Ismail

Ok I am stumped also, I have deleted the Index and re indexed, no joy. To be sure that I was adding the _price to the correct index aIso added a new property 'KW Test' with a value of 'testing' to the 'PropertyDetail' document type to ensure that this appears in the indexing Node fields in the code and it did. The _price fields gets added to the Indexing Node fields in the code but does not show up in LUKE. So frustrating close to the finishing line, I must have missed a vital something in terms of persisting the value in the index?

kevin,

one more thing we can try.  again delete the index.  This time from the root node republish the whole site including child pages, i dont mean republish xml cache but the whole site see if that fixes the problem.  Examine generates the index from the xml cache file it may be that the field is not in there (we also have memory cache and it may be there hence you see the field in templates etc) other than this i cant think of anything else.

Regards

Ismail

Ok I tried this and still no joy so I started to inspect the generated index files, I can see the padded values in there but no sign of the __price field, so out of curiosity I changed the field name that was being added to the index to 'kevin'....and BINGO! The padded value appears in the index and is visible in LUKE. Now I have no idea why this is the case but the field name of __price was causing the issue. I can now query the padded values at last. I can't believe it has taken this journey to achieve this functionlity in my app but nevertheless I want to thank you IMMENSELY for your help over the past few days and I hope this thread helps other people out who are trying to achieve a similar range search functionality.

Thanks again and all the best

Kev

Hi Ismail...once again....

At the risk of almost going insane with this. Now querying the Index from within the website is yielding no results using Examine. The generated lucene query appears to be correct and indeed when I use this query within LUKE the correct results are returned. Any ideas on why this might not be working internally within the site?

{ SearchIndexType: , LuceneQuery: +__NodeTypeAlias:propertydetail +paddedprice:[0 TO 300000] }

in luke which analyser are you using standard or whitespace? Also in the c# code where you build the query pad out the 0 see if that makes a difference. It may be luke is by default padding out?

Regards

Ismail

Was just trying a few different things and found that the default provider was not set to my DemoIndex and as soon as I changed it to this then the results came through. Next onto sorting the results!!! Thanks for your help once again.

 <ExamineSearchProviders defaultProvider="DemoSearcher">
    <providers>
      <add name="InternalSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
           analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"/>

      <add name="InternalMemberSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" enableLeadingWildcards="true"/>

      <add name="DemoSearcher" type="UmbracoExamine.LuceneExamineSearcher, UmbracoExamine"
            analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" 
              indexSet="DemoIndexSet"/>
    </providers>
  </ExamineSearchProviders>

is working on a reply...