Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Simon Dingley 1470 posts 3427 karma points c-trib
    Mar 13, 2019 @ 13:07
    Simon Dingley
    0

    Problems Sorting Examine Search Results

    I'm trying to sort some Examine Search results and encountering issues.

    Here is my search method:

    public ISearchResults Search(string query, string[] doctypes, int[] parents, BooleanOperation type = BooleanOperation.Or)
    {
        var searchString = string.IsNullOrWhiteSpace(query) ? string.Empty : query.Trim();
        ISearchCriteria searchCriteria = this.searcher.CreateSearchCriteria(UmbracoExamine.IndexTypes.Content, type);
    
        string[] searchableFields = {"title", "summary", "metaKeywords", "metaDescription", "bodyText", "tags", "nodeName", "contentBlocks", "grid"};
    
        IExamineValue[] terms = searchString.Split(new [] { " " }, StringSplitOptions.RemoveEmptyEntries).Select(x => $"{x}".MultipleCharacterWildcard()).ToArray();
    
        if (!terms.HasAny()) return this.searcher.Search(searchString, false);
    
        var filter = searchCriteria.GroupedNot(new[] { "umbracoNaviHide", "hideFromSearchResults" }, "1");
    
        filter.And().GroupedOr(searchableFields, terms);
    
        // Make sure we do not return any content that doesn't have a template as it will not be accessible 
        filter.Not().Field("template", "0");
    
        // Add to the query any parent documents that results must have
        if (parents.HasAny())
        {
            filter.And().GroupedOr(new [] { "searchablePath" }, parents.Select(p =>p.ToString().MultipleCharacterWildcard()).ToArray());
        }
    
        // Add to the query any selected document types we want to filter on
        if (doctypes.HasAny())
        {
            filter.And().GroupedOr(new [] { "nodeTypeAlias" }, doctypes);
        }
    
        filter.And().OrderBy(new SortableField("nodeName"));
    
        return this.searcher.Search(searchCriteria);
    }
    

    To be able to sort by nodeName I know I need to add it to the ExamineIndex.config but that comes with its own problems such as once I add one user field I have to explicitly define all. In order to simplify that issue I am making it sortable by doing the following in my ApplicationStarting event handler.

    protected override void ApplicationStarting(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
    {
        var externalIndexer = ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"];
    
        var externalIndexerFields = externalIndexer.IndexerData.UserFields;
        var sortableFieldNames = new[] { "nodeName", "title" };
    
        foreach (var f in externalIndexerFields.Where(x => sortableFieldNames.Contains(x.Name)))
        {
            f.EnableSorting = true;
        }
    }
    

    …an idea borrowed from the Examine issue #92 discussion.

    However, sorting is still not working!

    Likely related is the fact that without the following line my results come back with scores but with it the score is NaN for each result:

    filter.And().OrderBy(new SortableField("nodeName"));
    

    I've also tried the following with no success either:

    filter.And().OrderBy(new SortableField("nodeName", SortType.String));
    

    I essentially need to give the user the ability to order results by name/title, score or updated date. I know I could do it in other ways but I lose the performance benefit of Examine so looking for help in trying to resolve this.

    Thanks, Simon

  • Erik Eelman 79 posts 319 karma points
    Mar 13, 2019 @ 13:57
    Erik Eelman
    0

    Hi Simon,

    Did you try to add the nodeName field in the <IndexAttributeFields> section with enable sorting: <add Name="nodeName" EnableSorting="true"/>

    Don't forget to rebuild your index after the changes.

  • Simon Dingley 1470 posts 3427 karma points c-trib
    Mar 13, 2019 @ 20:58
    Simon Dingley
    0

    Hi Erik,

    See my previous post. Yes, I could do that however by doing that it means I have to then explicitly define all other attributes/fields that need to be indexed as Shannon touches on briefly in his reply to the Github issue:

    To have a field marked as sortable, you have to declare it sortable in the config, but by doing that it means that your index now has to declare all of the fields it wants to index explicitly which isn't ideal.

    Thanks, Simon

  • Erik Eelman 79 posts 319 karma points
    Mar 13, 2019 @ 21:29
    Erik Eelman
    101

    Hi Simon,

    You can try to hook into the IndexerDocumentWriting method and register it in de ApplicationStarted event (like the solution AMV has posted in the github issue)

     protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
            {
                var indexer = (UmbracoContentIndexer)ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"];
                indexer.DocumentWriting += Indexer_DocumentWriting;
            }
    

    And in the method add something like this:

    private void Indexer_DocumentWriting(object sender, Examine.LuceneEngine.DocumentWritingEventArgs e)
            {
                var field = new Field("__Sort_nodeName",
                e.Fields["nodeName"],
                Field.Store.YES, Field.Index.NOT_ANALYZED);
                e.Document.Add(field);
            }
    
  • Simon Dingley 1470 posts 3427 karma points c-trib
    Mar 14, 2019 @ 09:39
    Simon Dingley
    0

    Morning Erik,

    I tried that yesterday and it seemed to have no effect, however, I've tried it again today but this time deleted the examine indexes first and have made some progress! It now sorts by nodeName and I've also added updateDate but I am missing the result scores still but at least this is a step in the right direction so thanks!

    In case it's of use to others here is my implementation so far:

    /// <summary>
    /// Overridable method to execute when All resolvers have been initialized but resolution is not frozen so they can be modified in this method
    /// </summary>
    /// <param name="umbracoApplication"></param>
    /// <param name="applicationContext"></param>
    protected override void ApplicationStarting(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
    {
        var externalIndexer = ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"];
        externalIndexer.GatheringNodeData += Bootstrapper_GatheringNodeData;
    
        ((UmbracoContentIndexer)externalIndexer).DocumentWriting += Bootstrapper_DocumentWriting;
    
        var externalIndexerFields = externalIndexer.IndexerData.UserFields;
        var sortableStringFieldNames = new[] { "nodeName", "title" };
        var sortableNumericFieldNames = new[] { "updateDate" };
    
        foreach (var f in externalIndexerFields.Where(x => sortableStringFieldNames.Contains(x.Name)))
        {
            f.EnableSorting = true;
        }
    
        foreach (var f in externalIndexerFields.Where(x => sortableNumericFieldNames.Contains(x.Name)))
        {
            f.EnableSorting = true;
            f.Type = "LONG";
        }
    }
    
    /// <summary>
    /// Handles the DocumentWriting event of the Bootstrapper.
    /// </summary>
    /// <param name="sender">The source of the event.</param>
    /// <param name="e">The <see cref="Examine.LuceneEngine.DocumentWritingEventArgs"/> instance containing the event data.</param>
    private void Bootstrapper_DocumentWriting(object sender, Examine.LuceneEngine.DocumentWritingEventArgs e)
    {
        // Add a new sortable field with the node name
        e.Document.Add(new Field("__Sort_nodeName", e.Fields["nodeName"], Field.Store.YES, Field.Index.NOT_ANALYZED));
    
        // If we can't parse the field as a date we can exit here
        if (!DateTime.TryParse(e.Fields["updateDate"], out var updateDate)) return;
    
        // Create a new NumericField for storing our sortable date
        var updateDateSortField = new NumericField("__Sort_updateDate", Field.Store.YES, true);
    
        // Convert the date/time to a long
        updateDateSortField.SetLongValue(long.Parse(updateDate.ToString("yyyyMMddHHmmssfff")));
    
        // Add our new sortable field
        e.Document.Add(updateDateSortField);
    }
    

    Then in my controller when I am performing the search I make sure to add the following to my query when sorting by the document title or name:

    filter.And().OrderBy("title", "nodeName");
    

    or, when sorting by updateDate I add the following:

    filter.And().OrderByDescending(new SortableField("updateDate", SortType.Long));
    

    Cheers, Simon

  • Simon Dingley 1470 posts 3427 karma points c-trib
    Mar 14, 2019 @ 09:49
    Simon Dingley
    0

    It seems the score is only missing when sorting by a custom field so perhaps that is expected behaviour? If so, I would be happy to mark your answer as the solution - thanks Erik.

  • Erik Eelman 79 posts 319 karma points
    Mar 14, 2019 @ 10:04
    Erik Eelman
    2

    Great to see that you found a working solution. I checked some course material about sorting and indexing in umbraco and i found this note:

    NB: When we add sort field relevance / score is appearing as NAN. This makes sense as we are sorting ourselves and not relying on Lucene score, hence either Examine or Lucene does not give you a score as it is no longer relevant.

    So it seems normal behaviour that there is no score available where you enable custom sorting.

  • Simon Dingley 1470 posts 3427 karma points c-trib
    Mar 14, 2019 @ 10:16
    Simon Dingley
    0

    As I suspected then - thanks again Erik.

  • Erik Eelman 79 posts 319 karma points
    Mar 14, 2019 @ 10:17
    Erik Eelman
    0

    You're welcome :)

Please Sign in or register to post replies

Write your reply to:

Draft