fuzzy search examine lucene - Extending Umbraco and using the API

Owain Williams 481 posts 1413 karma points MVP 7x c-trib

Feb 17, 2017 @ 12:20

Fuzzy Search - Examine Lucene

I have been trying to get my search box to allow for Fuzzy search but I think I must be missing something somewhere. I've been following the tutorial from codeshare.co.uk on advanced searches and the search works as expected but now I am trying to extend this.

I thought it would be as simple as adding .Fuzzy(0.5f).Value to my SearchTerm but it doesn't seem to be the case.

The search continues to work but if I type 'tree' for example, I would expect to see results for 'trees'.

This is my controller code - which is idential to the codeshare examples other than the addition of a namespace and fuzzy to my HttpPost section where I've added Fuzzy.

    using System.Web.Mvc;
using Umbraco.Web.Mvc;
using wildsite.Models;
using Umbraco.Web;
using wildsite.Helpers;
using System.Collections.Generic;
using Examine.LuceneEngine.SearchCriteria;

namespace wildsite.Controllers
{
    public class SearchController : SurfaceController
    {
        #region Private Variables and Methods

        private SearchHelper _searchHelper { get { return new SearchHelper(new UmbracoHelper(UmbracoContext.Current)); } }

        private string PartialViewPath(string name)
        {
            return $"~/Views/Partials/Search/{name}.cshtml";
        }

        private List<SearchGroup> GetSearchGroups(SearchViewModel model)
        {
            List<SearchGroup> searchGroups = null;
            if (!string.IsNullOrEmpty(model.FieldPropertyAliases))
            {
                searchGroups = new List<SearchGroup>();
                searchGroups.Add(new SearchGroup(model.FieldPropertyAliases.Split(','), new string[] { model.SearchTerm }));
            }
            return searchGroups;
        }

        #endregion

        #region Controller Actions

        [HttpGet]
        public ActionResult RenderSearchForm(string query, string docTypeAliases, string fieldPropertyAliases, int pageSize, int pagingGroupSize)
        {
            SearchViewModel model = new SearchViewModel();
            if (!string.IsNullOrEmpty(query))
            {

                model.SearchTerm = query;
                model.DocTypeAliases = docTypeAliases;
                model.FieldPropertyAliases = fieldPropertyAliases;
                model.PageSize = pageSize;
                model.PagingGroupSize = pagingGroupSize;
                model.SearchGroups = GetSearchGroups(model);
                model.SearchResults = _searchHelper.GetSearchResults(model, Request.Form.AllKeys);
            }
            return PartialView(PartialViewPath("_SearchForm"), model);
        }



        [HttpPost]
        public ActionResult SubmitSearchForm(SearchViewModel model)
        {
            if (ModelState.IsValid)
            {
                if (!string.IsNullOrEmpty(model.SearchTerm))
                {
                    model.SearchTerm = model.SearchTerm.Fuzzy(0.1f).Value;
                    model.SearchGroups = GetSearchGroups(model);
                    model.SearchResults = _searchHelper.GetSearchResults(model, Request.Form.AllKeys);
                }
                return RenderSearchResults(model.SearchResults);
            }
            return null;
        }

        public ActionResult RenderSearchResults(SearchResultsModel model)
        {
            return PartialView(PartialViewPath("_SearchResults"), model);
        }

        #endregion

    }
}

Can anyone steer me in the right direction of where I've gone wrong?

Thanks,

Copy Link

Steven Harland 78 posts 518 karma points c-trib

Feb 17, 2017 @ 21:35

Hi Owain,

Here's one way to do it.

Modify your SearchGroup class so that SearchTerms becomes an array of IExamineValue instead of string:

using Examine.SearchCriteria;

namespace CodeShare.Library.Models
{
    public class SearchGroup
    {
        public string[] FieldsToSearchIn { get; set; }
        public IExamineValue[] SearchTerms { get; set; }

        public SearchGroup(string[] fieldsToSearchIn, IExamineValue[] searchTerms)
        {
            FieldsToSearchIn = fieldsToSearchIn;
            SearchTerms = searchTerms;
        }
    }
}

Then update your GetSearchGroups method as follows:

private List<SearchGroup> GetSearchGroups(SearchViewModel model)
{
    List<SearchGroup> searchGroups = null;
    if (!string.IsNullOrEmpty(model.FieldPropertyAliases))
    {
        searchGroups = new List<SearchGroup>();
        searchGroups.Add(new SearchGroup(model.FieldPropertyAliases.Split(','), new IExamineValue[] { model.SearchTerm.Fuzzy(0.5f) }));
    }
    return searchGroups;
}

Note the call to Fuzzy here which returns an IExamineValue.

Finally, you can remove this line:

model.SearchTerm = model.SearchTerm.Fuzzy(0.1f).Value;

Hope this helps.

Steven

Copy Link

Owain Williams 481 posts 1413 karma points MVP 7x c-trib

Feb 27, 2017 @ 11:33

Works perfectly - thanks for your help!

Copy Link

Edgar Rasquin 326 posts 925 karma points

Oct 17, 2019 @ 08:38

Hi Steven,

with my Search Term I got this working just fine.

My problem is to pass in multiple search words from a multi dropdown list:

enter image description here

They should be treated as "Or()"

Here is my code:

if (model.location != null)
        {
            IExamineValue[] stringValues = new IExamineValue[0];

            if (searchGroups == null)
            {
                searchGroups = new List<SearchGroup>();
            }

            foreach (string item in model.location)
            {
                searchGroups.Add(new SearchGroup(new string[] { "location" }, new[] { item.Boost(1f) }));
            }
        }

But this way its been treated as And() and therefore the search returns no results as soon as there are two locations checked.

Can you help me?

Copy Link

MuirisOG 382 posts 1284 karma points

Oct 17, 2019 @ 10:00

Hi Edgar,

here's a sample of our code, which (at the moment) is in a View.

First, set some variables (not all shown)...

var q = pSearchTerm;
var examineSearcherName = pExamineSearcherName;
var Searcher = ExamineManager.Instance.SearchProviderCollection[examineSearcherName];
var sc = Searcher.CreateSearchCriteria(Examine.SearchCriteria.BooleanOperation.Or);

Then do the search ...

//***
//* Get searchResults
//*
// exact phrase match - case sensitive, but include wildcard
// Note: p_categorylist is set to "No Search" on pages we want to exclude
var query = sc.GroupedOr(new[] { mainField, titleField }, q.MultipleCharacterWildcard()).Not().Field("p_categorylist", "No Search");
// split on words use fuzzy to make case-insensitive
foreach (var t in q.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries)) { query.Or().GroupedOr(new[] { mainField, titleField, metaKeywordsField, metaDescriptionField }, t.Fuzzy(0.8f)); }
// Note: 0.04f returns more partial matches than 0.05f
var searchResults = Searcher.Search(query.Compile()).OrderByDescending(x => x.Score).TakeWhile(x => x.Score > 0.04f);
//* End of "Get searchResults"

Copy Link

Edgar Rasquin 326 posts 925 karma points

Oct 17, 2019 @ 13:16

Hi MuirisOG,

thanks for your reply. Your approach is similar to my first implementation before including fuzzy search.

Before I changed

public string[] SearchTerms { get; set; }

into

public IExamineValue[] SearchTerms { get; set; }

I would just hand in the searchTerms as string[] arrays:

if (model.location != null)
        {
            string[] stringValues = new string[0];

            int dim = 0;
            foreach (string item in model.location)
            {
                dim++;
                Array.Resize(ref stringValues, dim);
                stringValues.SetValue(item, dim - 1);
            }

            if (searchGroups == null)
            {
                searchGroups = new List<SearchGroup>();
            }
            searchGroups.Add(new SearchGroup(new string[] { "location" }, stringValues ));
        }

But since I now need to place the value as an IExamineValue I don't know how to handle this anymore.

Copy Link

Stefano 61 posts 313 karma points c-trib

Feb 18, 2017 @ 10:46

I use this helper class to do fuzzy partial searches

public static class Examine
{
    /// <summary>
    ///     Builds and run the Lucene query.
    /// </summary>
    public static ISearchResults Search(string query)
    {
        var searcher = ExamineManager.Instance.SearchProviderCollection["ExternalSearcher"];
        var luceneQuery = new StringBuilder();

        if (!string.IsNullOrWhiteSpace(query))
        {
            // Split the words so that it will search for each with an OR. It's necessary because
            // name and surname are in two different fields.
            var searchTerms = query
                .Split(new[] {' '}, StringSplitOptions.RemoveEmptyEntries)
                .Select(QueryParser.Escape)
                .SelectMany(st => new[] {st + "*", st + "~0.6"});

            var byQuery = Search(searchTerms);
            luceneQuery.And(byQuery);
        }

        ISearchResults searchResults;
        // If no filters were selected search for nodes with ID 0 that will return no results.
        if (luceneQuery.Length > 0)
        {
            var rawQuery = searcher.CreateSearchCriteria().RawQuery(luceneQuery.ToString());
            searchResults = searcher.Search(rawQuery);
        }
        else
        {
            searchResults = searcher.Search(searcher.CreateSearchCriteria().Id(0).Compile());
        }

        return searchResults;
    }

    private static string Search(IEnumerable<string> queries)
    {
        return string.Join(" ", queries.Select(st => $"{st}"));
    }

    /// <summary>
    ///     Wraps the query with "+(" and ")" while adding it to the luceneQuery.
    /// </summary>
    private static void And(this StringBuilder luceneQuery, string query)
    {
        luceneQuery.Append("+(");
        luceneQuery.Append(query);
        luceneQuery.Append(")");
    }
}

Copy Link

Owain Williams 481 posts 1413 karma points MVP 7x c-trib

Feb 20, 2017 @ 15:26

Thanks! I've not had a chance to test this yet, hopefully tomorrow.

Appreciate the time you've taken though. Fingers crossed it's what I've been missing :)

Copy Link

sam woodcock 1 post 71 karma points

Jul 17, 2018 @ 13:00