lucene error after umrbaco update

ThomasHerckenrath 5 posts 25 karma points

Mar 01, 2021 @ 10:14

Hi community,

It seems like an Umbraco update broke our custom Lucene Search functionality on a website. I updated Umbraco from version 8.3.0 to 8.10.2, but got stuck in an error after this update.

The website is throwing me this error:

[ParseException: '*' or '?' not allowed as first character in WildcardQuery]
   Lucene.Net.QueryParsers.QueryParser.GetWildcardQuery(String field, String termStr) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:996
   Lucene.Net.QueryParsers.MultiFieldQueryParser.GetWildcardQuery(String field, String termStr) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\MultiFieldQueryParser.cs:202
   Lucene.Net.QueryParsers.QueryParser.Term(String field) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:1494
   Lucene.Net.QueryParsers.QueryParser.Clause(String field) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:1383
   Lucene.Net.QueryParsers.QueryParser.Query(String field) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:1301
   Lucene.Net.QueryParsers.QueryParser.Clause(String field) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:1387
   Lucene.Net.QueryParsers.QueryParser.Query(String field) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:1301
   Lucene.Net.QueryParsers.QueryParser.Parse(String query) in d:\Lucene.Net\FullRepo\trunk\src\core\QueryParser\QueryParser.cs:223

Could it be the case that this is a breaking and unknown bug in Umbraco? This is our custom searchmanager code which worked on version 8.3.0:

public class SearchManager
    {
        private const string INDEXNAME = "customIndex";
        private const string METAKEYWORDS = "keyword1902Seo"; 
        private const string MENUTITLE = "menuTitle";

        private readonly string[] _fields = { "pageDisplayName", "abstractNestedContentNestedContentTitle", "title1902Seo", "description1902Seo" };

        public IEnumerable<ISearchResult> GetSearchResults(string searchTerm, int limit = 500)
        {
            if (!string.IsNullOrEmpty(searchTerm) && ExamineManager.Instance.TryGetIndex(INDEXNAME, out var index))
            {
                var searcher = (BaseLuceneSearcher)index.GetSearcher();
                var query = searcher.CreateQuery("content", BooleanOperation.Or, searcher.LuceneAnalyzer, new LuceneSearchOptions { AllowLeadingWildcard = true });
                var searchQuery = CreateNativeSearchQuery(searchTerm);

                return query.NativeQuery(searchQuery).Execute(limit);
            }

            return new List<ISearchResult>();
        }

        private string CreateNativeSearchQuery(string searchTerm)
        {
            var wildCardTerms = $"*{searchTerm.Replace(" ", " * *")}*";

            var queryBuilder = new StringBuilder();
            queryBuilder.Append($"{METAKEYWORDS}:({wildCardTerms})^5");
            queryBuilder.Append($" {MENUTITLE}:({wildCardTerms})^3");

            foreach (var field in _fields)
            {
                queryBuilder.Append($" {field}:{wildCardTerms}");
            }

            return queryBuilder.ToString();
        }
    }

Copy Link

Warren Buckley 2106 posts 4836 karma points MVP 7x admin c-trib

Mar 01, 2021 @ 12:38

Hi Thomas 👋

Looking at the stack trace this is Lucene.NET library which Examine is using under the hood.

It seems like in a newer version of Lucene.NET you are unable to start or use * as the starting character,

I would try experimenting with your variable wildcardTerms to remove the beginning asterisks and compare with your current live site to see if it returns the same results as you got previously.

But if you do need the leading asterisks then a quick google has brought up this result which may help you.

https://github.com/castorini/Anserini/issues/61#issuecomment-222771120

Thanks,
Warren

Copy Link

ThomasHerckenrath 5 posts 25 karma points

Mar 09, 2021 @ 14:35

Hi Warren (or/and possibly others),

I looked into the possible solutions you provided, but nothing seems to work propperly. When I try to create a Query (in the above code at the line var query = searcher.CreateQuery(...);) I provide LuceneSearchOptions. When I look into this variable within my debugger, I see that the queryParser within the query variable does not allow leading wildcards (query -> queryParser -> allowLeadingWildcards = false) and I can not find any options to set this property to true. If I change the variable in the debugger, my code is working fine and leading wildcards are working, but I am unable to find the proper way to set this allowLeadingWildcards variable within the queryParser object of the query I create from the custom index. Any suggestions on this?

Copy Link

ThomasHerckenrath 5 posts 25 karma points

Mar 09, 2021 @ 15:19

Hi Again,

Managed to sort it out. The LuceneSearchOptions can be removed because it doesn't do anything. I changed to code to the snippet below and it works again (by casting the query to LuceneSearchQueryBase before setting the options):

if (!string.IsNullOrEmpty(searchTerm) && ExamineManager.Instance.TryGetIndex(INDEXNAME, out var index))
{
    var searcher = (BaseLuceneSearcher)index.GetSearcher();

    var query = searcher.CreateQuery("content", BooleanOperation.Or);

    ((LuceneSearchQueryBase)query).QueryParser.AllowLeadingWildcard = true;

    var searchQuery = CreateNativeSearchQuery(searchTerm);

    return query.NativeQuery(searchQuery).Execute(limit);
}

Copy Link

is working on a reply...

Flag this post as spam?

Lucene error after Umrbaco update