While performing a faceted query the where clause of the query performs an In expression.
var productSkus = products.Select(p => p.Sku).Distinct(); var query = UCommerce.Api.SearchLibrary.FacetedQuery(); var results = query.Where(x => x.DisplayOnSite && x.Sku.In(productSkus)); return results;
If more that 1024 sku's exist then Lucene.Net.Search.BooleanQuery throws TooManyClauses Exception (As each sku is essentially or'd)
Lucene exposes a setting to allow this value to be overridden, BooleanQuery.SetMaxClauseCount(int value)
However in our testing this doesn't seem to be honoured and if more than 1024 clauses are appended then it still throws. (Perhaps it is reset somewhere)
Does anyone have any pointers as to a solution or a workaround? (Assuming that reducing the number of results returned by the query is not acceptable). Note, additional facets are added to the query
I'm not sure as to why the max clause count is not overridden properly but as a workaround you could group the sku's per 1024 (or less), create multiple queries and combine the result sets afterwards. It's not ideal but it shouldn't cause much overhead when querying a local index.
The result of that function is then later composed with other criteria, if I chunk it then then I'll have to combine it in memory. I think I'll have to do some rewriting to bring the later composed criteria forward to so then can be evaulated before alongside each chunk. You're idea will definitely work though, it will just unfortunately need more changes in the calling code than I'd like but hey ho :-)
Not sure there is going to be a neat answer to this one.:-)
The thing is. There's two different assemblies in the runtime environment since Umbraco also uses Lucene.net. Now this doesn't necessarily cause two different assemblies - but Ubraco uses a different version (2.9.4.1) whereas we use version 3.0.3.0 (Also the one that RavenDb uses. AFAIK). So when you set it in the version you reference SetMaxClauseCount(int) it doesn't affect our type since it is different. Actually SetMaxClauseCount doesn't exists in 3.0.3.0. You access it using an auto-property: BooleanQuery.MaxClauseCount = value;
So the thing you need to try is to reference version 3.0.3.0 in you solution, set it using the auto-property, recompile and re-deploy.
FacetedQuery - Lucene.Net.Search.BooleanQuery - TooManyClauses
Hi all,
While performing a faceted query the where clause of the query performs an In expression.
If more that 1024 sku's exist then Lucene.Net.Search.BooleanQuery throws TooManyClauses Exception (As each sku is essentially or'd)
Lucene exposes a setting to allow this value to be overridden, BooleanQuery.SetMaxClauseCount(int value)
However in our testing this doesn't seem to be honoured and if more than 1024 clauses are appended then it still throws. (Perhaps it is reset somewhere)
Does anyone have any pointers as to a solution or a workaround? (Assuming that reducing the number of results returned by the query is not acceptable). Note, additional facets are added to the query
Any help would be greatly appreciated
Paul
Hi Paul,
I'm not sure as to why the max clause count is not overridden properly but as a workaround you could group the sku's per 1024 (or less), create multiple queries and combine the result sets afterwards. It's not ideal but it shouldn't cause much overhead when querying a local index.
Grtz Lennart
Thanks Grtz,
The result of that function is then later composed with other criteria, if I chunk it then then I'll have to combine it in memory. I think I'll have to do some rewriting to bring the later composed criteria forward to so then can be evaulated before alongside each chunk. You're idea will definitely work though, it will just unfortunately need more changes in the calling code than I'd like but hey ho :-)
Not sure there is going to be a neat answer to this one.:-)
Thanks again
Paul
Hi Paul (and Lennart),
The thing is. There's two different assemblies in the runtime environment since Umbraco also uses Lucene.net. Now this doesn't necessarily cause two different assemblies - but Ubraco uses a different version (2.9.4.1) whereas we use version 3.0.3.0 (Also the one that RavenDb uses. AFAIK). So when you set it in the version you reference SetMaxClauseCount(int) it doesn't affect our type since it is different. Actually SetMaxClauseCount doesn't exists in 3.0.3.0. You access it using an auto-property: BooleanQuery.MaxClauseCount = value;
So the thing you need to try is to reference version 3.0.3.0 in you solution, set it using the auto-property, recompile and re-deploy.
This might make the code run again.
Morten,
Thank you for that information!
Have included the code comments for how I implemented this in case anyone else hits this.
Thanks again!
Kind regards,
Paul
//NOTE I had to include references to two versions of Lucene.net
//Version 2.9.4.1 is used by umbraco, and can't be changed due to nuget dependencies
//Version 3.0.3.0 is used by uCommerce.
//To reference two versions:
//1) I downloaded the the Lucene.net 3.0.3.0 package into a temporary project (doing it in current sln tries to upgrade)
//2) Copied Lucene package to this slns packages folder
//3) Renamed Lucene.dll, .xml & .pdb to Lucene303.dll etc to prevent VS complaining that it is already added
//4) Added reference to Lucene303.dll
//5) In Lucene.dll reference, in properties window changed Aliases to UmbracoLucene
//6) In Lucene303.dll reference, in properties window changed Aliases to UcommerceLucene (so we can disambiguate between them)
//7) In this code included 'extern alias LuceneUCommerce;' at top of file to bring it into scope
//8) Ignored intellisense error (resharper) and used the MaxClauseCount property declared in Lucene 3.0.3.0
//9) Confirmed bool clauses can be set to a value of my choosing
//10) Profit
is working on a reply...