I have a done a few examine search setups now which is more or likly the same, this time though its really acting up. so a few stats first, its a umb 4.11.6(ill upgrade to 4.11.8 before production but it shouldnt change anything). The index is self consists of : Number of Fields: 16
Number of Documents: 116903
Number of Terms:310925
Its the biggest ive worked with so far but what i can read its no biggy.
So the method im having some trouble with is an autocomplete function that just return 10 items as json.
The method looks like this :
IEnumerable<SearchResult> advancedSearchResults = new List<SearchResult>();
Dictionary<string, string> searchParamsWithWeights = new Dictionary<string, string>();
searchParamsWithWeights.Add("Name", "^3");
searchParamsWithWeights.Add("Group", "1");
searchParamsWithWeights.Add("Road", "^3");
searchParamsWithWeights.Add("City", "^1");
searchParamsWithWeights.Add("Zip", "^1");
searchParamsWithWeights.Add("Phone", "^2");
searchParamsWithWeights.Add("Website", "^1");
searchParamsWithWeights.Add("Email", "^2");
searchParamsWithWeights.Add("Information", "^1");
searchParamsWithWeights.Add("MetaWords", "^3");
string luceneString = "+SearchPath:" + parentId + " ";
foreach (var item in searchParamsWithWeights)
{
luceneString += item.Key + ": ";
luceneString += "(+" + input.Replace(" ", "~0.5 +") + "~0.5)" + item.Value + " ";
}
var query = searchCriteria.RawQuery(luceneString);
advancedSearchResults = Searcher.Search(query).Where(x => x.Score >= 0.5f);
advancedSearchResults = advancedSearchResults.OrderBy(x => x.Fields["NodeTypeAlias"] == "faggruppe" ? 1 : 2);
Debug.WriteLine("Time1: " + s.ElapsedMilliseconds);
List<AutocompleteResult> resultList = new List<AutocompleteResult>();
foreach (var result in searchResults.Take(10))
{
resultList.Add(new AutocompleteResult { id = result.Id, data = result.Fields["Name"], type = result.Fields["Group"] });
}
if (resultList.Count < 10)
{
Debug.WriteLine("Time2: " + s.ElapsedMilliseconds);
foreach (var result in advancedSearchResults.Take(10 - resultList.Count))
{
if (!resultList.Exists(x => x.id == result.Id))
{
resultList.Add(new AutocompleteResult { id = result.Id, data = result.Fields["Name"], type = result.Fields["Group"] });
if (resultList.Count == 10)
{
break;
}
}
}
}
Debug.WriteLine("JustBeforeReturn: " + s.ElapsedMilliseconds);
return Json.Encode(resultList);
So ive only pasted some of the code, but the idea is that it does 2 searches, one more specific on the search the next with a lot of fuzzy and looking on a lot of params.
Im doing a few debug time traces which looks like this :
Time1: 456
Time2: 1263
JustBeforeReturn: 2011
So time 1 is being printed when the searches is done and in this search "advancedSearchResults" only contains 35 items or so, so its no biggy. But as soon as we touch the iEnumberable and take 10 items from it the times sky rockets to time 2.
So it might not be examine but im really wondering why it acting up so much. Ive tried tons of things, anyone has an idea ? There is no rocket sience in the above code.
IEnumerable describes behavior, while List is an implementation of that behavior. When you use IEnumerable, you give the compiler a chance to defer work until later, possibly optimizing along the way. If you use ToList() you force the compiler to reify the results right away.
Help, my examine code is acting slow
Hey guys and girls :)
I have a done a few examine search setups now which is more or likly the same, this time though its really acting up. so a few stats first, its a umb 4.11.6(ill upgrade to 4.11.8 before production but it shouldnt change anything). The index is self consists of :
Number of Fields: 16
Number of Documents: 116903
Number of Terms:310925
Its the biggest ive worked with so far but what i can read its no biggy.
So the method im having some trouble with is an autocomplete function that just return 10 items as json.
The method looks like this :
So ive only pasted some of the code, but the idea is that it does 2 searches, one more specific on the search the next with a lot of fuzzy and looking on a lot of params.
Im doing a few debug time traces which looks like this :
Time1: 456
Time2: 1263
JustBeforeReturn: 2011
So time 1 is being printed when the searches is done and in this search "advancedSearchResults" only contains 35 items or so, so its no biggy. But as soon as we touch the iEnumberable and take 10 items from it the times sky rockets to time 2.
So it might not be examine but im really wondering why it acting up so much. Ive tried tons of things, anyone has an idea ? There is no rocket sience in the above code.
Hi Rasmus,
I think the IEnumerable is the problem, and the Take(10) method is your bottleneck.
Try using a List or array instead as per the answer to the post below.
http://stackoverflow.com/questions/3628425/ienumerable-vs-list-what-to-use-how-do-they-work
IEnumerable describes behavior, while List is an implementation of that behavior. When you use IEnumerable, you give the compiler a chance to defer work until later, possibly optimizing along the way. If you use ToList() you force the compiler to reify the results right away.Hope that fixes it. Good luck.
Hey Dan
Thx for leading us in the direction, i think we found a work around for it.
A question you might be able to answer aswell ? Is this is a big lucene index ?
Personally I haven't indexed that many nodes in an Umbraco project before.
I have heard of people with 300k nodes in Umbraco with no issues.
There are articles in Google showing experiences of Lucene (without Umbraco) with much larger data sets (1 billion+).
Lucene should be fine with your data set.
Arh okay :)
Thx for your comments
its not nodes, its a custom indexer that takes some extra data and such.
is working on a reply...