Examine with a non-Umbraco dataService and SimpleDataIndexer
Hi everybody, I've run into an issue on the website I'm currently working on that has finally forced me to sign up to the forums. I've been through about a dozen other threads on this subject but none really shares my issue except for this one - unfortunately none of the solutions proposed there are solving my problem... which brings me to this thread.
The website I'm working on has two database connections: one for Umbraco of course and the other connects to a remote database which contains all employees (and their details) of the company the website is for. The website has an "employee directory" and each employee has their own profile page. So far, so good.
To enable searching on the website I've implemented ezSearch which works fine for searching through the Umbraco nodes. However, I also want to be able to search through the employee directory and list these results on the same page as the other search results (so I need to merge it with the ezSearch results).
This led me to configuring a custom IndexSet, ExamineIndexProvider and ExamineSearchProvider. I've read through several tutorials, blog posts and forum threads and they all go about it in the same manner I have. These are my settings (omitted miscellaneous index settings for brevity):
Now here comes the problem. No matter what I try (and I've been at it for a few hours now), when I try to load the website (both front- and back-office) I always get a YSOD with the following below (pointing to line 25). As I stated at the top of this post, there is a thread where users were experiencing the same error, but none of the solutions in that thread work for my project. Does anyone have any tips?
Configuration Error
Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.
Parser Error Message: Value cannot be null.
Parameter name: type
Source Error:
Line 23: <add name="ExternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"/>
Line 24:
Line 25: <add name="EmployeeIndexer" type="Examine.LuceneEngine.Providers.SimpleDataIndexer, Examine"
Line 26: dataService="Umbraco.Extensions.BLL.EmployeeDataService, Umbraco.Extensions.BLL"
Line 27: indexTypes="EmployeeDto"
Update: Apparently, the error wasn't being caused by the "type" attribute of EmployeeIndexer, but by the "dataService" attribute; I changed the assembly reference to "Umbraco.Extensions" (i.e. sans "BLL") and refreshed the page - no more config error!
Pretty frustrating as this cost me a few hours. But if I run into any further problems I'll post them in this thread.
Update: this has now mainly become an issue with incorporating my Employee search results with my ezSearch results. No matter what I try, I can't seem to get the ezSearch partial view macro to find results from my EmployeeIndexSet / EmployeeIndexer / EmployeeSearcher setup.
I have everything running perfectly in the Umbraco back-office; if I check the "Developer > Examine Management" tab, my EmployeeIndexer is there and has indexed 180 documents (the exact number of employees in the external database). The EmployeeSearcher also works, if I search for an employee's name it displays the desired result with the properties set as I defined in my EmployeeDataService BLL.
In trying to get the searcher running on the front-end I've done the following:
Made the application aware of the indexer by adding to the OnApplicationStarted() method:
Added certain variables and operations to the ezSearch partial view macro to include this indexer (misc. lines omitted):
var searcher = ExamineManager.Instance.SearchProviderCollection["ExternalSearcher"];
var searcherEmps = ExamineManager.Instance.SearchProviderCollection["EmployeeSearcher"];
var criteria = searcher.CreateSearchCriteria();
var criteriaEmps = searcherEmps.CreateSearchCriteria();
var criteria2 = criteria.RawQuery(query.ToString());
var criteriaEmps2 = criteriaEmps.RawQuery(query.ToString());
var results = searcher.Search(criteria2)
.Where(x => (
!Umbraco.IsProtected(int.Parse(x.Fields["id"]), x.Fields["path"]) ||
(
Umbraco.IsProtected(int.Parse(x.Fields["id"]), x.Fields["path"]) &&
Umbraco.MemberHasAccess(int.Parse(x.Fields["id"]), x.Fields["path"])
)) && (
(x.Fields["__IndexType"] == UmbracoExamine.IndexTypes.Content && Umbraco.TypedContent(int.Parse(x.Fields["id"])) != null) ||
(x.Fields["__IndexType"] == UmbracoExamine.IndexTypes.Media && Umbraco.TypedMedia(int.Parse(x.Fields["id"])) != null)
))
.ToList();
var resultsEmps = searcherEmps.Search(criteriaEmps2).Where(e => (e.Fields["__IndexType"] == "employee")).ToList();
// Merge the website results with the employee results
results.Concat(resultsEmps);
But the resultsEmp variable is always empty and I've been playing around with the macro for a few hours now... The Where() parameter is correct as when I use the EmployeeSearcher in Umbraco's back-office, the results have that exact string as their __IndexType value.
Does anyone have any clue what I'm missing here? I'm starting to get a bit desperate as I'm running out of ideas and hunches.
First things first - if you omit the Where() condition entirely, does resultsEmps contain any results at all?
The other thing is, what does the variable query contain (give us a sample)? I'm just wondering as you're using it to create a Raw Query rather than specify fields etc. to search.
You can search multiple indexes you need to use multisearcher, i see you are trying to do 2 searches and munge the results, see http://our.umbraco.org/forum/developers/extending-umbraco/43877-Search-both-content-and-PDFs-in-a-single-examine-query also you may want to try running the queries in luke or examine inspector and see if that works. Also you are doing a search then using linq to filter the search this can also be done using examine it will be more performant.
Hi Robert, thanks for taking the time to respond. I appreciate it.
Omitting the Where() condition was one of my latest attempts to get something out of it, but unfortunately it doesn't make any difference. As for the query variable, most of that has been taken from the initial code in the ezSearch macro (before modifying anything) so it's nothing I've specifically come up with. This is everything relevant I have which has to do with "query" though, all of which was already present in ezSearch before I started customizing:
var query = new StringBuilder();
query.AppendFormat("-{0}:1 ", model.HideFromSearchField);
switch (model.IndexType)
{
case UmbracoExamine.IndexTypes.Content:
query.AppendFormat("+({0}) ", contentPathFilter);
break;
case UmbracoExamine.IndexTypes.Media:
query.AppendFormat("+({0}) ", mediaPathFilter);
break;
default:
query.AppendFormat("+(({0}) ({1})) ", contentPathFilter, mediaPathFilter);
break;
}
// Ensure page contains all search terms in some way
foreach (var term in model.SearchTerms)
{
var groupedOr = new StringBuilder();
foreach (var searchField in model.SearchFields)
{
groupedOr.AppendFormat("{0}:{1}* ", searchField, term);
}
query.Append("+(" + groupedOr.ToString() + ") ");
}
// Rank content based on positon of search terms in fields
for (var i = 0; i < model.SearchFields.Count; i++)
{
foreach (var term in model.SearchTerms)
{
query.AppendFormat("{0}:{1}*^{2} ", model.SearchFields[i], term, model.SearchFields.Count - i);
}
}
The lines which immediately follows this block of code are my "var criteria2 = ..." and "var criteriaEmps2 = ..." lines from my previous post.
Also, it is worth noting that when I put Visual Studio into debug mode and inspect the variables line by line, everything seems to be working properly up until the "var resultsEmps = ..." line. "searcherEmps" contains the correct Examine information, "criteriaEmps" and "criteriaEmps2" likewise. But yet, nothing can be found.
Hi Ismail, thanks for responding also. To address a few points from your post:
I had a look at the thread you linked (and the one which was linked from there), but I'm not sure that's what I'm looking for. As I wrote in my first post, the employees I'm trying to add to the results page are from an external database and therefore the ExamineSearchProvider is of a different type (i.e. not "UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine" but "Examine.LuceneEngine.Providers.LuceneSearcher, Examine"). If I try to transform it into a MultiIndexSearcher and add multiple IndexSets to it this throws errors when I refresh the website.
Also, the Index and its Searcher appear to be functioning perfectly in the back-office (Developer > tab "Examine Management"), indexing 180 objects and returning them there when I use the search tool. So that leads me to believe there's nothing wrong with the way this part of my customization is set up. Unless I'm missing something, I think it has something to do with how the ezSearch macro works. My custom lines from my previous post(s) are all based on how the default implementation of the ezSearch macro performs the search.
On a side note: the reason I'm using LINQ to filter the results is because that line was also based on the default ezSearch implementation for the "ExternalSearcher" ExamineSearchProvider.
Have you tried running the raw query that you generate for employee search in the backend or using luke do you get results? Also can you paste the full query here.
Hi again Ismail, it turns out that the raw query was interfering with the EmployeeSearcher somehow. What I've done is eliminated a few steps (i.e. those which have to do with StringBuilder and RawQuery) and executed a search on the EmployeeSearcher straight away - this leads to it returning results! Finally!
I'm still not sure why the out-of-the-box approach which is used for the ExternalSearcher works in ezSearch but not for my own index (if anyone has a hunch, let me know) but I'm glad that I've been able to sort this out.
My current (functioning) code snippet, the relevant bits:
string[] employeeFields = new string[] {
"urlName", "name", "menuTitle", "metaKeywords", "metaDescription", "defaultContent"
};
var searcherEmps = ExamineManager.Instance.SearchProviderCollection["EmployeeSearcher"];
var criteriaEmps = searcherEmps.CreateSearchCriteria().GroupedOr(employeeFields, model.SearchTerm).Compile();
var resultsEmps = searcherEmps.Search(criteriaEmps).ToList();
// Merge the employee results with the website results, in that order
results = resultsEmps.Concat(results).ToList();
As you can see if you compare this with my previous code (in the OP), instead of using StringBuilder to format a query and using it to supply criteriaEmps with a "raw query", I now simply provide criteriaEmps with a compiled query based on the fields which are present in the EmployeeIndexSet and then proceed to search it.
Thanks to everyone for pitching in with ideas and if anyone has further suggestions for improvements, let me know. Hopefully this thread will help someone out in the future.
Glad you fixed it, i would have a play at getting multisearcher working, its more performant also the results will be sorted according to score, currently you have 2 queries each sorting by score then you are merging the two however the overall result set is not sorted by score.
Thanks for the advice Ismail. I'll keep that in mind for next time I need to combine the results of two indici on one search results page.
For this specific implementation it's not an issue as the two result sets are going to be displayed separate from one another on the page (i.e. grouped by headings - "Employee results" and "Website results"), so as long as they are sorted by their scores individually it will be fine. As long as they both count towards the paging feature - which they do - all is well.
Examine with a non-Umbraco dataService and SimpleDataIndexer
Hi everybody, I've run into an issue on the website I'm currently working on that has finally forced me to sign up to the forums. I've been through about a dozen other threads on this subject but none really shares my issue except for this one - unfortunately none of the solutions proposed there are solving my problem... which brings me to this thread.
The website I'm working on has two database connections: one for Umbraco of course and the other connects to a remote database which contains all employees (and their details) of the company the website is for. The website has an "employee directory" and each employee has their own profile page. So far, so good.
To enable searching on the website I've implemented ezSearch which works fine for searching through the Umbraco nodes. However, I also want to be able to search through the employee directory and list these results on the same page as the other search results (so I need to merge it with the ezSearch results). This led me to configuring a custom IndexSet, ExamineIndexProvider and ExamineSearchProvider. I've read through several tutorials, blog posts and forum threads and they all go about it in the same manner I have. These are my settings (omitted miscellaneous index settings for brevity):
web.config
ExamineIndex.config
ExamineSettings.config
Now here comes the problem. No matter what I try (and I've been at it for a few hours now), when I try to load the website (both front- and back-office) I always get a YSOD with the following below (pointing to line 25). As I stated at the top of this post, there is a thread where users were experiencing the same error, but none of the solutions in that thread work for my project. Does anyone have any tips?
Configuration Error
Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.
Parser Error Message: Value cannot be null. Parameter name: type
Source Error:
Update: Apparently, the error wasn't being caused by the "type" attribute of EmployeeIndexer, but by the "dataService" attribute; I changed the assembly reference to "Umbraco.Extensions" (i.e. sans "BLL") and refreshed the page - no more config error!
Pretty frustrating as this cost me a few hours. But if I run into any further problems I'll post them in this thread.
Update: this has now mainly become an issue with incorporating my Employee search results with my ezSearch results. No matter what I try, I can't seem to get the ezSearch partial view macro to find results from my EmployeeIndexSet / EmployeeIndexer / EmployeeSearcher setup.
I have everything running perfectly in the Umbraco back-office; if I check the "Developer > Examine Management" tab, my EmployeeIndexer is there and has indexed 180 documents (the exact number of employees in the external database). The EmployeeSearcher also works, if I search for an employee's name it displays the desired result with the properties set as I defined in my EmployeeDataService BLL.
In trying to get the searcher running on the front-end I've done the following:
Made the application aware of the indexer by adding to the OnApplicationStarted() method:
Added certain variables and operations to the ezSearch partial view macro to include this indexer (misc. lines omitted):
But the resultsEmp variable is always empty and I've been playing around with the macro for a few hours now... The Where() parameter is correct as when I use the EmployeeSearcher in Umbraco's back-office, the results have that exact string as their __IndexType value.
Does anyone have any clue what I'm missing here? I'm starting to get a bit desperate as I'm running out of ideas and hunches.
Hey William,
First things first - if you omit the Where() condition entirely, does resultsEmps contain any results at all?
The other thing is, what does the variable query contain (give us a sample)? I'm just wondering as you're using it to create a Raw Query rather than specify fields etc. to search.
- Rob.
William,
You can search multiple indexes you need to use multisearcher, i see you are trying to do 2 searches and munge the results, see http://our.umbraco.org/forum/developers/extending-umbraco/43877-Search-both-content-and-PDFs-in-a-single-examine-query also you may want to try running the queries in luke or examine inspector and see if that works. Also you are doing a search then using linq to filter the search this can also be done using examine it will be more performant.
Regards
Ismail
Hi Robert, thanks for taking the time to respond. I appreciate it.
Omitting the Where() condition was one of my latest attempts to get something out of it, but unfortunately it doesn't make any difference. As for the query variable, most of that has been taken from the initial code in the ezSearch macro (before modifying anything) so it's nothing I've specifically come up with. This is everything relevant I have which has to do with "query" though, all of which was already present in ezSearch before I started customizing:
The lines which immediately follows this block of code are my "var criteria2 = ..." and "var criteriaEmps2 = ..." lines from my previous post.
Also, it is worth noting that when I put Visual Studio into debug mode and inspect the variables line by line, everything seems to be working properly up until the "var resultsEmps = ..." line. "searcherEmps" contains the correct Examine information, "criteriaEmps" and "criteriaEmps2" likewise. But yet, nothing can be found.
Hi Ismail, thanks for responding also. To address a few points from your post:
I had a look at the thread you linked (and the one which was linked from there), but I'm not sure that's what I'm looking for. As I wrote in my first post, the employees I'm trying to add to the results page are from an external database and therefore the ExamineSearchProvider is of a different type (i.e. not "UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine" but "Examine.LuceneEngine.Providers.LuceneSearcher, Examine"). If I try to transform it into a MultiIndexSearcher and add multiple IndexSets to it this throws errors when I refresh the website.
Also, the Index and its Searcher appear to be functioning perfectly in the back-office (Developer > tab "Examine Management"), indexing 180 objects and returning them there when I use the search tool. So that leads me to believe there's nothing wrong with the way this part of my customization is set up. Unless I'm missing something, I think it has something to do with how the ezSearch macro works. My custom lines from my previous post(s) are all based on how the default implementation of the ezSearch macro performs the search.
On a side note: the reason I'm using LINQ to filter the results is because that line was also based on the default ezSearch implementation for the "ExternalSearcher" ExamineSearchProvider.
William,
Have you tried running the raw query that you generate for employee search in the backend or using luke do you get results? Also can you paste the full query here.
Regards
Ismail
Hi again Ismail, it turns out that the raw query was interfering with the EmployeeSearcher somehow. What I've done is eliminated a few steps (i.e. those which have to do with StringBuilder and RawQuery) and executed a search on the EmployeeSearcher straight away - this leads to it returning results! Finally!
I'm still not sure why the out-of-the-box approach which is used for the ExternalSearcher works in ezSearch but not for my own index (if anyone has a hunch, let me know) but I'm glad that I've been able to sort this out.
My current (functioning) code snippet, the relevant bits:
As you can see if you compare this with my previous code (in the OP), instead of using StringBuilder to format a query and using it to supply criteriaEmps with a "raw query", I now simply provide criteriaEmps with a compiled query based on the fields which are present in the EmployeeIndexSet and then proceed to search it.
Thanks to everyone for pitching in with ideas and if anyone has further suggestions for improvements, let me know. Hopefully this thread will help someone out in the future.
William,
Glad you fixed it, i would have a play at getting multisearcher working, its more performant also the results will be sorted according to score, currently you have 2 queries each sorting by score then you are merging the two however the overall result set is not sorted by score.
Regards
Ismail
Thanks for the advice Ismail. I'll keep that in mind for next time I need to combine the results of two indici on one search results page.
For this specific implementation it's not an issue as the two result sets are going to be displayed separate from one another on the page (i.e. grouped by headings - "Employee results" and "Website results"), so as long as they are sorted by their scores individually it will be fine. As long as they both count towards the paging feature - which they do - all is well.
Hi William,
Just wondering if you where able to finish extending EZSearch to search thru your external database. How did you resolve the dataService "null" error?
is working on a reply...