Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • William Parr 9 posts 31 karma points
    Nov 24, 2014 @ 14:56
    William Parr
    0

    Examine with a non-Umbraco dataService and SimpleDataIndexer

    Hi everybody, I've run into an issue on the website I'm currently working on that has finally forced me to sign up to the forums. I've been through about a dozen other threads on this subject but none really shares my issue except for this one - unfortunately none of the solutions proposed there are solving my problem... which brings me to this thread.

    The website I'm working on has two database connections: one for Umbraco of course and the other connects to a remote database which contains all employees (and their details) of the company the website is for. The website has an "employee directory" and each employee has their own profile page. So far, so good.

    To enable searching on the website I've implemented ezSearch which works fine for searching through the Umbraco nodes. However, I also want to be able to search through the employee directory and list these results on the same page as the other search results (so I need to merge it with the ezSearch results). This led me to configuring a custom IndexSet, ExamineIndexProvider and ExamineSearchProvider. I've read through several tutorials, blog posts and forum threads and they all go about it in the same manner I have. These are my settings (omitted miscellaneous index settings for brevity):

    web.config

    <section name="Examine" type="Examine.Config.ExamineSettings, Examine" requirePermission="false" />
    <section name="ExamineLuceneIndexSets" type="Examine.LuceneEngine.Config.IndexSets, Examine" requirePermission="false" />
    

    ExamineIndex.config

    <IndexSet SetName="EmployeeIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/Employee/">
      <IndexAttributeFields>
      </IndexAttributeFields>
      <IndexUserFields>
        <add Name="id" />
        <add Name="urlName" />
        <add Name="name" />
        <add Name="menuTitle" />
        <add Name="defaultContent" />
      </IndexUserFields>
      <IncludeNodeTypes>
      </IncludeNodeTypes>
    </IndexSet>
    

    ExamineSettings.config

        <Examine>
          <ExamineIndexProviders>
            <providers>
              <add name="EmployeeIndexer" type="Examine.LuceneEngine.Providers.SimpleDataIndexer, Examine" 
                dataService="Umbraco.Extensions.BLL.EmployeeDataService, Umbraco.Extensions.BLL" 
                indexTypes="EmployeeDto" 
                interval="10" 
                runAsync="true" />
            </providers>
          </ExamineIndexProviders>
    
          <ExamineSearchProviders defaultProvider="ExternalSearcher">
            <providers>
              <add name="EmployeeSearcher" type="Examine.LuceneEngine.Providers.LuceneSearcher, Examine" 
                 analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" 
                 enableLeadingWildcards="true" 
                 indexSets="EmployeeIndexSet" />
            </providers>
          </ExamineSearchProviders>
        </Examine>
    

    Now here comes the problem. No matter what I try (and I've been at it for a few hours now), when I try to load the website (both front- and back-office) I always get a YSOD with the following below (pointing to line 25). As I stated at the top of this post, there is a thread where users were experiencing the same error, but none of the solutions in that thread work for my project. Does anyone have any tips?

    Configuration Error

    Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

    Parser Error Message: Value cannot be null. Parameter name: type

    Source Error:

    Line 23: <add name="ExternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"/>
    Line 24: 
    Line 25: <add name="EmployeeIndexer" type="Examine.LuceneEngine.Providers.SimpleDataIndexer, Examine" 
    Line 26: dataService="Umbraco.Extensions.BLL.EmployeeDataService, Umbraco.Extensions.BLL" 
    Line 27: indexTypes="EmployeeDto" 
    
  • William Parr 9 posts 31 karma points
    Nov 24, 2014 @ 15:30
    William Parr
    1

    Update: Apparently, the error wasn't being caused by the "type" attribute of EmployeeIndexer, but by the "dataService" attribute; I changed the assembly reference to "Umbraco.Extensions" (i.e. sans "BLL") and refreshed the page - no more config error!

    Pretty frustrating as this cost me a few hours. But if I run into any further problems I'll post them in this thread.

  • William Parr 9 posts 31 karma points
    Nov 26, 2014 @ 15:13
    William Parr
    0

    Update: this has now mainly become an issue with incorporating my Employee search results with my ezSearch results. No matter what I try, I can't seem to get the ezSearch partial view macro to find results from my EmployeeIndexSet / EmployeeIndexer / EmployeeSearcher setup.

    I have everything running perfectly in the Umbraco back-office; if I check the "Developer > Examine Management" tab, my EmployeeIndexer is there and has indexed 180 documents (the exact number of employees in the external database). The EmployeeSearcher also works, if I search for an employee's name it displays the desired result with the properties set as I defined in my EmployeeDataService BLL.

    In trying to get the searcher running on the front-end I've done the following:

    Made the application aware of the indexer by adding to the OnApplicationStarted() method:

    public void OnApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
    {
      ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"].GatheringNodeData += OnGatheringNodeData;
      ExamineManager.Instance.IndexProviderCollection["EmployeeIndexer"].GatheringNodeData += OnGatheringNodeData;
    }
    

    Added certain variables and operations to the ezSearch partial view macro to include this indexer (misc. lines omitted):

    var searcher = ExamineManager.Instance.SearchProviderCollection["ExternalSearcher"];
    var searcherEmps = ExamineManager.Instance.SearchProviderCollection["EmployeeSearcher"];
    var criteria = searcher.CreateSearchCriteria();
    var criteriaEmps = searcherEmps.CreateSearchCriteria();
    
    var criteria2 = criteria.RawQuery(query.ToString());
    var criteriaEmps2 = criteriaEmps.RawQuery(query.ToString());
    
    var results = searcher.Search(criteria2)
      .Where(x => (
        !Umbraco.IsProtected(int.Parse(x.Fields["id"]), x.Fields["path"]) ||
        (
          Umbraco.IsProtected(int.Parse(x.Fields["id"]), x.Fields["path"]) &&
          Umbraco.MemberHasAccess(int.Parse(x.Fields["id"]), x.Fields["path"])
        )) && (
          (x.Fields["__IndexType"] == UmbracoExamine.IndexTypes.Content && Umbraco.TypedContent(int.Parse(x.Fields["id"])) != null) ||
          (x.Fields["__IndexType"] == UmbracoExamine.IndexTypes.Media && Umbraco.TypedMedia(int.Parse(x.Fields["id"])) != null)
        ))
      .ToList();
    
    var resultsEmps = searcherEmps.Search(criteriaEmps2).Where(e => (e.Fields["__IndexType"] == "employee")).ToList();
    
    // Merge the website results with the employee results
    results.Concat(resultsEmps);
    

    But the resultsEmp variable is always empty and I've been playing around with the macro for a few hours now... The Where() parameter is correct as when I use the EmployeeSearcher in Umbraco's back-office, the results have that exact string as their __IndexType value.

    Does anyone have any clue what I'm missing here? I'm starting to get a bit desperate as I'm running out of ideas and hunches.

  • Robert Foster 459 posts 1820 karma points MVP 2x admin c-trib
    Nov 26, 2014 @ 15:32
    Robert Foster
    0

    Hey William,

    First things first - if you omit the Where() condition entirely, does resultsEmps contain any results at all?

    The other thing is, what does the variable query contain (give us a sample)?  I'm just wondering as you're using it to create a Raw Query rather than specify fields etc. to search.

    - Rob.

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Nov 26, 2014 @ 15:44
    Ismail Mayat
    0

    William,

    You can search multiple indexes you need to use multisearcher, i see you are trying to do 2 searches and munge the results, see http://our.umbraco.org/forum/developers/extending-umbraco/43877-Search-both-content-and-PDFs-in-a-single-examine-query also you may want to try running the queries in luke or examine inspector and see if that works. Also you are doing a search then using linq to filter the search this can also be done using examine it will be more performant.  

    Regards

    Ismail

  • William Parr 9 posts 31 karma points
    Nov 26, 2014 @ 15:47
    William Parr
    0

    Hi Robert, thanks for taking the time to respond. I appreciate it.

    Omitting the Where() condition was one of my latest attempts to get something out of it, but unfortunately it doesn't make any difference. As for the query variable, most of that has been taken from the initial code in the ezSearch macro (before modifying anything) so it's nothing I've specifically come up with. This is everything relevant I have which has to do with "query" though, all of which was already present in ezSearch before I started customizing:

    var query = new StringBuilder();
    query.AppendFormat("-{0}:1 ", model.HideFromSearchField);
    
    switch (model.IndexType)
    {
      case UmbracoExamine.IndexTypes.Content:
        query.AppendFormat("+({0}) ", contentPathFilter);
        break;
      case UmbracoExamine.IndexTypes.Media:
        query.AppendFormat("+({0}) ", mediaPathFilter);
        break;
      default:
        query.AppendFormat("+(({0}) ({1})) ", contentPathFilter, mediaPathFilter);
        break;
    }
    
    // Ensure page contains all search terms in some way
    foreach (var term in model.SearchTerms)
    {
      var groupedOr = new StringBuilder();
      foreach (var searchField in model.SearchFields)
      {
        groupedOr.AppendFormat("{0}:{1}* ", searchField, term);
      }
      query.Append("+(" + groupedOr.ToString() + ") ");
    }
    
    // Rank content based on positon of search terms in fields
    for (var i = 0; i < model.SearchFields.Count; i++)
    {
      foreach (var term in model.SearchTerms)
      {
        query.AppendFormat("{0}:{1}*^{2} ", model.SearchFields[i], term, model.SearchFields.Count - i);
      }
    }
    

    The lines which immediately follows this block of code are my "var criteria2 = ..." and "var criteriaEmps2 = ..." lines from my previous post.

    Also, it is worth noting that when I put Visual Studio into debug mode and inspect the variables line by line, everything seems to be working properly up until the "var resultsEmps = ..." line. "searcherEmps" contains the correct Examine information, "criteriaEmps" and "criteriaEmps2" likewise. But yet, nothing can be found.

  • William Parr 9 posts 31 karma points
    Nov 26, 2014 @ 16:29
    William Parr
    0

    Hi Ismail, thanks for responding also. To address a few points from your post:

    I had a look at the thread you linked (and the one which was linked from there), but I'm not sure that's what I'm looking for. As I wrote in my first post, the employees I'm trying to add to the results page are from an external database and therefore the ExamineSearchProvider is of a different type (i.e. not "UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine" but "Examine.LuceneEngine.Providers.LuceneSearcher, Examine"). If I try to transform it into a MultiIndexSearcher and add multiple IndexSets to it this throws errors when I refresh the website.

    Also, the Index and its Searcher appear to be functioning perfectly in the back-office (Developer > tab "Examine Management"), indexing 180 objects and returning them there when I use the search tool. So that leads me to believe there's nothing wrong with the way this part of my customization is set up. Unless I'm missing something, I think it has something to do with how the ezSearch macro works. My custom lines from my previous post(s) are all based on how the default implementation of the ezSearch macro performs the search.

    On a side note: the reason I'm using LINQ to filter the results is because that line was also based on the default ezSearch implementation for the "ExternalSearcher" ExamineSearchProvider.

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Nov 26, 2014 @ 16:52
    Ismail Mayat
    0

    William,

    Have you tried running the raw query that you generate for employee search in the backend or using luke do you get results? Also can you paste the full query here.

    Regards

    Ismail

  • William Parr 9 posts 31 karma points
    Nov 27, 2014 @ 11:18
    William Parr
    0

    Hi again Ismail, it turns out that the raw query was interfering with the EmployeeSearcher somehow. What I've done is eliminated a few steps (i.e. those which have to do with StringBuilder and RawQuery) and executed a search on the EmployeeSearcher straight away - this leads to it returning results! Finally!

    I'm still not sure why the out-of-the-box approach which is used for the ExternalSearcher works in ezSearch but not for my own index (if anyone has a hunch, let me know) but I'm glad that I've been able to sort this out.

    My current (functioning) code snippet, the relevant bits:

    string[] employeeFields = new string[] {
      "urlName", "name", "menuTitle", "metaKeywords", "metaDescription", "defaultContent"
    };
    
    var searcherEmps = ExamineManager.Instance.SearchProviderCollection["EmployeeSearcher"];
    var criteriaEmps = searcherEmps.CreateSearchCriteria().GroupedOr(employeeFields, model.SearchTerm).Compile();
    
    var resultsEmps = searcherEmps.Search(criteriaEmps).ToList();
    
    // Merge the employee results with the website results, in that order
    results = resultsEmps.Concat(results).ToList();
    

    As you can see if you compare this with my previous code (in the OP), instead of using StringBuilder to format a query and using it to supply criteriaEmps with a "raw query", I now simply provide criteriaEmps with a compiled query based on the fields which are present in the EmployeeIndexSet and then proceed to search it.

    Thanks to everyone for pitching in with ideas and if anyone has further suggestions for improvements, let me know. Hopefully this thread will help someone out in the future.

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Nov 27, 2014 @ 11:34
    Ismail Mayat
    0

    William,

    Glad you fixed it, i would have a play at getting multisearcher working, its more performant also the results will be sorted according to score, currently you have 2 queries each sorting by score then you are merging the two however the overall result set is not sorted by score.

    Regards

     

    Ismail

  • William Parr 9 posts 31 karma points
    Nov 27, 2014 @ 11:38
    William Parr
    0

    Thanks for the advice Ismail. I'll keep that in mind for next time I need to combine the results of two indici on one search results page.

    For this specific implementation it's not an issue as the two result sets are going to be displayed separate from one another on the page (i.e. grouped by headings - "Employee results" and "Website results"), so as long as they are sorted by their scores individually it will be fine. As long as they both count towards the paging feature - which they do - all is well.

  • Rosauro Luz 3 posts 72 karma points
    Aug 18, 2015 @ 19:54
    Rosauro Luz
    0

    Hi William,

    Just wondering if you where able to finish extending EZSearch to search thru your external database. How did you resolve the dataService "null" error?

Please Sign in or register to post replies

Write your reply to:

Draft