Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jamie Attwood 144 posts 357 karma points
    Dec 11, 2019 @ 22:43
    Jamie Attwood
    0

    Querying Members using Examine and UDI's

    8 hours of hair pulling trying to match a UDI value from the examine Fields["key", "Value"] with a UDI provided for the key value Fields["key", "Value"] . I just can't get a search to return anything, any help appreciated.

    I am trying to return a list of members that have a content picker content property item in common.

    If I do a "select all" members query and output the field values, I can see all the UDI's listed out on the page, but when add a query field where I try to match a known UDI as the item to match on, the result list is empty.

    I have created a custom "External Member Searcher" with all the properties specified. That is all working as it should as seen when I select all.

    The code:

    var searcher = ExamineManager.Instance.SearchProviderCollection["ExternalMemberSearcher"];
    var searchCriteria = searcher.CreateSearchCriteria(IndexTypes.Member, BooleanOperation.Or);
    var query = searchCriteria.Field("advisor", Udi.Create(Constants.UdiEntityType.Document, currentAdvisor.GetKey()).ToString());
    var searchResults = searcher.Search(query.Compile());
    

    So I want to return all members that have the provided UDI matching the value in their advisor content picker - again, I can see a list of the text values getting outputted if I select all members with this statement, so it's not an issue of mismatched UDI's or that the values don't exist - they do.

    var searchResults = searcher.Search(searchCriteria.SearchIndexType, true);//returns all Members
    

    Not sure what is the issue here. If I output the raw query, here is the resulting lucene query (the field is called "Advisor":

    Raw Query: { SearchIndexType: member, LuceneQuery: +(advisor:umb://document/89db3329c94b49aea45c455b5ea3452d) +__IndexType:member }
    
  • Jamie Attwood 144 posts 357 karma points
    Dec 12, 2019 @ 18:25
    Jamie Attwood
    0

    So update on this... I had to bail on my original approach and build out raw queries instead. Looking deeper into this, the umb sytax crashes examine due to the colon (:) character as that is used in the lucene queries as a key/value separator. This results in the following error:

    Lucene.Net.QueryParsers.ParseException: Cannot parse '+__IndexType:member +(advisor:umb://document/89db3329c94b49aea45c455b5ea3452d*)': Encountered " ":" ": "" at line 1, column 33. Was expecting one of: <AND> ... <OR> ... <NOT> ... "+" ... "-" ... "(" ... ")" ... "*" ... "^" ... <QUOTED> ... <TERM> ... <FUZZY_SLOP> ... <PREFIXTERM> ... <WILDTERM> ... "[" ... "{" ... <NUMBER> ...
    

    Even if I try the lucene escape character "\" this still does not work. So for now, I am providing the guid, replacing the "-" characters and enabling wildcard searches before and after the text string. I am sure there will be performance issues against this but I have no other option for now.

    var searcher = ExamineManager.Instance.SearchProviderCollection["ExternalMemberSearcher"];
    var criteria = searcher.CreateSearchCriteria();
    
    var contentTypeFilter = string.Format("__IndexType:{0}", UmbracoExamine.IndexTypes.Member);
    var searchKey = "advisor";
    //var searchValue = Udi.Create(Constants.UdiEntityType.Document, currentAdvisor.GetKey()).ToString().Replace(":","\\:"); // This does not work as the umb colon causes a lucene sytax error
    var searchValue = currentAdvisor.GetKey().ToString().Replace("-","");
    
    var query = new StringBuilder();
    query.AppendFormat("+{0} ", contentTypeFilter);
    query.AppendFormat("+({0}:*{1}*)", searchKey, searchValue);
    var searchResults = searcher.Search(criteria.RawQuery(query.ToString()));
    

    Any Examine experts out there, I would love to hear any suggestions!. This has to be coming up more than I see it in the forums. To be honest, I am really confused with the whole UDI concept. If this is the way to reference moving forward, a guid seems like so much better solution. Currently, in umbraco 7.15.x I can't even get a content items UDI without working magic with GetKey(). You would think that there would be a GetUdi() static method out there... Seems like the syntax is really throwing a wrench into many aspects of Umbraco 7. Just my two cents....

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 13:13
    Dee
    0

    Hi Jamie,

    could you please provide how you created the custom external member searcher? Would really appreciate it.

    Thanks

    Dee

  • Jamie Attwood 144 posts 357 karma points
    Aug 28, 2020 @ 13:20
    Jamie Attwood
    0

    What are you trying to do exactly? You should be able to search members with the default configuration.

    Cheers,

    Jamie

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 13:23
    Dee
    0

    the default configuration is limited to a small subset of fields. I have a lot of custom fields for the members, which are not getting indexed.

    the documented approach here is also not working, which should extend the member index by all fields except of the built in ones:

    https://our.umbraco.com/documentation/Reference/Searching/Examine/indexing/#changing-ivaluesetvalidator

    I reported already the documented issue. Here another post, mentioning that the approach is not working: https://our.umbraco.com/forum/using-umbraco-and-getting-started/102514-search-member-index-by-custom-fields

    So i guess, the only option available is creating my own custom member index having all fields needed in it, and i found this post by you.

  • Jamie Attwood 144 posts 357 karma points
    Aug 28, 2020 @ 13:44
    Jamie Attwood
    0

    Dee, you should be able to copy the InternalMemberIndexSet and add your own fields to it. Here is what I did:

    In /Config/ExamineIndex.config I added a new IndexSet. It's in here that you can add the aliases for your custom member fields that you want to index. Standard field types should be straightforward, but I ran into a lot of issues with content pickers, media pickers etc.:

    <!--  External Members Custom Indexset -->
      <IndexSet SetName="ExternalMemberIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/ExternalMember/" >
        <IndexAttributeFields>
          <add Name="id" />
          <add Name="nodeName"/>
          <add Name="updateDate" />
          <add Name="writerName" />
          <add Name="loginName" />
          <add Name="email" />
          <add Name="nodeTypeAlias" />
          <add Name="advisor" />
        </IndexAttributeFields>
      </IndexSet>
    

    In /Config/ExamineSettings.config I created a new indexer in the ExamineIndexerProviders section:

    <!--  Custom External Member Indexer -->
            <add name="ExternalMemberIndexer" type="UmbracoExamine.UmbracoMemberIndexer, UmbracoExamine" 
                 supportUnpublished="true" supportProtected="true" />
    

    and finally added a new ExamineSearchProvider

    <!--  Custom Member Searcher -->
          <add name="ExternalMemberSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine" 
               analyzer="Lucene.Net.Analysis.KeywordAnalyzer, Lucene.Net" indexSets="ExternalMemberIndexSet" enableLeadingWildcard="true"/>
    

    That should be it and you can query your new index via the standard Examine querying... hope that helps...

  • Jamie Attwood 144 posts 357 karma points
    Aug 28, 2020 @ 13:46
    Jamie Attwood
    0

    So now I see that you are probably working in U8.... not sure if the above still applies as this was the mechanism in V7... I'll look into that...

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 13:48
    Dee
    0

    Jamie, Thanks a lot! Although I see that it was for v7, but i appreciate your effort!

    Yes, I am struggeling a lot with the v8... Unfortunately, there is no appropriate documentation, so all I have is the our community and Trial and error by my side

  • Jamie Attwood 144 posts 357 karma points
    Aug 28, 2020 @ 13:52
    Jamie Attwood
    0

    Yep, it's not the same... damn. Sorry about that. Yeah I hear you on the docs. I keep saying we need complete code (including Using statements, etc.) , working examples/downloadables for every topic. Top notch documentation is the only means to get greater usage and more Umbraco uptake by the global community....

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 13:59
    Dee
    0

    Exactly! Your words in HQ's ears.

    I also experienced that a lot of advanced questions asked here in the community remain unanswered. Painful...

    I will keep trying to find a solution, cause I simply have to...

  • Jamie Attwood 144 posts 357 karma points
    Aug 28, 2020 @ 14:01
    Jamie Attwood
    0

    Good luck and make sure you post your solution here if you can make it work as I'll need the info as I am now porting the same solution to V8...

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 14:03
    Dee
    0

    Thanks! I will

  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 21:46
    Dee
    0

    Hey Jamie,

    I managaed to create a custom member index with additional fields:

    Umbraco 8 Version:

    public class MemberIndexCreator : LuceneIndexCreator, IUmbracoIndexesCreator
        {
            private readonly IProfilingLogger _profilingLogger;
            private readonly ILocalizationService _localizationService;
            private readonly IPublicAccessService _publicAccessService;
    
            // Since Umbraco 8 has dependency injection out of the box, we can use it to inject
            // the different services that we need.
            public MemberIndexCreator(IProfilingLogger profilingLogger,
                ILocalizationService localizationService,
                IPublicAccessService publicAccessService
            )
            {
                _profilingLogger = profilingLogger;
                _localizationService = localizationService;
                _publicAccessService = publicAccessService;
            }
    
            // Noticed that we return a collection of indexes? Technically you
            // can create multiple indexes in an indexCreator :) You can have a look at
            // UmbracoIndexesCreator.cs in the CMS core and see how the CMS does that.
            public override IEnumerable<IIndex> Create()
            {
                var excludeFields = Umbraco.Core.Constants.Conventions.Member.GetStandardPropertyTypeStubs().Keys;
    
                var includeFields = new List<string>
                {
                    "id","nodeName","updateDate","loginName","email","__Key",
                    "leadsEmail", "services"
                };
    
                var index = new UmbracoMemberIndex("FeedbaxMemberIndex",
                    new UmbracoFieldDefinitionCollection(),
                    CreateFileSystemLuceneDirectory("FeedbaxMembers"),
                    new CultureInvariantWhitespaceAnalyzer(),
                    _profilingLogger,
                    new MemberValueSetValidator(null, null, includeFields, excludeFields));
    
                return new[] { index };
            }
        }
    
        public class MemberComponent : IComponent
        {
            private readonly IExamineManager _examineManager;
            private readonly MemberIndexCreator _memberIndexCreator;
    
            public MemberComponent(IExamineManager examineManager, MemberIndexCreator memberIndexCreator)
            {
                _examineManager = examineManager;
                _memberIndexCreator = memberIndexCreator;
            }
    
            public void Initialize()
            {
                // Because the Create method returns a collection of indexes,
                // we have to loop through them.
                foreach (var index in _memberIndexCreator.Create())
                {
                    _examineManager.AddIndex(index);
                }
            }
    
            public void Terminate() { }
        }
    
        public class MemberComposer : IUserComposer
        {
            public void Compose(Composition composition)
            {
                composition.Components().Append<MemberComponent>();
                composition.RegisterUnique<MemberIndexCreator>();
            }
        }
    
  • Dee 90 posts 274 karma points
    Aug 28, 2020 @ 21:59
    Dee
    0

    With this approach all property types of the member type "Member" get included automatically, without the need, adding them manually:

    public override IEnumerable<IIndex> Create()
            {
                var index = new UmbracoMemberIndex("FeedbaxMemberIndex",
                    new UmbracoFieldDefinitionCollection(),
                    CreateFileSystemLuceneDirectory("FeedbaxMembers"),
                    new CultureInvariantWhitespaceAnalyzer(),
                    _profilingLogger,
                    new MemberValueSetValidator(includeItemTypes: new string[] { "Member" }, null, null, null));
    
                return new[] { index };
            }
    
  • Jamie Attwood 144 posts 357 karma points
    Aug 31, 2020 @ 17:20
    Jamie Attwood
    0

    Great! thanks for this Dee!

  • mmaty 83 posts 207 karma points
    1 week ago
    mmaty
    0

    @Jamie, @Dee: Could you find a solution for querying UDIs from the Index? I have a ContentPicker, which stores an UDI into a property.

    This results in an index field containing the udi.

    I want to find all contents refering to a given content a selected by the ContentPicker. But I don't see a way to query for the UDI. A wildcard at the beginning of the search expression ist not really a solution, since it has a very bad performance.

    Any insights to this?

  • Dee 90 posts 274 karma points
    1 week ago
    Dee
    0

    Hey Mmaty,

    could you please provide your code?

    querying UDIs should be nothing else than querying an indexed field by its value. As long as you have the fieldName and the documenttype which is containing the field, then you should be fine.

    Dee

  • Jamie Attwood 144 posts 357 karma points
    1 week ago
    Jamie Attwood
    0

    It does not work with UDI's - at least in U7. The best that I could figure out is that the colon character (:) in the UDI crashes the query syntax which uses the colon as the key/value separator. I ended up having to retrieve the guid value, strip out the dashes from the guid and then enable wildcard so the search could start at the front and the back of the string. I know there is performance issues, but even escaping the colon character out of the udi won't work. This is an examine bug in my opinion. I should have posted one. If you find a solution please let me know! Thanks and good luck!

  • Dee 90 posts 274 karma points
    1 week ago
    Dee
    1

    Here is an example, where I get a GUID as a parameter and search it within the members index. "key" is a guid, stored in fields named "reviewItems".

                    if (!ExamineManager.Instance.TryGetIndex(Constants.UmbracoIndexes.MembersIndexName, out IIndex index) || !(index is IUmbracoIndex))
                        throw new InvalidOperationException("The required index MembersIndex was not found");
    
                    var fields = new string[1] { "reviewItems" };
    
                    var booleanOperation = index.GetSearcher()
                        .CreateQuery("member", Examine.Search.BooleanOperation.And)
                        .ManagedQuery(key, fields);
    
                    var member = _publishedContentQuery.Search(booleanOperation, 0, int.MaxValue, out long totalRecords)
                        .Select(x => x.Content as Umbraco.Web.PublishedModels.Member)
                        .Single();
    

    This example works fine within U8

  • mmaty 83 posts 207 karma points
    1 week ago
    mmaty
    0

    @dee: Could you explain, what _publishedContentQuery is?

  • Dee 90 posts 274 karma points
    1 week ago
    Dee
    0

    it provides you access to the contentcache.

    you need to inject this to your controller:

        private readonly IPublishedContentQuery _publishedContentQuery;
    
        public SubmitReviewController(IPublishedContentQuery publishedContentQuery)
        {
            _publishedContentQuery = publishedContentQuery;
        }
    
  • mmaty 83 posts 207 karma points
    1 week ago
    mmaty
    0

    @Dee: Thanks a lot. I didn't manage to get a valid Examine query based on the documentation. But ManagedQuery somehow gets it.

    For the records: This is my working example (using a content query):

    var examineManager = Current.Factory.GetInstance<IExamineManager>();
    string propertyAlias = "relatedContent";
    string myUdiString = "umb://document/e0c40ee351c743cea8257ff0dfc02f55";
    IIndex index;
    if (!examineManager.TryGetIndex( "InternalIndex", out index ))
    {
        return null;
    }
    
    var fields = new string[1] { propertyAlias };
    
    var booleanOperation = index.GetSearcher()
        .CreateQuery("content", Examine.Search.BooleanOperation.And)
        .ManagedQuery(myUdiString, fields);
    
    var results = Current.Factory.GetInstance<IPublishedContentQuery>().Search(booleanOperation, 0, int.MaxValue, out long totalRecords);
    
  • Jamie Attwood 144 posts 357 karma points
    1 week ago
    Jamie Attwood
    0

    Nice to hear that it works in U8! Can't find ManagedQuery in U7...

  • mmaty 83 posts 207 karma points
    1 week ago
    mmaty
    0

    I didn't want to use IPublishedQuery, so I replaced the last line with

    var results = index.GetSearcher().CreateQuery().ManagedQuery(myUdiString, fields).Execute();
    

    That way I can query unpublished content.

  • Jamie Attwood 144 posts 357 karma points
    1 week ago
    Jamie Attwood
    0

    @mmaty I think @Dee's solution is only dealing with GUIDS as well as opposed to UDIs (not 100%) on that. I still think UDIs are a confusing way to refer to a GUID IMHO. I would love to know why the syntax is the way it is. Thanks for sharing your solution.

  • mmaty 83 posts 207 karma points
    1 week ago
    mmaty
    0

    Well, I definitely have UDIs in the index and the query string is

    "umb://document/e0c40ee351c743cea8257ff0dfc02f55"
    

    And this works with the above solution. But I'm afraid that won't help you in any way for V7.

  • Dee 90 posts 274 karma points
    1 week ago
    Dee
    0

    glad that it helped

Please Sign in or register to post replies

Write your reply to:

Draft