Hey Guys,

after experiencing huge performance issues, when working with the Member Service and large amount of members, I had to switch to the examine index, in terms of quering the members.

I managed to create a custom member index which works fine, as long as it is not getting updated. It does not matter, if the member gets update through the UI or BO.

Now I am experiencing different issues with the examine index, and to be honest, its driving me nuts... As a developer you have to choose between poor performance of the member service and the instability of the examine index, combined with lack of appropriate documentation. Poor developer experience these days working with Umbraco.

So back to the issue:

Every member can update his profile, which means updating some of the member properties. In total there are about 70 properties, which could be updated by the members. All properties are added to the custom member index during the index creation and I can find them as it should be.

When a member updates one or more profile properties, the index gets instable. The updated member can not be found by the index until the index is rebuilt.

Looking into the logs, I can see that sometimes after updating a member through the UI, examine throws exceptions like:

Error indexing queue items: System.ArgumentNullException: The value cannot be NULL. Parametername: key

or

Error indexing queue items: System.ArgumentException: Target array is not long enough to copy all items. Check array index and length.

I also found out, that even when there are no examine errors after updating a member through the BO, the updated member just disappears from the resultsets of the index. I took snapshots of the indexed fields of the member before and after the member update through the BO and found out, that more than 10 fields and their values just disappeared from the index of that member after the update. After rebuilding the index, all field values return.

I have no clue why this is happening and really hope for some help from the community.

Any ideas?

The only option available right now is to rebuild the index after each member update. But this can't be the solution, since this is an expensive operation.

Index Creation:

public class MemberIndexCreator : LuceneIndexCreator, IUmbracoIndexesCreator
    {
        private readonly IProfilingLogger _profilingLogger;
        private readonly ILocalizationService _localizationService;
        private readonly IPublicAccessService _publicAccessService;

        // Since Umbraco 8 has dependency injection out of the box, we can use it to inject
        // the different services that we need.
        public MemberIndexCreator(IProfilingLogger profilingLogger,
            ILocalizationService localizationService,
            IPublicAccessService publicAccessService
        )
        {
            _profilingLogger = profilingLogger;
            _localizationService = localizationService;
            _publicAccessService = publicAccessService;
        }

        // Noticed that we return a collection of indexes? Technically you
        // can create multiple indexes in an indexCreator :) You can have a look at
        // UmbracoIndexesCreator.cs in the CMS core and see how the CMS does that.
        public override IEnumerable<IIndex> Create()
        {
            var index = new UmbracoMemberIndex("FeedbaxMemberIndex",
                new UmbracoFieldDefinitionCollection(),
                CreateFileSystemLuceneDirectory("FeedbaxMembers"),
                //new CultureInvariantWhitespaceAnalyzer(),
                new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30),
                _profilingLogger,
                new MemberValueSetValidator(includeItemTypes: new string[] { "Member" }, null, null, null));

            return new[] { index };
        }
    }

    public class MemberComponent : IComponent
    {
        private readonly IExamineManager _examineManager;
        private readonly MemberIndexCreator _memberIndexCreator;

        public MemberComponent(IExamineManager examineManager, MemberIndexCreator memberIndexCreator)
        {
            _examineManager = examineManager;
            _memberIndexCreator = memberIndexCreator;
        }

        public void Initialize()
        {
            // Because the Create method returns a collection of indexes,
            // we have to loop through them.
            foreach (var index in _memberIndexCreator.Create())
            {
                _examineManager.AddIndex(index);
            }
        }

        public void Terminate() { }
    }

    public class MemberComposer : IUserComposer
    {
        public void Compose(Composition composition)
        {
            composition.Components().Append<MemberComponent>();
            composition.RegisterUnique<MemberIndexCreator>();
        }
    }

Searching:

if (!ExamineManager.Instance.TryGetIndex("FeedbaxMemberIndex", out IIndex index) || !(index is IUmbracoIndex))
                throw new InvalidOperationException("The required index ExternalIndex was not found");

            index.FieldDefinitionCollection.AddOrUpdate(new FieldDefinition("minimumProjectSize", FieldDefinitionTypes.Integer));

            var booleanOperation = index.GetSearcher()
                .CreateQuery("member", BooleanOperation.And)
                .Field("published", "1");

            if (serviceIds.Length > 0)
                booleanOperation = booleanOperation.And().GroupedOr(new string[] { "services" }, serviceIds);

            if (secondaryIds.Length > 0)
                booleanOperation = booleanOperation.And().GroupedOr(new string[] { "services" }, secondaryIds);

            if (focusAreaIds.Length > 0)
                booleanOperation = booleanOperation.And().GroupedOr(new string[] { "focusAreas" }, focusAreaIds);

            if (!string.IsNullOrEmpty(model.HourRateFilter))
                booleanOperation = booleanOperation.And().Field("hourRate", model.HourRateFilter);

            if (!string.IsNullOrEmpty(model.CompanySizeFilter))
                booleanOperation = booleanOperation.And().Field("size", model.CompanySizeFilter);

            if (!string.IsNullOrEmpty(model.CityFilter))
                booleanOperation = booleanOperation.And().Field("city", model.CityFilter);

            if (!string.IsNullOrEmpty(model.ProjectSizeFilter))
                booleanOperation = booleanOperation.And().RangeQuery<int>(new string[] { "minimumProjectSize" }, 0, Convert.ToInt32(model.ProjectSizeFilter));

            var sortedBooleanOperation = booleanOperation.OrderBy(new SortableField("Name", SortType.String));

            long totalRecords;

            var members = Umbraco.Web.Composing.Current.UmbracoHelper.ContentQuery.Search(
                (IQueryExecutor)sortedBooleanOperation, 
                (model.PageIndex - 1) * model.PageSize, 
                model.PageSize, 
                out totalRecords);

Modify part of member properties:

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult SaveCompanyInfo(MemberModel model)
{
    if (TryValidateModel(model.CompanyInfoModel))
    {
        model = UpdateCompanyInfo(model);

        return PartialView("~/Views/Partials/Provider/_EditCompanyProfile.cshtml", model);
    }

    return CurrentUmbracoPage();
}

private MemberModel UpdateCompanyInfo(MemberModel model)
{
    var member = Services.MemberService.GetById(model.Id);

    var sanitizedCompanyDescription = _sanitizer.Sanitize(model.CompanyInfoModel.CompanyDescription);

    string imageName = "logo.jpg";
    if (model.CompanyInfoModel.UploadedLogo != null)
        member.SetValue(Current.Services.ContentTypeBaseServices, "logo", imageName, model.CompanyInfoModel.UploadedLogo.InputStream);

    imageName = "header.jpg";
    if (model.CompanyInfoModel.UploadedHeaderImage != null)
        member.SetValue(Current.Services.ContentTypeBaseServices, "headerImage", imageName, model.CompanyInfoModel.UploadedHeaderImage.InputStream);

    member.Name = model.CompanyInfoModel.CompanyName;
    member.SetValue("slogan", model.CompanyInfoModel.Slogan);
    member.SetValue("minimumProjectSize", model.CompanyInfoModel.MinimumProjectSize);
    member.SetValue("hourRate", model.CompanyInfoModel.HourRate);
    member.SetValue("size", model.CompanyInfoModel.CompanySize);
    member.SetValue("founded", model.CompanyInfoModel.CompanyFounded);
    member.SetValue("description", sanitizedCompanyDescription);
    member.SetValue("youtubeVideo", model.CompanyInfoModel.YoutubeVideo);

    member.SetValue("lastUpdateTimestamp", DateTime.Now);

    Services.MemberService.Save(member);

    Logger.Info(this.GetType(), string.Format("Company Info edited successfully by Member-Name: {0}, Member-Id: {1}", member.Name, member.Id));

    ModelState.Clear();
    return UpdateMemberModel(model, member);
}

Thanks

Dee