Hi Keilo,

I haven't released the source as yet, as I think there's more I want to do, and I'm looking at submitting it back to the core at some time.  However, you can edit the controller to change the behaviour of the members link in the list for example.

One of the changes I'm planning is to have the ability to specify which fields to show in the list (within reason).  If you think there are any columns that should be displayed by default please let me know.

I should be finding time over the next week or so to include additional functionality (including an advanced filter and member export functionality)

Having said that, I may try to organise to move the project to Github over the next couple of days, so stay tuned :)

- Rob.

Hi Robert

Thanks for the pointers.

I have created 5 members( with custom properties) which were showing up nicely, then I wanted to see how usable it would be with more members so I imported around few hundred members with around 15 custom properties, via CMSimport package (max: 500 entry import), and after successfull import the MemberListview doesnt show the list of members.

For some reason, it gets the paging that correspond to number of entries i presume, but no entries are shown in the listview. I also noticed that when trying to open the tab, it really drag the server..

I am not sure if this is because of the new Member API (i.e. i dont know what method is used to get the list of members; Examine, Member.GetAllAsList(), XPATH or some new v7 magic )

This is how it looks on a bare v7.1.4 (also tested on 7.2) install with few hundred members with 15+ custom properties imported via CMSimport as mentioned above (for reference, the member search, tree etc. all works on imported members, also the first 5 members with same properties were showing up before import):

ah, I'll make sure that gets fixed up this week (most likely tomorrow).

I must admit I didn't test it with a great deal of members.

I do know the call to query the data is rather inefficient, so I'll make sure to optimise it as well.

Thanks,

Rob.

Thanks for the update Robert, looking forward

Could you make a filter based on group? i would like to export the list filtered to csv, selecting the fields i want to export.

These things and a config window to select what fields to show on the list would be great!

Thanks for this package, this is something Umbraco has to include on default installation!

@keilo I was wondering whether you could upload somewhere a copy of the Umbraco database you're testing the members with - I'm hard pressed for time at the moment, but want to get this resolved, and having a ready set of test data would help immensely if possible...

Ta,

Rob.

Hi Robert

I would love to but the one I tested was real member data. But the process can easily be replicated if you create a Excel/CSV file of 200-300 records with columns name, email, pw etc. and use the CMSImport package (no license required for imports upto 500) to batch import them as member.

Thats pretty much what I have done. Im pretty sure there was no specific field imported as all of them were simple text fields.

Hope that can help

Hey, got an update: Today I was able to finally spend some time on the MemberListView, and have managed to improve the speed using the built-in Examine Index. However, with a large number of people there's still a significant delay in getting results back (now 19seconds down from 1.3minutes for 1,000 records).

I'm currently working on the filtering, with the ability to implement custom filtering based on MemberType properties, and optionally expose an Export mechanism (will be configurable by configuration).

Should have something to show in a few days hopefully.

• Rob.

Hi Robert

Great to hear you are pushing the functionality full-force!

Am I reading it correctly, that using Examine Index on 1,000 records returns the result 1.3 minutes - 19seconds? or its mean to be 19 sec to return 1K records?

I was reading about this article from a Umbraco fellow who wrote a member api dll which reads direct from database to handle large, if not mistaken 5-10K of members. It was an old article (pre v7) and Im not really up-to-date on the fastest approach on v7 API still.

Looking forward to see how your development progress. Sounds like its gonna be filling the gap on member filtering/export in the v7!

Cheers!

What I meant was that prior to optimisation getting back a result-set of 1k members was taking over a minute to complete (including rendering on the browser).  Actually, when I think about it, part of that was probably Umbraco starting up. Subsequent requests (i.e. for next page etc.) were taking at least 46 seconds.

After optimising by using Examine to avoid hitting the database at all, the response time dropped to around 19 seconds.

Note that this is measured using the Network Profiling tools of the browser.  The actual time taken to get the members back from Examine is too small to measure, but then there's sorting and skip/take performed to get to the right page etc.  The higher the page number, the longer it takes to respond.

I'm going to be looking at other ways to optimise this today.

- Rob.

Hey guys,

Just a quick note to say I've uploaded a new version - this one uses the InternalMemberIndex to return the results pretty darn fast (split second for records < 1000, a few seconds on around 5k records).

The other enhancements include:

• Filter on Member Type, IsApproved and IsLockedOut.
• Export filtered members to CSV file - currently it's a basic set of data including Name, Login Name, Email address and MemberType (along with ID and Key).

You'll hvae to get your  hands dirty with this release and modify the InternalMemberIndex as follows:

  <IndexSet SetName="InternalMemberIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/InternalMember/">
    <IndexAttributeFields>
      <add Name="id" />
      <add Name="nodeName" EnableSorting="true"/>
      <add Name="updateDate" />
      <add Name="writerName" />
      <add Name="loginName" />
      <add Name="email" EnableSorting="true"/>
      <add Name="nodeTypeAlias" />
    </IndexAttributeFields>
    <IndexUserFields>
      <add Name="umbracoMemberApproved" />
      <add Name="umbracoMemberLockedOut" />
      <add Name="umbracoMemberLastLockoutDate" Type="DATETIME" />
      <add Name="umbracoMemberLastLogin" Type="DATETIME" />
      <add Name="umbracoMemberPasswordRetrievalAnswer" />
      <add Name="umbracoMemberPasswordRetrievalQuestion" />
      <add Name="comments" />
    </IndexUserFields>
  </IndexSet>

Let me know if there are any issues, I've tested it pretty extensively, but you never know.

- Rob.

Hi Rob 

This is looking great! Insanely fast!!! Very excited.

Couple of questions:

1) All I have to do with Examine.cong is add the attribute EnableSorting="true" to the 

<add Name="nodeName" /> and <add Name="email" />

question: if one to show custom fields in the listview, does it mean that one can put EnableSorting="true" and that will enable listview sorting for that field?

2) I didnt have the section called <IndexUserFields>..</IndexUserFields> from a standard install of v7, so I added this as per your notes.

question: What is the exact purpose of adding this and how does it take role in the listing and/or export 

3) I guess this one would be of interest to anyone fiddling with members, how to add custom fields to the listview and in-relation how to customize the fields for export so only the fields specified can be exported (defined in back-end).

If one to add custom fields (whether for view in the listview or in export CSV), what section(s) one needs to edit in Examine conf and backend code?

Cheers!

Hi Keilo,

the EnableSorting bit doesn't really have any bearing on the results - I was playing around with using Examine Sorting to sort the results, but I found it too unreliable.

The IndexUserFields section can be used to include fields in the search results.  If you use a javascript debugger to step through the memberListView.controller.js file (put a break point on line 69 in the reloadView method), you will notice that the list of results that comes back contains a list of properties.

This propereties list is built based on the fields that Examine returns, and so by including a field in the IndexUserFields section will effectively populate this extra properties list.

You could then modify the memberlistview.html file (the view that returns the list) so that it exposes any of those properties as another column in the table.

Sorting on the extra properties wouldn't work though - at present it's only the name and email address that is sorted.

My next bit of work on this is targeted at making the list and filter extensible by configuration - most of the backend work has been done with this in mind.  However it could take a while as I'm rather busy with client work; so unless someone wants to pay me to do it, this is going to have to take a back seat for a little while :|

Hope this helps,

- Rob.

Hi again Rob

Thank you for the clarification. So is it safe to remove/not-add EnableSorting in Examine config, or is there any potential benefit to keep it?

I have managed to add a custom column by .properties.customfield and it shows up nicely.

However I cant seem to figure out how to include custom fields in Excel Export function... Is there something Im missing? If i can get that bit to work that would be awesome.

Cheers!

Yes - export currently doesn't support additional fields - I'm planning on adding a way to make this extensible as well, but again that will have to wait until I have some more time

Thanks for the great work Robert. I have played around with a large number of members and speed enhancements with Examine is simply great!.

One feedback from end-user point-of-view is the Search filter entry. When a user tries to enter a filter search, the filter kicks-in after 3/4 chars (before completing the entry) and the entire listview disappears and appears when resultset returned. I know this is a minor thing but just sharing as a feedback, Im sure you are aware of it.

Regarding editing the Export functionality to include some custom fields, would you be able to share the source? Perhaps I can give it a try as I only want two custom properties to be in the export.

Thanks again!

cheers

Hey Keilo,

Here's the code for the method that does the CSV export:

        [HttpGet]
        [HttpQueryStringFilter("queryStrings")]
        public HttpResponseMessage GetMembersExport(FormDataCollection queryStrings)
        {
            // Base Query data
            string memberType = queryStrings.HasKey("memberType") ? queryStrings.GetValue<string>("memberType") : "";
            string orderBy = queryStrings.HasKey("orderBy") ? queryStrings.GetValue<string>("orderBy") : "email";
            Direction orderDirection = queryStrings.HasKey("orderDirection") ? queryStrings.GetValue<Direction>("orderDirection") : Direction.Ascending;

            string filter = queryStrings.HasKey("filter") ? queryStrings.GetValue<string>("filter") : "";

            int totalMembers = 0;

            var members = Mapper.Map<IEnumerable<MemberExportModel>>(MemberSearch.PerformMemberSearch(filter, queryStrings, out totalMembers,
                                                                                                        memberType,
                                                                                                        orderBy: orderBy, 
                                                                                                        orderDirection: orderDirection));

            var content = members.CreateCSV();

            // see http://stackoverflow.com/questions/9541351/returning-binary-file-from-controller-in-asp-net-web-api
            // & http://stackoverflow.com/questions/12975886/how-to-download-a-file-using-web-api-in-asp-net-mvc-4-and-jquery
            // We really should use an async version - the above reference includes an example.
            HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
            result.Content = new StringContent(content);
            result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
            result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
            result.Content.Headers.ContentDisposition.FileName = string.Format("Members_{0:yyyyMMdd}.csv", DateTime.Now);

            return result;
        }

You could re-create the function in your own `UmbracoAuthorizedApiController` controller and just point the resource.js file at the new controller; and then replace the MemberExportModel with your own version (and create the necessary mapping configuration).

the Mapping Configuration I'm currently using for MemberExportModel is this (create a class derived from `MapperConfiguration` and override `ConfigureMappings`:

            //FROM MemberListItem to MemberExportModel.
            config.CreateMap<MemberListItem, MemberExportModel>()
                .ForMember(member => member.MemberType,
                    expression => expression.MapFrom(item => item.ContentType.Name));

            config.CreateMap<IEnumerable<MemberListItem>, IEnumerable<MemberExportModel>>()
                  .ConvertUsing(results => results.Select(Mapper.Map<MemberExportModel>));

            //FROM SearchResult to MemberExportModel.
            config.CreateMap<SearchResult, MemberExportModel>()
                .ConvertUsing(result => Mapper.Map<MemberExportModel>(Mapper.Map<MemberListItem>(result)));

            config.CreateMap<IEnumerable<MemberListItem>, IEnumerable<MemberExportModel>>()
                  .ConvertUsing(items => items.Select(Mapper.Map<MemberExportModel>));

That's about all you need - you'll be able to draw out the appropriate fields from the Properties on the MemberListItem.  The code to effectively serialise a list of objects to CSV should be able to handle any object you throw at it, so just create an export model that exposes the fields you want to export as properties and you're done.

- Rob.

is working on a reply...