Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • jake williamson 207 posts 873 karma points
    Sep 09, 2019 @ 00:33
    jake williamson
    0

    how to enable lucene sorting on date fields in umbraco 8?

    hey out there,

    so... i'm trying to get to grips with the new examine set up in v8. i'm getting there but hit a blocker when it comes to dates and date sorting.

    back in the v7 days we'd add something like this to <IndexAttributeFields> in config\ExamineIndex.config:

    <add Name="postDate" EnableSorting="true" Type="DateTime" />
    

    trick is, we no longer have the config\ExamineIndex.config file in v8!

    i've worked through the code at https://our.umbraco.com/documentation/reference/searching/examine/examine-events and it feels like it's something i maybe have to do here now?

    the other issue is the way the dates are stored. interestingly, date picker fields are being saved in a different format to v7:

    enter image description here

    in v7 it looks like:

    enter image description here

    at the moment i'm getting String was not recognized as a valid DateTime errors when attempting to format my postDate field.

    ultimately i want to be able to sort on the postDate field and i'm wondering if once i get that sorted the format will change?

    as ever, any suggestions are greatly received ;)

  • Shaishav Karnani from digitallymedia.com 354 posts 1638 karma points
    Sep 09, 2019 @ 05:34
    Shaishav Karnani from digitallymedia.com
    1

    Hi Jake,

    I have created a field postDate and tried below query. It is returning me results in postDate descending order. So, by default, it seems to work fine and no need to have additional EnableSorting="true".

    var results = examineSearcher.CreateQuery("content").Field("__IndexType", "content").OrderByDescending(new SortableField[] { new SortableField("postDate") }).Execute();
    

    Please try and let us know your views.

    Regards,

    Shaishav

  • Nik 1617 posts 7263 karma points MVP 7x c-trib
    Sep 09, 2019 @ 08:06
    Nik
    2

    Hi Jake,

    By default Umbraco's external index is storing all fields as a text string that is "non-sortable". Shaishav's method kinda overrides that a little bit but I believe it is only sorting it in an alphabetical approach.

    Because in V8 of Umbraco the config files for Examine are gone, you now need to write C# code to modify how indexing occurs, such as naming a specific field to be a specific type etc.

    In order to do this you need a custom Component and Composer.

    public class IndexerComponent : IComponent
    {
        private readonly IExamineManager examineManager;
    
        public IndexerComponent(IExamineManager examineManager)
        {
            this.examineManager = examineManager; 
        }
    
        public void Initialize()
        {
            var externalIndex = examineManager.Indexes.FirstOrDefault(i=>i.Name == "ExternalIndex");
            if(externalIndex != null)
            {
                externalIndex.FieldDefinitionCollection.AddOrUpdate(new FieldDefinition("postDate", FieldDefinitionType.Long);
            }
        }
    
        public void Terminate(){}
    
    }
    

    You might get away with just this, but I believe you also still need Shaishav's code to tell the search to sort by it.

    Thanks

    Nik

  • jake williamson 207 posts 873 karma points
    Sep 09, 2019 @ 09:11
    jake williamson
    101

    thank you the replies chaps ;)

    shaishav, you and i had come to the same conclusion! i'd been messing around with the OrderByDescending stuff and came up with identical code which i thought was working...

    however i realised that all the test items i had used dates that were organically ordering rather than being in a correct order...

    nik, interestingly i found a post from you back in july https://our.umbraco.com/forum/umbraco-8/98167-using-rangequery-on-custom-dynamically-added-fields-umbraco-8 outlining the externalIndex.FieldDefinitionCollection.AddOrUpdate solution.

    so i've implemented the above and it appears to be working - happy days!

    however...

    the weird thing is that the postDate field no longer appears in the back office:

    enter image description here

    and in luke it doesn't appear to have a value:

    enter image description here

    to get it actually outputting a value i can see, i've changed the property name to postDateForSorting and then set the value as part of calling IndexProviderTransformingIndexValues. the date value is formatted using an idea taken from https://stackoverflow.com/questions/4565303/lucene-net-how-can-i-add-a-date-filter-to-my-search-results

    e.ValueSet.TryAdd("postDateForSorting", ((DateTime)value).ToString("yyyyMMddHHmmssfff"));
    

    i can now see the value in the backoffice and in luke:

    enter image description here

    and sorting on the new postDateForSorting also appears to be working.

    i'm still slightly mystified as to what formatting is being applied to the createdDate and updateDate fields that the core creates?!

    seems dates and lucene are a bit of a pain one way and another though!

    this feels like it might be solved, if anything else comes up with the dates i'll post a (ahem) update.

    thanks for your help chaps ;)

  • David Armitage 510 posts 2081 karma points
    Jan 11, 2020 @ 13:22
    David Armitage
    0

    Hey Guys,

    I found a good link here which helped me. https://www.justnik.me/blog/indexing-sort-able-dates-in-umbraco-version-8

  • David Armitage 510 posts 2081 karma points
    Jan 11, 2020 @ 13:42
    David Armitage
    0

    Hi Guys,

    So I take that back. There were a few items not ordered correctly.

    I played with the code a little and found out that I could convert the date to a string and then add that as a custom search field.

    Here is my completed code

    Component

    public class ExamineComponents : IComponent
        {
            private readonly IUmbracoContextFactory _umbracoContextFactory;
            private readonly IExamineManager _examineManager;
            private readonly ILogger _logger;
    
            public ExamineComponents(IUmbracoContextFactory umbracoContextFactory, IExamineManager examineManager, ILogger logger)
            {
                _umbracoContextFactory = umbracoContextFactory;
                _examineManager = examineManager ?? throw new ArgumentNullException(nameof(examineManager));
                _logger = logger ?? throw new ArgumentNullException(nameof(logger));
            }
    
            public void Initialize()
            {
                if (!_examineManager.TryGetIndex(Constants.UmbracoIndexes.ExternalIndexName, out IIndex index))
                    throw new InvalidOperationException($"No index found by name {Constants.UmbracoIndexes.ExternalIndexName}");
    
                if (!(index is BaseIndexProvider indexProvider))
                    throw new InvalidOperationException("Could not cast");
    
                indexProvider.TransformingIndexValues += IndexProviderTransformingIndexValues;
            }
    
            private void IndexProviderTransformingIndexValues(object sender, IndexingItemEventArgs e)
            {
                if (int.TryParse(e.ValueSet.Id, out var nodeId))
                {
                    switch (e.ValueSet.ItemType)
                    {
                        case "blogDetailsPage":
    
                            using (UmbracoContextReference umbracoContextReference = _umbracoContextFactory.EnsureUmbracoContext())
                            {
                                var contentNode = umbracoContextReference.UmbracoContext.Content.GetById(nodeId);
                                if (contentNode != null)
                                {
                                    var date = contentNode.Value<DateTime>("date");
                                    e.ValueSet.Set("sortableDate", date.Date.ToString("yyyyMMddHHmmss"));
                                }
                            }
                            break;
                    }
                }
    
            }
    
            public void Terminate() { }
        }
    

    Composer

    using Umbraco.Core;
    using Umbraco.Core.Composing;
    
    namespace Website.Core.Events
    {
        [RuntimeLevel(MinLevel = RuntimeLevel.Run)]
        public class ExamineComposerPublic : IUserComposer
        {
            public void Compose(Composition composition)
            {
                composition.Components().Append<ExamineComponents>();
            }
        }
    }
    

    And this is how I use the new sort field.

    Sort Descending

    examineQuery.OrderByDescending(new SortableField[] { new SortableField("sortableDate") });
    

    Sort Ascending

    examineQuery.OrderBy(new SortableField[] { new SortableField("sortableDate") });
    

    One last thing!!

    This might not be too obvious for every one but each of the content nodes you are searching (in my case all the blog articles) need to be re-published before the new sortableDate field is generates. Before you run the search nip into the back end and re-publish.

  • jake williamson 207 posts 873 karma points
    Jan 29, 2020 @ 23:41
    jake williamson
    3

    ok people, resurrecting this post as i've found the definitive 'it really does work as expected and it came from the training' ;)

    part one is creating a component for examine events:

    public class ExamineEvents : IComponent
    {
        private readonly IExamineManager _examineManager;
    
        public ExamineEvents(IExamineManager examineManager)
        {
            _examineManager = examineManager;
        }
    
        public void Initialize()
        {
            if (!_examineManager.TryGetIndex("ExternalIndex", out IIndex index))
            {
                throw new InvalidOperationException("No index found by name ExternalIndex");
            }
    
            index.FieldDefinitionCollection.AddOrUpdate(new FieldDefinition("postDate", FieldDefinitionTypes.DateTime));
    
            //we need to cast because BaseIndexProvider contains the TransformingIndexValues event
            if (!(index is BaseIndexProvider indexProvider))
            {
                throw new InvalidOperationException("Could not cast");
            }
        }
    
        public void Terminate()
        {
        }
    }
    

    so the trick here is that its taking the "postDate" property (which is a date picker datatype) and setting it's type to DateTime. the added bonus is it's using the doctype property rather than a new property. nice.

    and then (and this was the bit that confussed the hell out of me...) put this into your query:

    .OrderByDescending(new SortableField("postDate", SortType.Long))
    

    so this is setting the "SortType" for the "SortableField" to a.... long?!?! er... why?

    well tbh i don't know the why, all i know is that this works as it should and it comes from the (training!) horses mouth so i'm running with it ;)

    hope this is useful for people out there as a quick google search throws up a whole load of people asking how the hell to order by dates using examine/lucence and umbraco...

    cheers,

    jake

  • Bo Jacobsen 608 posts 2406 karma points
    Aug 31, 2021 @ 10:12
    Bo Jacobsen
    0

    This should be the right answer.

    Because if you do not use SortType.Long in your .OrderByDescending(new SortableField("postDate")) then 635907024000000000 will come before 637657488000000000 and i dunno why.

    635907024000000000 is equal to 10. Februar 2016 and 637657488000000000 is equal to 28. august 2021.

    Thanks Jake :)

Please Sign in or register to post replies

Write your reply to:

Draft