Sign in Register
Go to solution
Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Joe Harvey 34 posts 174 karma points
    Jul 21, 2022 @ 14:18
    Joe Harvey
    0

    Lucene.Net.Store.AlreadyClosedException 'this IndexReader is closed' Thrown For Simple Lucene Queries/Examine Search

    Umbraco Version: 8.18.4

    Examine Version: 1.2.2

    Lucene Version: 3.0.3

    I have a recurring issue on a clients production site, where some simple Lucene queries to get an Examine.ISearchResults result set are throwing a Lucene.Net.Store.AlreadyClosedException with the message 'this IndexReader is closed' which, until I have added some try/catch blocks around this code, is producing a YsoD/Server Error page.

    This issue has been going on for some time, but has become much more noticeable and longer lasting of recent. Previously I had seen these issues fleetingly and now Monday and Wednesday this week, my homepage has thrown these exceptions continually for between 30-50 minutes until finally resolving themselves with no interaction from me (at which point the results of the Lucene queries are cached for either 1h or 12h, so aren't seen again for sometime.

    For example:

    IIndex index;
ExamineManager.Instance.TryGetIndex(UmbracoIndexes.MembersIndexName, out index);
ISearcher searcher = index.GetSearcher();
return searcher.CreateQuery("member")
    .Field("umbracoMemberApproved", "0")
    .OrderBy(new SortableField[] { new SortableField("lastName"), new SortableField("firstName") })
    .Execute(9999);

    Which occasionally results in:

    Lucene.Net.Store.AlreadyClosedException:
   at Lucene.Net.Index.IndexReader.EnsureOpen (Lucene.Net, Version=3.0.3.0, Culture=neutral, PublicKeyToken=85089178b9ac3181: d:\Lucene.Net\FullRepo\trunk\src\core\Index\IndexReader.cs:204)
   at Lucene.Net.Index.IndexReader.IncRef (Lucene.Net, Version=3.0.3.0, Culture=neutral, PublicKeyToken=85089178b9ac3181: d:\Lucene.Net\FullRepo\trunk\src\core\Index\IndexReader.cs:163)
   at Examine.LuceneEngine.LuceneSearchResults+SkipEnumerable.GetEnumerator (Examine, Version=1.2.2.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Examine\LuceneEngine\LuceneSearchResults.cs:251)
   at Examine.SearchResultsBase.GetEnumerator (Examine, Version=1.2.2.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Examine\SearchResultsBase.cs:129)
   at Joogle.Controllers.HomeController+<>c__DisplayClass22_0.<getLeavers>b__0 (Joogle, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\User\source\repos\Dubai\Controllers\HomeController.cs:440)
   at Umbraco.Core.Cache.AppCacheExtensions+<>c__DisplayClass0_0`1.<GetCacheItem>b__0 (Umbraco.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Umbraco.Core\Cache\AppCacheExtensions.cs:22)
   at Umbraco.Core.Cache.FastDictionaryAppCacheBase+<>c__DisplayClass21_0.<GetSafeLazy>b__0 (Umbraco.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Umbraco.Core\Cache\FastDictionaryAppCacheBase.cs:285)
   at Umbraco.Core.Cache.WebCachingAppCache.GetInternal (Umbraco.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Umbraco.Core\Cache\WebCachingAppCache.cs:189)
   at Umbraco.Core.Cache.WebCachingAppCache.Get (Umbraco.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=null: D:\a\1\s\src\Umbraco.Core\Cache\WebCachingAppCache.cs:41)
   at Umbraco.Core.Cache.DeepCloneAppCache.Get (Umbraco.Core, Version=8.0.0.0, Culture=neutral, PublicKeyToken=null)
   at Joogle.Controllers.HomeController.getLeavers (Joogle, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\User\source\repos\Dubai\Controllers\HomeController.cs:434)
   at Joogle.Controllers.HomeController.Index (Joogle, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\User\source\repos\Dubai\Controllers\HomeController.cs:73)
   at lambda_method (Anonymously Hosted DynamicMethods Assembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)
   at System.Web.Mvc.ActionMethodDispatcher.Execute (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.ReflectedActionDescriptor.Execute (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c.<BeginInvokeSynchronousActionMethod>b__9_0 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`2.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass11_0.<InvokeActionMethodFilterAsynchronouslyRecursive>b__0 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass11_2.<InvokeActionMethodFilterAsynchronouslyRecursive>b__2 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass11_2.<InvokeActionMethodFilterAsynchronouslyRecursive>b__2 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass11_2.<InvokeActionMethodFilterAsynchronouslyRecursive>b__2 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass7_0.<BeginInvokeActionMethodWithFilters>b__1 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`1.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass3_6.<BeginInvokeAction>b__4 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass3_1.<BeginInvokeAction>b__1 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResult`1.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Controller+<>c.<BeginExecuteCore>b__152_1 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Controller.EndExecuteCore (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Controller+<>c.<BeginExecute>b__151_2 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Controller.EndExecute (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.MvcHandler+<>c.<BeginProcessRequest>b__20_1 (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncVoid`1.CallEndDelegate (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.Async.AsyncResultWrapper+WrappedAsyncResultBase`1.End (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.MvcHandler.EndProcessRequest (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest (System.Web.Mvc, Version=5.2.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+<>c__DisplayClass285_0.<ExecuteStepImpl>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker+<>c__DisplayClass4_0.<Invoke>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule.OnExecuteRequestStep (Microsoft.AspNet.TelemetryCorrelation, Version=1.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.HttpApplication+<>c__DisplayClass284_0.<OnExecuteRequestStep>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStepImpl (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStep (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

    I have seen multiple Our forum posts & Umbraco-CMS Issues with similar issues, most of which fail to come to any conclusive cause and become a dumping ground for related queries. In most cases, the suggested fixes are to upgrade to newer versions of Umbraco and/or Examine package. I have now done so (as above, I think I am running the latest supported versions possible with Umbraco v8) and I am still seeing these issues.

    Worth noting that my hosted UAT and Prod environments where I see these issues are using Azure AppServices, with seperate back-office and front-end nodes, manually configured as Master/Replica instances and with all Umbraco Azure configuration (like Umbraco.Core.MainDom.Lock, Umbraco.Core.LocalTempStorage and Umbraco.Examine.LuceneDirectoryFactory) implemented as per v8 documentation.

    I have disabled all 'Scale Out' load balancing on my front-end node(s) and ensured that all addition AppService deployment slots (like my Staging slot) have been turned off in an effort to ensure no other instances of the application could be interfering with the file system/Lucene directories, yet I still continue to see the issues.

    Having dug into the Lucene.Net code referenced in the last/top two lines of the stack trace, my assumption is that I somehow end up with an IndexReader with a refCount of <= 0 and therefore once IncRef() (lines 158 - 166) is called and that in turn calls EnsureOpen() (lines 200 - 206) it identifies that their are no existing references to my IndexReader and so it throws the exception.

    Quite how my IndexReader would get into that state is beyond me (as it looks like DecRef on lines 179 - 192 should deal with that scenario and commit and close the IndexReader gracefully). In any case, perhaps there is some exception handling missing further up the chain to catch that exception and (Re)Open a new IndexReader should it ever get into that scenario?

    Any advice or suggestions would be very welcomed as I am at my wits end with this and have run out of ideas. My next steps will be to write off Examine/Lucene search with Umbraco as not fit for purpose and re-write a large portion of my site to function without the use of it :-(

    Copy Link
  • Joe Harvey 34 posts 174 karma points
    Jul 21, 2022 @ 16:36
    Joe Harvey
    100

    Update: One thing I excluded from my code snippet above, was that I was caching the result set of the lucene query for 60 mins, for example:

    ISearchResults approvedMembers = AppCaches.RuntimeCache.GetCacheItem<ISearchResults >(
    "Approved_Members_" + DateTime.Now.ToString("yyyy-MM-dd"),
    () =>
    {
        IIndex index;
        ExamineManager.Instance.TryGetIndex(UmbracoIndexes.MembersIndexName, out index);
        ISearcher searcher = index.GetSearcher();
        return = searcher.CreateQuery("member")
            .Field("umbracoMemberApproved", "1")
            .AndNot(q => q.Field("hideInMemberDirectory", "1"))
            .OrderBy(new SortableField[] { new SortableField("lastName"), new SortableField("firstName") })
            .Execute(9999);
    },
    TimeSpan.FromMinutes(cachePeriod)
);

    This was based on the assumption that I would have an actual result set of data records and caching it would save me looking this back up on every page load.

    Having looked in more detail at Examine.ISearchResults I now see it is just a wrapper around IEnumerable<ISearchResult>....

    I am not totally clear on what this would mean, but I suspect it might mean that rather than caching the resultant data to come back from the query itself as I thought I was, I was likely caching a reference to the lucene query, and/or to the examine index file on disk (which is relatively volotile I think and replaced if any changes are made to the underlying object it represents) or IndexReader itself.

    I suspect that attempting to re-use this cached 'stale' pointer at a later date is probably what was causing the 'this IndexReader is closed' exception.

    I have now altered my code to take the ISearchResults, loop over each result and return the objects I am interested in in a List<T> instead. Will review and confirm if this does indeed resolve the issue.

    Copy Link
  • Johan Reitsma 68 posts 234 karma points MVP
    Oct 06, 2022 @ 08:35
    Johan Reitsma
    0

    Hi Joe,

    Did you ever found out the solution? We currently have a similar issue with our news collection.

    Greetings,

    Johan

    Copy Link
  • Joe Harvey 34 posts 174 karma points
    Oct 06, 2022 @ 10:57
    Joe Harvey
    0

    Hi Johan,

    Yes, thankfully my changes in my previous response resolved this for me.

    That being, converting my search results which were in the form of ISearchResults (which is essentially a "Lazily Evaluated" IEnumerable<>) to a more 'absolute (maybe "Eagerly Evaluated"?) List<>

    For example, replace this troublesome code (which can lead to the Lucene.Net.Store.AlreadyClosedException):

    ISearchResults pages = AppCaches.RuntimeCache.GetCacheItem<ISearchResults>(
    "My_Cache_Key",
    () =>
    {
        ISearchResults pages = [SOME CODE WHICH RETURNS ISearchResults];
        return pages;
    },
    TimeSpan.FromMinutes(60)
);

    With the below:

    List<IPublishedContent> pages = AppCaches.RuntimeCache.GetCacheItem<List<IPublishedContent>>(
    "My_Cache_Key",
    () =>
    {
        List<IPublishedContent> pages = new List<IPublishedContent>();
        ISearchResults results = [SOME CODE WHICH RETURNS ISearchResults];
        foreach (ISearchResult result in results) 
        {
            IPublishedContent page = Umbraco.Content.GetById(result.Id);
            pages.Add(page)
        }
        return pages;
    },
    TimeSpan.FromMinutes(60)
);

    This means you end up caching a more 'absolute' set of data, as opposed to caching the lazily evaluated 'pointer'.

    Let me know how you get on?

    Copy Link
Please Sign in or register to post replies

Write your reply to:

Draft