Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Bradley Kronson 58 posts 109 karma points
    Feb 17, 2014 @ 09:21
    Bradley Kronson
    0

    Examine - rebuilding index speeds

    I'm wondering if anyone can comment on speeds in time with rebuilding an Examine index in Umbraco - how long should it take? I have an index set of approx 15000 nodes, it can take 12 hours to a day to rebuild the index, is this normal ? If not, what should I be looking at to improve speeds, as this is causing huge issues with my site I am developing....

  • Ismail Mayat 4288 posts 9247 karma points MVP 2x admin c-trib
    Feb 17, 2014 @ 18:45
    Ismail Mayat
    0

    Bradley,

    That does not sound right to me. What version of umbraco are you using? also how many properties do you have on your nodes and are you doing anything with gatheringnode data event?

    Regards

    Ismail

  • Bradley Kronson 58 posts 109 karma points
    Feb 17, 2014 @ 19:49
    Bradley Kronson
    0

    Hey Ismail

    Thanks for you response.

    I'm using Umbraco 6.1.6 for this site. Definition for the index in question (I have a number in the site, this is a fairly simple one that takes for ever to complete, the others, well no comment). The user properties indexed are not a huge amount as you can see, from the Examine Inspector in the site, I see 19 fields, 15012 documents and 113818 terms

        <IndexSet SetName="QuickPicDataIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/QuickPicData/" IndexParentId="1075" >
            <IndexAttributeFields>
                <add Name="id" />
                <add Name="nodeName" />
                <add Name="updateDate" Type="DateTime" EnableSorting="True" />
                <add Name="createDate" Type="DateTime" EnableSorting="True" />
                <add Name="writerName" />
                <add Name="path" />
                <add Name="nodeTypeAlias" />
                <add Name="parentID" />
                <add Name="sortOrder" EnableSorting="true" Type="Int" /> 
            </IndexAttributeFields>
            <IndexUserFields>
                <add Name="umbracoExtension"/>
                <add Name="fleetCategoryManufacturer"/>
                <add Name="isDuoporta" />
                <add Name="fileThumbnail" />
                <add Name="hasRelatedMediaItems" />
            </IndexUserFields>
            <IncludeNodeTypes/>
            <ExcludeNodeTypes />
        </IndexSet>

    The entire site has around 15000 content nodes, around 240000 media items and around 8000 members in it. Not sure if these should impact the creation of the examine index above - especially since it contains only the content nodes

    This particular index has a DocumentWriting event connected to it, code is below

    protected void IndexerDocumentWriting(object sender, DocumentWritingEventArgs e)
        {
            var luceneDocument = e.Document;
            var p = e.Fields["__Path"];
            if (p != null)
            {
                var newVal = p.Replace(","" ");
                luceneDocument.Add(new Lucene.Net.Documents.Field("__luceneFriendlyAncestors", newVal, Lucene.Net.Documents.Field.Store.YESLucene.Net.Documents.Field.Index.ANALYZED));
            }  
        }  

    Perhaps something else that can give a clue. The site seems to lose its umbraco.config file quite frequently. Its there and then it isnt for a long time and then it comes back. No real indication as to what is causing this either.

    Thanks for you time
    Bradley
  • Ismail Mayat 4288 posts 9247 karma points MVP 2x admin c-trib
    Feb 18, 2014 @ 09:03
    Ismail Mayat
    0

    Bradley,

    I think this is one for Shannon who authored Examine. If you having issues with your umbraco.config as well then I think issue is due to sheer amount of nodes. If I remember rightly and a rebuild the umbraco.config file is used and i suspect that is very big? In fact can you report back actual size of it.

    If you tweet @Shazwazza with a link this post he may be able to help further.

    Regards

    Ismail

  • Bradley Kronson 58 posts 109 karma points
    Feb 18, 2014 @ 10:05
    Bradley Kronson
    0

    Thanks will try Shannon, the file (when I have seen it) is around 80-90mb in size, not too big from some of the other sites we have

     

     

  • Bradley Kronson 58 posts 109 karma points
    Feb 18, 2014 @ 15:55
    Bradley Kronson
    0

    I don't this if this helps, but the index files are created in the relevant examine folder - after this the machine just seems to keep running some process, processor usage is higher than it should be and memory usage climbs quite dramatically. And the buttons for Rebuild/Optimize never seem to be activated again.

  • Bradley Kronson 58 posts 109 karma points
    Feb 18, 2014 @ 15:56
    Bradley Kronson
    0

    I tried this on a version of the database which does not have all the media items in the database. Just a large subset of the content nodes (around 13000) This takes under ten seconds.

  • Stefan Kip 1578 posts 4013 karma points c-trib
    Feb 18, 2014 @ 15:58
    Stefan Kip
    1

    Yes I recognize this behavior. It's like Examine is 'stuck'.
    I did create an issue ticket for this behavior on Codeplex, but it seems all information on Codeplex is gone.

    I always fix this by killing the w3wp process a few times, each time after an index is created.

  • Bradley Kronson 58 posts 109 karma points
    Feb 18, 2014 @ 16:07
    Bradley Kronson
    0

    Thats quite a scary thing to do, when do you know when the indexes have completed rebuilding :)

  • Stefan Kip 1578 posts 4013 karma points c-trib
    Feb 18, 2014 @ 16:09
    Stefan Kip
    0

    Well, just monitor the App_Data\TEMP\ExamineIndexes\ folder, when no new index folders appear and the last folder was created a few minutes ago, you kinda know it just hangs.

  • Heather Floyd 482 posts 627 karma points MVP c-trib
    Sep 01, 2015 @ 18:09
    Heather Floyd
    0

    I know this is an old post, but it helped me troubleshoot the same sort of issue I was having today. In addition, this post helped me fix it: https://our.umbraco.org/forum/getting-started/installing-umbraco/57118-622-InternalIndexer-error-ArgumentNullException-Parameter-name-contentType?p=0

Please Sign in or register to post replies

Write your reply to:

Draft