Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Marcelo 18 posts 131 karma points
    Nov 14, 2018 @ 19:25
    Marcelo
    0

    How to repubish the entirely website contents, after include a new document type property?

    Hello,

    I am new in Umbraco world and I am developing my first project usign version 7.12.3, but I ended up with a question that even after 2 days searching in google I could not find a solution or workaround.

    I will try to explain in a very clear way:

    On my Umbraco Backoffice, I have:

    • Home (document type, composed with WebsiteBase)
    • WebsiteBase (document type - which I use for compositions).
    • [Another documents types]

    So basically, I have created my website at Content section with many childrens, everything ok, as I am going to have many contents, I started to use the Examine for some searchs, which is working gracefully too.

    Then, I figured out that I was needing a new Document Type property, I went to the Document Type 'WebsiteBase' and created the property 'umbracoNaviHide' and changed my Examine search to check this new property! But I notice that the Examine was indexing the Contents without that new property that I have just created.

    To try to refresh the Contents, I did:

    • Click with right button over 'Content root' and click on 'Republish entire site'.
    • Delete the file 'umbraco.config' and restart the umbraco application.
    • Inside the ApplicationEventHandler class at ApplicationStarted method I writed the following rows:
      • applicationContext.Services.ContentService.RebuildXmlStructures();
      • applicationContext.Services.ContentService.RePublishAll();
      • applicationContext.Services.ContentService.BuildXmlCache();

    Those tries recreated the umbraco.config file, but ALWAYS without the new property created inside the Document Type. The ONLY way to get the new property inside the cmsContentXML table was to go to each website page and click the button 'Save and Publish', after click that button the cmsContentXml table and umbraco.config (by consequence) were refreshed for that specific content id.

    Okay, It is possible to do this manual procedure for each website page, but my question is, what about tomorrow? If I need to create a new document type property again, with 400 hundred contents Am I going to have to Publish by EACH page?

    Does not exist a easy way to republish all contents after a new document type property insert?

  • Marc Goodson 2155 posts 14408 karma points MVP 9x c-trib
    Nov 14, 2018 @ 20:55
    Marc Goodson
    0

    Hi Marcelo

    I think RePublishAll, calls RebuildXmlStructures

    https://github.com/umbraco/Umbraco-CMS/blob/2e135d2006212df799765552bc0e916b9711cee9/src/Umbraco.Core/Services/ContentService.cs#L1001

    https://github.com/umbraco/Umbraco-CMS/blob/2e135d2006212df799765552bc0e916b9711cee9/src/Umbraco.Core/Services/ContentService.cs#L2202

    Which 'should' be clearing the cmsContentXML table... I wonder if it's timing out and rolling back?

    Did you pass in just the ids of the ContentType that had changed?

    RebuildXmlStructures(params int[] contentTypeIds)

    It would be interesting to see if you removed manually the entries, and ran the Data Integrity healthcheck (although this should be same as calling them in code) whether the entries are recreated (with the new property)

    Essentially when you change a document type, this should trigger the removal of any entries from cmsContentXml table (using the methods above).

    After an upgrade from v6 to v7, I'll often empty this table completely and rebuild from the data integrity healthcheck, but in the scenario you describe I'd expect it to work.

    Once the cmsContentXml table is refreshed, then the Umbraco.Config cache file on disk and examine indexes can be rebuilt, as these take the cms ContentXml table as their source...

  • Marcelo 18 posts 131 karma points
    Nov 14, 2018 @ 23:13
    Marcelo
    0

    Hello Marc,

    First of all, thank you for your reply.

    Yes,indeed the RePublishAll() is calling RebuildXmlStructures() internally, I ended up trying every method that I found inside the 'ContentService' object.

    On my initial tests I didn't pass the Document Type Id that I have changed, because my intention was to rebuild all the website doesn't mattering if it has changed or not.

    Anyway I just did this test:

    protected override void ApplicationStarted(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
            {
                base.ApplicationStarted(umbracoApplication, applicationContext);
    
                applicationContext.Services.ContentService.RebuildXmlStructures(new int[] { 1171, 1103 });
                applicationContext.Services.ContentService.BuildXmlCache();
            }
    

    No lucky, I checked the umbraco.config file and the cmsContentXml table but the Contents ids 1171 and 1103 continue without the new property 'umbracoNaviHide'.

    The last test I did was to truncate the cmsContentXml table and delete the umbraco.config file, when I restarted the application everything was created again, but still without the 'umbracoNaviHide' property :(

    The only way that I am able to refresh the cmsContentXml table is when I go to the Contents root and I choose one page and I click 'Save and publish' button. By doing this procedure, the table cmsContentXml is updated for that specific page with the new property 'umbracoNaviHide' and the 'umbraco.config' file is updated after I stop and start the application again.

  • Marc Goodson 2155 posts 14408 karma points MVP 9x c-trib
    Nov 15, 2018 @ 08:49
    Marc Goodson
    0

    Hi Marcelo

    The ids that you pass to RebuildXmlStructures should be the ids of the DocumentType's that have changed, and not the ids of the Content Items, if that makes sense?

    (but also this should just 'work')

    What I'm interested in is if you delete one entry from the cmsXMLContent table, and then run the Data integrity healthcheck, interested to see if it detects the missing item, and if rebuilding from the healthcheck dashboard, causes the recreation with the new property.

    Otherwise is there anything in the trace logs - app_data/logs during the rebuild process that would give a clue?

    regards

    marc

  • Marcelo 18 posts 131 karma points
    Nov 15, 2018 @ 13:59
    Marcelo
    0

    Hi Marc,

    Regarding the Health ckeck test, I think I did what you have said, I will explain step by step whith print screens of what I did:

    1 - I found a line inside the cmsContentXml which has not the new property called 'umbracoNaviHide':

    nodeId  xml
    1101    <secao id="1101" key="04502aa7-d0d9-4f17-b0b1-119b974a74de" parentID="1100" level="2" creatorID="0" sortOrder="0" createDate="2018-10-30T22:22:56" updateDate="2018-11-13T15:56:02" nodeName="Sobre Nós" urlName="sobre-nos" path="-1,1100,1101" isDoc="" nodeType="1081" creatorName="Marcelo" writerName="Marcelo" writerID="0" template="1068" nodeTypeAlias="secao" isPublished="true">
      <palavrasChaveSEO><![CDATA[[]]]></palavrasChaveSEO>
      <validarAgeGate>1</validarAgeGate>
    </secao>
    

    2 - I stoped the application and I executed this query inside the database:

    Delete [cmsContentXml] where nodeId = 1101;
    

    3 - I started the application and opened the Health Check > Data Integrity: topic_3

    4 - I clicked the green button and the Umbraco fixed my integrity error: topic_4

    5 - I executed the bellow query inside the database:

    SELECT [nodeId]
          ,[xml]
      FROM [cmsContentXml];
    

    6 - The result was:

    nodeId  xml
    1101    <secao id="1101" key="04502aa7-d0d9-4f17-b0b1-119b974a74de" parentID="1100" level="2" creatorID="0" sortOrder="0" createDate="2018-10-30T22:22:56" updateDate="2018-11-13T15:56:02" nodeName="Sobre Nós" urlName="sobre-nos" path="-1,1100,1101" isDoc="" nodeType="1081" creatorName="Marcelo" writerName="Marcelo" writerID="0" template="1068" nodeTypeAlias="secao" isPublished="true">
      <palavrasChaveSEO><![CDATA[[]]]></palavrasChaveSEO>
      <validarAgeGate>1</validarAgeGate>
    </secao>
    

    I believe it is not rebuilding the XML correctly.

    After this test, I went to the Content section to confirm if the new property was visible inside the backoffice, which it is: enter image description here

    The I clicked the button "Save and Publish" check the Content Info History print: enter image description here

    I executed the bellow query inside the database:

    SELECT [nodeId]
          ,[xml]
      FROM [cmsContentXml];
    

    And now voilà! the new property appeared inside the cmsContentXml table:

    nodeId  xml
    1101    <secao id="1101" key="04502aa7-d0d9-4f17-b0b1-119b974a74de" parentID="1100" level="2" creatorID="0" sortOrder="0" createDate="2018-10-30T22:22:56" updateDate="2018-11-15T11:44:46" nodeName="Sobre Nós" urlName="sobre-nos" path="-1,1100,1101" isDoc="" nodeType="1081" creatorName="Marcelo" writerName="Marcelo" writerID="0" template="1068" nodeTypeAlias="secao" isPublished="true"><palavrasChaveSEO><![CDATA[[]]]></palavrasChaveSEO><umbracoNaviHide>0</umbracoNaviHide><validarAgeGate>1</validarAgeGate></secao>
    

    The only way I can have the Xml update is by clicking the button "Save and Publish" :(

    Am I missing something?

  • Marc Goodson 2155 posts 14408 karma points MVP 9x c-trib
    Nov 18, 2018 @ 16:39
    Marc Goodson
    0

    Hi Marcelo

    I think I understand more what you mean, when RebuildXmlStructures is called, it will recreate the entry in cmxContentXml table, but it will take this entry from the database for the current 'published version' if the item (and if the db's published version, matches the version in the Umbraco cache, it seems to me it will use that)

    https://github.com/umbraco/Umbraco-CMS/blob/c27b9d76dbb0c72a700a6af13f4b50c50994a4b8/src/Umbraco.Core/Persistence/Repositories/ContentRepository.cs#L216

    Now your current saved version of your item won't have an umbracoNaviHide property, so I think that's why RebuildXmlStructures won't add one in... but when you 'load' the item into the Umbraco backoffice to edit, and press Save and Publish, then the new umbracoNaviHide property is 'saved' and therefore added to the Xml Representation of the object.

    (if you delete one of the entries from cmsContentXml that has already gained the umbracoNaviHide property, and trigger RebuildXmlStructures - then the entry should be recreated with the umbracoNaviHide property as when you saved and published the item previously, the underlying published database entry was updated!)

    In your case of adding an umbracoNaviHide property, I suppose this isn't too bad a consequence, as in order to 'tick' an item as requiring to be hidden, you will need to save and publish the page, thus updating the content store and the cmsContentXml table, when using .visible() helper to check if an item is hidden, the absence of the property should return false.

    You can check if a property exists using the HasProperty helper.

    bool showMenuItem = item.HasProperty("umbracoNaviHide") && item.GetPropertyValue<bool>("umbracoNaviHide")
    

    So in future if you add other properties to your doc types, and the page is already published, then only saving and publishing from the backoffice will add the new property to that items published cache, and be wary to 'code around the scenario' where the property might not exist... which would be a big problem if your checkbox logic was around the other way and called 'ShowOnNavigation'...

    regards

    marc

  • Marcelo 18 posts 131 karma points
    Nov 19, 2018 @ 12:41
    Marcelo
    100

    Hi Marc,

    Thank you again for the reply.

    Then basically, the table cmsContentXml will only be updated after I "Save and Publish" the content ], right?

    If I wasn't using the Examine would be possible to check if the 'umbracoNaviHide' exists with 'HasProperty' method, but I intended to use the Examine to fast return All contents with 'umbracoNaviHide' setted to FALSE, here is my Examine Search Code:

    public IEnumerable<IPublishedContent> GetAllChildren(int startNodeId, bool onlyVisible = false)
            {
                var _search = Searcher.CreateSearchCriteria().Field("@searchPath", startNodeId.ToString());
    
                if (onlyVisible)
                    _search.And().Field("umbracoNaviHide", "0");
    
                var results = Searcher.Search(_search.Compile());
                foreach (var result in results)
                {
                    var node = UmbracoHelper.TypedContent(result.Id);
                    if (node != null)
                        yield return node;
                }
            }
    

    For this specific Examine code if the cmsContentXml table is not updated with the new property, the Examine won't index correctly and I get no results, my new question is: Is there any way to check if the property exists before to get the TypedContent() in Examine?

  • Marc Goodson 2155 posts 14408 karma points MVP 9x c-trib
    Nov 19, 2018 @ 19:46
    Marc Goodson
    0

    Hi Marcelo

    I wonder if it would be a possible workaround for you to hook into the Examine GatheringNodeData event, which fires for each item, at the point it is added to the index.

    And here, look at the properties on the item that is about to be indexed.

    Look for the value umbracoNaviHide

    and create a new field in the index called hiddenFromNavigation

    if umbracoNaviHide doesn't exist, set the value of hiddenFromNavigation to be 0, and if umbracoNaviHide does exist, then set the hiddenFromNavigation value to be the same.

    Once you rebuild your indexes, the event should fire for each item and this way your new hiddenFromNavigation index property will always have false for something that hasn't been saved and published, or be false for something that has been published, and isn't hidden...so you can consistently query against it?

      public class UmbracoBooter : IApplicationEventHandler
        {
           public void OnApplicationStarted(UmbracoApplicationBase 
        umbracoApplication, ApplicationContext applicationContext)
               {
                   var externalIndexSet = ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"];
                   externalIndexSet.GatheringNodeData += ExternalIndexProvider_GatheringNodeData;
               }
    
    
           }
    
         private void ExternalIndexProvider_GatheringNodeData(object sender, IndexingNodeDataEventArgs e)
               {
                  if (e.IndexType == IndexTypes.Content){
                          var hiddenFromNavigation = 0;
                          if (e.Fields.Keys.Contains("umbracoNaviHide")){
                                hiddenFromNavigation = e.Fields["umbracoNaviHide"];
                          }
                           e.Fields.Add("hiddenFromNavigation", hiddenFromNavigation);                
                   }
               }
    
  • Marcelo 18 posts 131 karma points
    Nov 19, 2018 @ 20:09
    Marcelo
    0

    Hi Marc,

    It's possible to use the 'GatheringNodeData' approach for this specific case of 'umbracoNaviHide' property, it completely make sense.

    For me to have an entirely solution, is there any event during the Document Type updates? And Is there a way to call the Examine Index update programatically?(because I know it is possible by button inside backoffice or deleting the cache folder inside the App_Data)

    I was wondering to hook to some event and when some Document Type changes some property, the event would call a Examine Index refresh. Is this make sense?

    Thank you

  • Marc Goodson 2155 posts 14408 karma points MVP 9x c-trib
    Nov 19, 2018 @ 23:01
    Marc Goodson
    0

    Hi Marcelo

    The ContentTypeService does have events see:

    https://our.umbraco.com/documentation/reference/events/contenttypeservice-events

    But say you had a new property you wanted to add, and somehow it had meaning that you wanted to infer, without saving and publishing the page it had been added to...

    eg perhaps 'hideFromSiteMap'

    You could perhaps detect in a ContentTypeService event that a new property had been added to a document type, but just triggering the rebuild of examine would not be enough, you'd need to also code the GatheringNode event, to interpret the absence of the new 'non published' field, and add a new index item, to be able to query, so you wouldn't be able to automate this... if this is what you were after.

    Essentially if you have this kind of property in the future, you would add it, and then work a way to code around its absence from items that are already published, usually it's not a problem as when you add a new property it's usually for an editor to add some value, and so therefore the page is saved and published, but this gatheringnode trick would probably work in scenarios where that wasn't the case.

    regards

    Marc

  • Marcelo 18 posts 131 karma points
    Nov 20, 2018 @ 11:18
    Marcelo
    0

    Hello Marc,

    Yeah, I see your concerns and you are right, I could even coding some way to create the 'GatheringNode' dynamically, but that would be a nightmare code to maintain in future. At the end, the best approach for my needs would be a way to republish all nodes from the website (only the already published nodes) so the Umbraco would refresh the Xml by it self :)

    Anyway, thank you so much for all your explainings, I will talk with my team to decide which coding way is better to fit our project.

    regards,

    Marcelo

Please Sign in or register to post replies

Write your reply to:

Draft