I am new in Umbraco world and I am developing my first project usign version 7.12.3, but I ended up with a question that even after 2 days searching in google I could not find a solution or workaround.
I will try to explain in a very clear way:
On my Umbraco Backoffice, I have:
Home (document type, composed with WebsiteBase)
WebsiteBase (document type - which I use for compositions).
[Another documents types]
So basically, I have created my website at Content section with many childrens, everything ok, as I am going to have many contents, I started to use the Examine for some searchs, which is working gracefully too.
Then, I figured out that I was needing a new Document Type property, I went to the Document Type 'WebsiteBase' and created the property 'umbracoNaviHide' and changed my Examine search to check this new property! But I notice that the Examine was indexing the Contents without that new property that I have just created.
To try to refresh the Contents, I did:
Click with right button over 'Content root' and click on 'Republish entire site'.
Delete the file 'umbraco.config' and restart the umbraco application.
Inside the ApplicationEventHandler class at ApplicationStarted method I writed the following rows:
Those tries recreated the umbraco.config file, but ALWAYS without the new property created inside the Document Type. The ONLY way to get the new property inside the cmsContentXML table was to go to each website page and click the button 'Save and Publish', after click that button the cmsContentXml table and umbraco.config (by consequence) were refreshed for that specific content id.
Okay, It is possible to do this manual procedure for each website page, but my question is, what about tomorrow? If I need to create a new document type property again, with 400 hundred contents Am I going to have to Publish by EACH page?
Does not exist a easy way to republish all contents after a new document type property insert?
Which 'should' be clearing the cmsContentXML table... I wonder if it's timing out and rolling back?
Did you pass in just the ids of the ContentType that had changed?
RebuildXmlStructures(params int[] contentTypeIds)
It would be interesting to see if you removed manually the entries, and ran the Data Integrity healthcheck (although this should be same as calling them in code) whether the entries are recreated (with the new property)
Essentially when you change a document type, this should trigger the removal of any entries from cmsContentXml table (using the methods above).
After an upgrade from v6 to v7, I'll often empty this table completely and rebuild from the data integrity healthcheck, but in the scenario you describe I'd expect it to work.
Once the cmsContentXml table is refreshed, then the Umbraco.Config cache file on disk and examine indexes can be rebuilt, as these take the cms ContentXml table as their source...
Yes,indeed the RePublishAll() is calling RebuildXmlStructures() internally, I ended up trying every method that I found inside the 'ContentService' object.
On my initial tests I didn't pass the Document Type Id that I have changed, because my intention was to rebuild all the website doesn't mattering if it has changed or not.
No lucky, I checked the umbraco.config file and the cmsContentXml table but the Contents ids 1171 and 1103 continue without the new property 'umbracoNaviHide'.
The last test I did was to truncate the cmsContentXml table and delete the umbraco.config file, when I restarted the application everything was created again, but still without the 'umbracoNaviHide' property :(
The only way that I am able to refresh the cmsContentXml table is when I go to the Contents root and I choose one page and I click 'Save and publish' button. By doing this procedure, the table cmsContentXml is updated for that specific page with the new property 'umbracoNaviHide' and the 'umbraco.config' file is updated after I stop and start the application again.
The ids that you pass to RebuildXmlStructures should be the ids of the DocumentType's that have changed, and not the ids of the Content Items, if that makes sense?
(but also this should just 'work')
What I'm interested in is if you delete one entry from the cmsXMLContent table, and then run the Data integrity healthcheck, interested to see if it detects the missing item, and if rebuilding from the healthcheck dashboard, causes the recreation with the new property.
Otherwise is there anything in the trace logs - app_data/logs during the rebuild process that would give a clue?
I think I understand more what you mean, when RebuildXmlStructures is called, it will recreate the entry in cmxContentXml table, but it will take this entry from the database for the current 'published version' if the item (and if the db's published version, matches the version in the Umbraco cache, it seems to me it will use that)
Now your current saved version of your item won't have an umbracoNaviHide property, so I think that's why RebuildXmlStructures won't add one in... but when you 'load' the item into the Umbraco backoffice to edit, and press Save and Publish, then the new umbracoNaviHide property is 'saved' and therefore added to the Xml Representation of the object.
(if you delete one of the entries from cmsContentXml that has already gained the umbracoNaviHide property, and trigger RebuildXmlStructures - then the entry should be recreated with the umbracoNaviHide property as when you saved and published the item previously, the underlying published database entry was updated!)
In your case of adding an umbracoNaviHide property, I suppose this isn't too bad a consequence, as in order to 'tick' an item as requiring to be hidden, you will need to save and publish the page, thus updating the content store and the cmsContentXml table, when using .visible() helper to check if an item is hidden, the absence of the property should return false.
You can check if a property exists using the HasProperty helper.
So in future if you add other properties to your doc types, and the page is already published, then only saving and publishing from the backoffice will add the new property to that items published cache, and be wary to 'code around the scenario' where the property might not exist... which would be a big problem if your checkbox logic was around the other way and called 'ShowOnNavigation'...
Then basically, the table cmsContentXml will only be updated after I "Save and Publish" the content ], right?
If I wasn't using the Examine would be possible to check if the 'umbracoNaviHide' exists with 'HasProperty' method, but I intended to use the Examine to fast return All contents with 'umbracoNaviHide' setted to FALSE, here is my Examine Search Code:
public IEnumerable<IPublishedContent> GetAllChildren(int startNodeId, bool onlyVisible = false)
{
var _search = Searcher.CreateSearchCriteria().Field("@searchPath", startNodeId.ToString());
if (onlyVisible)
_search.And().Field("umbracoNaviHide", "0");
var results = Searcher.Search(_search.Compile());
foreach (var result in results)
{
var node = UmbracoHelper.TypedContent(result.Id);
if (node != null)
yield return node;
}
}
For this specific Examine code if the cmsContentXml table is not updated with the new property, the Examine won't index correctly and I get no results, my new question is: Is there any way to check if the property exists before to get the TypedContent() in Examine?
I wonder if it would be a possible workaround for you to hook into the Examine GatheringNodeData event, which fires for each item, at the point it is added to the index.
And here, look at the properties on the item that is about to be indexed.
Look for the value umbracoNaviHide
and create a new field in the index called hiddenFromNavigation
if umbracoNaviHide doesn't exist, set the value of hiddenFromNavigation to be 0, and if umbracoNaviHide does exist, then set the hiddenFromNavigation value to be the same.
Once you rebuild your indexes, the event should fire for each item and this way your new hiddenFromNavigation index property will always have false for something that hasn't been saved and published, or be false for something that has been published, and isn't hidden...so you can consistently query against it?
public class UmbracoBooter : IApplicationEventHandler
{
public void OnApplicationStarted(UmbracoApplicationBase
umbracoApplication, ApplicationContext applicationContext)
{
var externalIndexSet = ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"];
externalIndexSet.GatheringNodeData += ExternalIndexProvider_GatheringNodeData;
}
}
private void ExternalIndexProvider_GatheringNodeData(object sender, IndexingNodeDataEventArgs e)
{
if (e.IndexType == IndexTypes.Content){
var hiddenFromNavigation = 0;
if (e.Fields.Keys.Contains("umbracoNaviHide")){
hiddenFromNavigation = e.Fields["umbracoNaviHide"];
}
e.Fields.Add("hiddenFromNavigation", hiddenFromNavigation);
}
}
It's possible to use the 'GatheringNodeData' approach for this specific case of 'umbracoNaviHide' property, it completely make sense.
For me to have an entirely solution, is there any event during the Document Type updates? And Is there a way to call the Examine Index update programatically?(because I know it is possible by button inside backoffice or deleting the cache folder inside the App_Data)
I was wondering to hook to some event and when some Document Type changes some property, the event would call a Examine Index refresh. Is this make sense?
But say you had a new property you wanted to add, and somehow it had meaning that you wanted to infer, without saving and publishing the page it had been added to...
eg perhaps 'hideFromSiteMap'
You could perhaps detect in a ContentTypeService event that a new property had been added to a document type, but just triggering the rebuild of examine would not be enough, you'd need to also code the GatheringNode event, to interpret the absence of the new 'non published' field, and add a new index item, to be able to query, so you wouldn't be able to automate this... if this is what you were after.
Essentially if you have this kind of property in the future, you would add it, and then work a way to code around its absence from items that are already published, usually it's not a problem as when you add a new property it's usually for an editor to add some value, and so therefore the page is saved and published, but this gatheringnode trick would probably work in scenarios where that wasn't the case.
Yeah, I see your concerns and you are right, I could even coding some way to create the 'GatheringNode' dynamically, but that would be a nightmare code to maintain in future. At the end, the best approach for my needs would be a way to republish all nodes from the website (only the already published nodes) so the Umbraco would refresh the Xml by it self :)
Anyway, thank you so much for all your explainings, I will talk with my team to decide which coding way is better to fit our project.
How to repubish the entirely website contents, after include a new document type property?
Hello,
I am new in Umbraco world and I am developing my first project usign version 7.12.3, but I ended up with a question that even after 2 days searching in google I could not find a solution or workaround.
I will try to explain in a very clear way:
On my Umbraco Backoffice, I have:
So basically, I have created my website at Content section with many childrens, everything ok, as I am going to have many contents, I started to use the Examine for some searchs, which is working gracefully too.
Then, I figured out that I was needing a new Document Type property, I went to the Document Type 'WebsiteBase' and created the property 'umbracoNaviHide' and changed my Examine search to check this new property! But I notice that the Examine was indexing the Contents without that new property that I have just created.
To try to refresh the Contents, I did:
Those tries recreated the umbraco.config file, but ALWAYS without the new property created inside the Document Type. The ONLY way to get the new property inside the cmsContentXML table was to go to each website page and click the button 'Save and Publish', after click that button the cmsContentXml table and umbraco.config (by consequence) were refreshed for that specific content id.
Okay, It is possible to do this manual procedure for each website page, but my question is, what about tomorrow? If I need to create a new document type property again, with 400 hundred contents Am I going to have to Publish by EACH page?
Does not exist a easy way to republish all contents after a new document type property insert?
Hi Marcelo
I think RePublishAll, calls RebuildXmlStructures
https://github.com/umbraco/Umbraco-CMS/blob/2e135d2006212df799765552bc0e916b9711cee9/src/Umbraco.Core/Services/ContentService.cs#L1001
https://github.com/umbraco/Umbraco-CMS/blob/2e135d2006212df799765552bc0e916b9711cee9/src/Umbraco.Core/Services/ContentService.cs#L2202
Which 'should' be clearing the cmsContentXML table... I wonder if it's timing out and rolling back?
Did you pass in just the ids of the ContentType that had changed?
RebuildXmlStructures(params int[] contentTypeIds)
It would be interesting to see if you removed manually the entries, and ran the Data Integrity healthcheck (although this should be same as calling them in code) whether the entries are recreated (with the new property)
Essentially when you change a document type, this should trigger the removal of any entries from cmsContentXml table (using the methods above).
After an upgrade from v6 to v7, I'll often empty this table completely and rebuild from the data integrity healthcheck, but in the scenario you describe I'd expect it to work.
Once the cmsContentXml table is refreshed, then the Umbraco.Config cache file on disk and examine indexes can be rebuilt, as these take the cms ContentXml table as their source...
Hello Marc,
First of all, thank you for your reply.
Yes,indeed the RePublishAll() is calling RebuildXmlStructures() internally, I ended up trying every method that I found inside the 'ContentService' object.
On my initial tests I didn't pass the Document Type Id that I have changed, because my intention was to rebuild all the website doesn't mattering if it has changed or not.
Anyway I just did this test:
No lucky, I checked the umbraco.config file and the cmsContentXml table but the Contents ids 1171 and 1103 continue without the new property 'umbracoNaviHide'.
The last test I did was to truncate the cmsContentXml table and delete the umbraco.config file, when I restarted the application everything was created again, but still without the 'umbracoNaviHide' property :(
The only way that I am able to refresh the cmsContentXml table is when I go to the Contents root and I choose one page and I click 'Save and publish' button. By doing this procedure, the table cmsContentXml is updated for that specific page with the new property 'umbracoNaviHide' and the 'umbraco.config' file is updated after I stop and start the application again.
Hi Marcelo
The ids that you pass to RebuildXmlStructures should be the ids of the DocumentType's that have changed, and not the ids of the Content Items, if that makes sense?
(but also this should just 'work')
What I'm interested in is if you delete one entry from the cmsXMLContent table, and then run the Data integrity healthcheck, interested to see if it detects the missing item, and if rebuilding from the healthcheck dashboard, causes the recreation with the new property.
Otherwise is there anything in the trace logs - app_data/logs during the rebuild process that would give a clue?
regards
marc
Hi Marc,
Regarding the Health ckeck test, I think I did what you have said, I will explain step by step whith print screens of what I did:
1 - I found a line inside the cmsContentXml which has not the new property called 'umbracoNaviHide':
2 - I stoped the application and I executed this query inside the database:
3 - I started the application and opened the Health Check > Data Integrity:
4 - I clicked the green button and the Umbraco fixed my integrity error:
5 - I executed the bellow query inside the database:
6 - The result was:
I believe it is not rebuilding the XML correctly.
After this test, I went to the Content section to confirm if the new property was visible inside the backoffice, which it is:
The I clicked the button "Save and Publish" check the Content Info History print:
I executed the bellow query inside the database:
And now voilà! the new property appeared inside the cmsContentXml table:
The only way I can have the Xml update is by clicking the button "Save and Publish" :(
Am I missing something?
Hi Marcelo
I think I understand more what you mean, when RebuildXmlStructures is called, it will recreate the entry in cmxContentXml table, but it will take this entry from the database for the current 'published version' if the item (and if the db's published version, matches the version in the Umbraco cache, it seems to me it will use that)
https://github.com/umbraco/Umbraco-CMS/blob/c27b9d76dbb0c72a700a6af13f4b50c50994a4b8/src/Umbraco.Core/Persistence/Repositories/ContentRepository.cs#L216
Now your current saved version of your item won't have an umbracoNaviHide property, so I think that's why RebuildXmlStructures won't add one in... but when you 'load' the item into the Umbraco backoffice to edit, and press Save and Publish, then the new umbracoNaviHide property is 'saved' and therefore added to the Xml Representation of the object.
(if you delete one of the entries from cmsContentXml that has already gained the umbracoNaviHide property, and trigger RebuildXmlStructures - then the entry should be recreated with the umbracoNaviHide property as when you saved and published the item previously, the underlying published database entry was updated!)
In your case of adding an umbracoNaviHide property, I suppose this isn't too bad a consequence, as in order to 'tick' an item as requiring to be hidden, you will need to save and publish the page, thus updating the content store and the cmsContentXml table, when using .visible() helper to check if an item is hidden, the absence of the property should return false.
You can check if a property exists using the HasProperty helper.
So in future if you add other properties to your doc types, and the page is already published, then only saving and publishing from the backoffice will add the new property to that items published cache, and be wary to 'code around the scenario' where the property might not exist... which would be a big problem if your checkbox logic was around the other way and called 'ShowOnNavigation'...
regards
marc
Hi Marc,
Thank you again for the reply.
Then basically, the table cmsContentXml will only be updated after I "Save and Publish" the content ], right?
If I wasn't using the Examine would be possible to check if the 'umbracoNaviHide' exists with 'HasProperty' method, but I intended to use the Examine to fast return All contents with 'umbracoNaviHide' setted to FALSE, here is my Examine Search Code:
For this specific Examine code if the cmsContentXml table is not updated with the new property, the Examine won't index correctly and I get no results, my new question is: Is there any way to check if the property exists before to get the TypedContent() in Examine?
Hi Marcelo
I wonder if it would be a possible workaround for you to hook into the Examine GatheringNodeData event, which fires for each item, at the point it is added to the index.
And here, look at the properties on the item that is about to be indexed.
Look for the value umbracoNaviHide
and create a new field in the index called hiddenFromNavigation
if umbracoNaviHide doesn't exist, set the value of hiddenFromNavigation to be 0, and if umbracoNaviHide does exist, then set the hiddenFromNavigation value to be the same.
Once you rebuild your indexes, the event should fire for each item and this way your new hiddenFromNavigation index property will always have false for something that hasn't been saved and published, or be false for something that has been published, and isn't hidden...so you can consistently query against it?
Hi Marc,
It's possible to use the 'GatheringNodeData' approach for this specific case of 'umbracoNaviHide' property, it completely make sense.
For me to have an entirely solution, is there any event during the Document Type updates? And Is there a way to call the Examine Index update programatically?(because I know it is possible by button inside backoffice or deleting the cache folder inside the App_Data)
I was wondering to hook to some event and when some Document Type changes some property, the event would call a Examine Index refresh. Is this make sense?
Thank you
Hi Marcelo
The ContentTypeService does have events see:
https://our.umbraco.com/documentation/reference/events/contenttypeservice-events
But say you had a new property you wanted to add, and somehow it had meaning that you wanted to infer, without saving and publishing the page it had been added to...
eg perhaps 'hideFromSiteMap'
You could perhaps detect in a ContentTypeService event that a new property had been added to a document type, but just triggering the rebuild of examine would not be enough, you'd need to also code the GatheringNode event, to interpret the absence of the new 'non published' field, and add a new index item, to be able to query, so you wouldn't be able to automate this... if this is what you were after.
Essentially if you have this kind of property in the future, you would add it, and then work a way to code around its absence from items that are already published, usually it's not a problem as when you add a new property it's usually for an editor to add some value, and so therefore the page is saved and published, but this gatheringnode trick would probably work in scenarios where that wasn't the case.
regards
Marc
Hello Marc,
Yeah, I see your concerns and you are right, I could even coding some way to create the 'GatheringNode' dynamically, but that would be a nightmare code to maintain in future. At the end, the best approach for my needs would be a way to republish all nodes from the website (only the already published nodes) so the Umbraco would refresh the Xml by it self :)
Anyway, thank you so much for all your explainings, I will talk with my team to decide which coding way is better to fit our project.
regards,
Marcelo
is working on a reply...