Feedback needed: improved dependency checking and loading
Going over some sites, submitted by Courier users, and trying to finetune the way Courier handles dependencies, to focus on getting as few items include, while still include needed dependencies for the site to function.
So going over standard use-cases, I see a possible change in this, to make it more efficient.
So the way it works now: Courier is provided with a number of selected items to package, these are known as an "item", so courier package these, and the package returns with a list of dependencies, it detects are needed for that specific item to function, (a doc needs a doctype, template and so on) so these dependencies are processed the same way, and each dependency returns a list of dependencies, this specific dependency needs to function. This is what happens when you pick a single document and are asked to transfer 40 items...
To avoid too many unneeded dependencies, Courier can compare against the target of the deployment, so if a dependency is unchanged, it is not included in the deployment, But we still need to load the dependencies of the dependency, as they could have changed. So, Item requires dependency, dependency is skipped if identical, but dependencies of the dependency is loaded into the graph to see if they have changed.
There is also a small variation with content and property data, as property data is an item by themselfs, but content cannot function without property data, so these are always added to graph, no matter what.
-----
So this the scenario I wish we could change to:
Items are loaded as normal, compared to the target of the deployment, and skipped if identical.
Change: if an item has a hard dependency like doc -> property data, it will include the property data in the comparison, so when considering if a document is changed it will look at doc+properties instead of just the document. - so if either doc or properties have changed, both will be included to transfer the complete document, all the time.
Change: An item requires a dependency, courier determines the dependency has changed, so it loads the dependency and queues the dependencies of the dependency. If the dependency has not changed, it will not include it, nor will it queue its children, where as now, it still queues the dependency dependencies.
So a common scenario, you transfer a document, here courier will look at the property data for comparison, then it will queue its doctype, template and parent document. if doctype is changed, it will go through its dependencies, if not, it wont, same with template, parent doc and so on. So in most cases, you would only transfer doc + properties, but if a dependency did change, it will continue down including the chain of dependencies, untill they stop changing.
----
Note: this does not affect the way revisions are done in the courier section since it doesnt package items against a known target, it simply collects everything needed for the selected items to function.
Hi Per, it definitely makes sense as right now Courier shows way much more items to transfer than actually needed.
Related question: Sometimes when transferring a changed document, Courier suggests to transfer (along with templates, macros etc) another documents, which are not seem to be related in any way to the originally changed document. I have no sensible explanation why could this happen. Is there a way to fine-tune this in the current version, or this will be addressed by the change your are suggesting?
It's because courier finds linked documents in templates and rich text editors, so if you link to a page from another courier includes the page to ensure that it can convert the id in the markup to a guid and back again, otherwise your links will break.
Yeah, we can add some additional information on which items are skipped, but we can't show everything, since if an item is skipped based, then you will not see the child items of it in the overview, since courier never processes them, and therefore doesnt actually know they could eventually be included
Feedback needed: improved dependency checking and loading
Going over some sites, submitted by Courier users, and trying to finetune the way Courier handles dependencies, to focus on getting as few items include, while still include needed dependencies for the site to function.
So going over standard use-cases, I see a possible change in this, to make it more efficient.
So the way it works now:
Courier is provided with a number of selected items to package, these are known as an "item", so courier package these, and the package returns with a list of dependencies, it detects are needed for that specific item to function, (a doc needs a doctype, template and so on) so these dependencies are processed the same way, and each dependency returns a list of dependencies, this specific dependency needs to function.
This is what happens when you pick a single document and are asked to transfer 40 items...
To avoid too many unneeded dependencies, Courier can compare against the target of the deployment, so if a dependency is unchanged, it is not included in the deployment, But we still need to load the dependencies of the dependency, as they could have changed. So, Item requires dependency, dependency is skipped if identical, but dependencies of the dependency is loaded into the graph to see if they have changed.
There is also a small variation with content and property data, as property data is an item by themselfs, but content cannot function without property data, so these are always added to graph, no matter what.
-----
So this the scenario I wish we could change to:
Items are loaded as normal, compared to the target of the deployment, and skipped if identical.
Change: if an item has a hard dependency like doc -> property data, it will include the property data in the comparison, so when considering if a document is changed it will look at doc+properties instead of just the document. - so if either doc or properties have changed, both will be included to transfer the complete document, all the time.
Change: An item requires a dependency, courier determines the dependency has changed, so it loads the dependency and queues the dependencies of the dependency. If the dependency has not changed, it will not include it, nor will it queue its children, where as now, it still queues the dependency dependencies.
So a common scenario, you transfer a document, here courier will look at the property data for comparison, then it will queue its doctype, template and parent document. if doctype is changed, it will go through its dependencies, if not, it wont, same with template, parent doc and so on. So in most cases, you would only transfer doc + properties, but if a dependency did change, it will continue down including the chain of dependencies, untill they stop changing.
----
Note: this does not affect the way revisions are done in the courier section since it doesnt package items against a known target, it simply collects everything needed for the selected items to function.
Hi Per, it definitely makes sense as right now Courier shows way much more items to transfer than actually needed.
Related question: Sometimes when transferring a changed document, Courier suggests to transfer (along with templates, macros etc) another documents, which are not seem to be related in any way to the originally changed document. I have no sensible explanation why could this happen. Is there a way to fine-tune this in the current version, or this will be addressed by the change your are suggesting?
It's because courier finds linked documents in templates and rich text editors, so if you link to a page from another courier includes the page to ensure that it can convert the id in the markup to a guid and back again, otherwise your links will break.
You can turn some if off, by disabling locallinks as dependencies tho: https://github.com/umbraco/Courier/blob/master/Documentation/Architecture/Provider-Configuration.md#templates
Unfortunately it's not the case - the documents are completely unrelated - the original item simply does not have any locallinks or pickers.
You can see this issue was reported also here: http://our.umbraco.org/forum/umbraco-pro/courier/43665-Transferring-nodes-with-no-dependencies-copies-random-items!
Would it be possible for the UI (and possibly a log) to show "Skipped packaging dependency: Folder (doctype), because target is already up to date"?
Yeah, we can add some additional information on which items are skipped, but we can't show everything, since if an item is skipped based, then you will not see the child items of it in the overview, since courier never processes them, and therefore doesnt actually know they could eventually be included
is working on a reply...