Hi Jonas

So just trying to figuring where this goes wrong, a couple of things you can do to help diagnose:

- If you do the packaging of the site, and dont deploy it, the packaged files are kept in a directory in /app_data/courier/revisions/__guid  (where the guid is random) the latest folder is your deployment. 

There are a number of things to look at in these files, but the first is to figure out if the search page has its dependencies in place, so if find the seach page in the /__guid/revision/documents folder, there should be some xml in that .courier file that states the dependencies of that document.

Is Static Content outside your siteroot or a child page? 

Also, look in courier.config, I think by default it doesnt force parents to be included, unless they are part of the deployment, (which in your case they should be anyway) but try setting that to true:

<documentItemProvider>
  <includeParents>true</includeParents>
</documentItemProvider>

Static Content is under the SiteRoot.

I did try setting 'includeParents' to true. No change in the behavior.

I'll get back to you on the Search page dependencies in a bit. ......

Search Page Info:

Search Page has ID of 1366, uniqueID of '88E49DAE-2F00-4EE4-AA91-2ABD1A043BA3',
and a parent of 1192 (SiteRoot), uniqueID = '7e1eea48-da7b-4906-a72c-3ace613f12ad',

In the Search_1366.courier file:

<Dependency>
--<IsChild>false</IsChild>
--<ItemId>
----<Id>7e1eea48-da7b-4906-a72c-3ace613f12ad</Id>
----<ProviderId>d8e6ad83-e73a-11df-9492-0800200c9a66</ProviderId>
--</ItemId>
--<Name>Parent Document: 7e1eea48-da7b-4906-a72c-3ace613f12ad</Name>
</Dependency>

(This is the SiteRoot node)

<Dependency>
--<IsChild>true</IsChild>
--<ItemId>
----<Id>88e49dae-2f00-4ee4-aa91-2abd1a043ba3</Id>
----<ProviderId>e0472594-e73b-11df-9492-0800200c9a66</ProviderId>
--</ItemId>
--<Name>Document Property data</Name>
</Dependency>

(This is a dependency to its properties)

<Dependency>
--<IsChild>false</IsChild>
--<ItemId>
----<Id>StandardPage</Id>
----<ProviderId>25867200-e67e-11df-9492-0800200c9a66</ProviderId>
--</ItemId>
--<Name>Template: StandardPage</Name>
</Dependency>
<Dependency>
--<IsChild>false</IsChild>
--<ItemId>
----<Id>StandardPage</Id>
----<ProviderId>d8e6ad84-e73a-11df-9492-0800200c9a66</ProviderId>
--</ItemId>
--<Name>Document type: StandardPage</Name>
</Dependency>

(The Search Page is a "Standard Page")

Also, the macro Search.cshtml (which is the Search textbox on every page) has an input parameter which is a content picker, and its value in the master.master file is set to the Search Page node (1366). In the Courier Revision folder, the reference to this macro in the Master page is:

&lt;umbraco:Macro Alias="Search" runat="server" SearchPage="88e49dae-2f00-4ee4-aa91-2abd1a043ba3" /&gt

Note that the Search page GUID is correctly set here. In the destination site, this value is converted to the correct ID, it's just that the page is not published, and it's not set to be a child of the SiteRoot node.

Any suggestions?

Per my client's request, I have submitted a support ticket to the Courier Helpdesk for this issue.

Well, no-one has responded to my support ticket for a week now. My client is getting pretty upset at the lack of support.

Due to the rollback issue is Umbraco 4.9.1, I upgraded Umbraco to 4.11.5. Using this release, there is no release of Courier that I have gotten to work.

I tried various version of the Courier, including nightly builds:

Using .80 release on both servers  - crashes right at the start.

Using .77 release on both servers - Crashes right at the start of transfer

Using .53 release on both servers - images/pdf all transferred fine
Content - errors out

Try .53 release again - images/pdf all transferred fine
- Content failed near end
  - Data was there, not all parented correctly or published

Run Content transfer agaion - Failed near end
 - Content all transferred, but MasterPages not updated with new IDs
 - Can View Site? no

Downgrade to .39 (original) release on both servers
- images.pdf transfer as 0-byte files (known bug)

Using .39 (original) release on both servers - delete Umbraco.Courier.CacheHandler.V4.dll
 - img/pdf files trasnferred as 0-byte files

Using .39 (original) release on both servers - new cache dll from .53 nightly
 - img/pdf files trasnferred as 0-byte files

Using .39 (original) release on both servers - new cache dll from .77 nightly
 - img/pdf files trasnferred as 0-byte files

The last few tests tried to fix the 0-byte file transfers, with no luck.

At this point, I am going to revert Umbraco back to 4.9.1, which seemed to at least work with mutiple deployment passes. This version of Umbraco, however, has a bug in the Rollback command that the upgrade addressed. Maybe we can live with that bug, as long as Courier (sort of) works with that release in our environment. The thing that worries me is that in the initial tests, I didn't check for 0-byte files, not knowing about the issue. If that still happens in the original setup, then we are SOL for a Courier solution. Our only fallback is to do weekly full backup/restore of the 2 Umbraco databases as our method of publishing to the production site.

The client is willing to pay for some support, but it's not encouraging that even a HelpDesk submission gets no response.

Final attempt:

Umbraco 4.11.5, Courier 2.7.3.39 - By transferring small pieces at a time, the only problem that I cannot oversome is the 0-byte files (Courier truncates file to 0 bytes). According to http://our.umbraco.org/forum/umbraco-pro/courier/37560-Courier-empties-all-files, Per suggests replacing just the CacheHandler DLL from the Jan 16th release. This made no difference in my environment. I know that this is a race condition, so it may not even happen in production, but that's small consolation to a client that needs a reliable system.

Basicly, all items should handle their own dependencies, so if item A depends on B,C and D, then courier *should* arrange this in the dependency tree that it creates during packaging and extraction.

If the dependencies are not packaged, as in not found the packaged folder, then the issue happens when courier tries to pull data from the DB, but this should get into your error_log - if you have debugmode turned on.

if the items actually do get into the folder, but doesnt get loaded during the extraction phase, then an exception makes the whole procress exit before it gets to these items, again, that should be in the error log.

2.7.5 has been pushed out, which fixes the race-condition and also has the right set of dlls which is an issue on many of the nightly builds due the transition to multiple version-support (for v4 and v6 schemas) 

/per

Oh, one thing! does the document types of these documents with the missing dependencies have special datatypes? types which depends on the httpcontext or some external dll? - this could make the packaging process exit unexpected as it cannot handle these things in a background thread.

Thanks for the reply.

From all I can see, the dependencies get packaged up correctly, but the failures comes during the loading stage.

When I started this project, I realized that non-standard data types would not be supported, so I limited myself to the basic ones, with the only addition being uComponents URLPicker (which I know transfers just fine). The only difference between nodes that transferred fine vs the failures is that all of the ones that failed the final re-parenting are items that only are referenced from the master template - header menu, footer menu, a set of quick link menus, and the search results page. All of these are references in Master.master by their node ID, and the data type is set to Content Picker, so Courier knows to re-map the IDs in the mast er file. In the case of the Search results page, it is also passed in by ID to the Search Control on the master page, also set to Content Picker.

Unfortunately, the errors are never very useful to me, not having the source:

3/13/2013 1:20:29 PM; System.NullReferenceException: Object reference not set to an instance of an object.
   at Umbraco.Courier.Core.Storage.RevisionStorage.#hd(String folderName, String path, Revision& revision)
   at Umbraco.Courier.Core.Storage.RevisionStorage.GetFromDirectory(String name, String directory, Boolean lazyLoad)
   at Umbraco.Courier.UI.Dialogs.CommitItem.#Ti(String id)
   at Umbraco.Courier.UI.Dialogs.CommitItem.GetProviderItems(String providerId, String revision)

I can't find any problems in the data itself - all nodes are published, Mandatory fields filled, links working on the source site, etc.

For now, I have advised the client to simply start with identical databases and sites (via SQL Server backups and copying the media folder), then using Courier for the incremental transfers as changes are done (possibly re-synching on a periodic basis). Keeping the list of items to be transferred short insures reliable transfers.

I'm glad to hear that the race condition bug is out in a full release. I was able to get files to transfer fine with the build after .39, but there seemed to be other issues.

We'll test and see. Thanks.

Okey, if the situation doest improve, I can take a lot at the generated output from courier, to see if I can deploy the files you packaged up and see if I ca spot any obvious issues

is working on a reply...