I didn't want to mix this question in with my other one.
If I push content from my local machine to the staging machine that is in AWS, there seems to be a unix folder/path problem.
When I list the contents of the folder on the server, there are files with a backslash in the name.
[ec2-user@ip- 9d3bb626-088b-440f-9576-7b615066b671]$ ls -l total 20
-rw-r--r-- 1 webapp webapp 247 Jan 24 10:46 _dependencies.config
-rw-r--r-- 1 webapp webapp 105 Jan 24 10:46 _items.config
-rw-r--r-- 1 webapp webapp 504 Jan 24 10:46 uSync\Media\217fb473-54d9-4a1b-a7b0-1a0fba42f66a.config
-rw-r--r-- 1 webapp webapp 187 Jan 24 10:46 uSync\Media\46e307fd-223f-4523-9f1b-af353b9eb45c_clean.config
-rw-r--r-- 1 webapp webapp 720 Jan 24 10:46 uSync\Media\46e307fd-223f-4523-9f1b-af353b9eb45c.config
Error vm.errorTitle Error while processing Push - (export/Report/0/)
[InternalServerError] Folder :
[9d3bb626-088b-440f-9576-7b615066b671/uSync] does not exsit when it
should Error while processing Push - (export/Report/0/)
[InternalServerError] Folder :
[9d3bb626-088b-440f-9576-7b615066b671/uSync] does not exsit when it
should Action: export Step: Report (5) PageNumber: 0 ClassName:
System.IO.DirectoryNotFoundException: Folder :
[9d3bb626-088b-440f-9576-7b615066b671/uSync] does not exsit when it
should ---> System.Exception: Folder
/var/app/current/umbraco/Data/TEMP/uSync/receive/9d3bb626-088b-440f-9576-7b615066b671/uSync
does not exist Server role Single --- End of inner exception stack
trace --- at
uSync.Expansions.Core.Physical.SyncPackFileService.EnsurePackFolder(String
folder) at
uSync.Expansions.Core.Services.SyncPackService.EnsurePackFolder(SyncPackRequest
request) at
uSync.Expansions.Core.Services.SyncPackService.ReportChanges(SyncPackRequest
request) at
uSync.Publisher.Services.SyncPublishService.ReportChanges(SyncPublishRequest
request) at
uSync.Publisher.Controllers.uSyncReceiveApiController.DoPublisherStep(PublisherActionRequest
options, Func`2 action) at
uSync.Publisher.Controllers.uSyncReceiveApiController.ReportPackFolder(PublisherActionRequest
options) at lambdamethod3286(Closure , Object , Object[] ) at
Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper
mapper, ObjectMethodExecutor executor, Object controller, Object[]
arguments) at
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
at
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State&
next, Scope& scope, Object& state, Boolean& isCompleted) at
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed
context) at
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State&
next, Scope& scope, Object& state, Boolean& isCompleted) at
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Awaited|250(ResourceInvoker
invoker, Task lastTask, State next, Scope scope, Object state, Boolean
isCompleted)
at uSync.Publisher.Extensions.HttpClientExtensions.EnsureRemoteResponse(HttpResponseMessage
response)
at
uSync.Publisher.Publishers.SyncRealtimePublisher.RunRemotePublishRequest(PublisherActionRequest
request, String method)
at
uSync.Publisher.Publishers.SyncRealtimePublisher.RunRemotePublishRequest(PublisherActionRequest
request, String method)
at
uSync.Publisher.Publishers.SyncRealtimePublisher.PushRunReport(PublisherActionRequest
request)
at
uSync.Publisher.Controllers.uSyncPublisherApiController.PerformAction(PublisherActionRequest
request)
Looking at the msg not sure the \ are the problem (i think that is part of the zip folder creation) at least not the main error.
looks like missing the root of the temp folder uSync is using to zip things up, /var/app/current/umbraco/Data/TEMP/uSync/receive/9d3bb626-088b-440f-9576-7b615066b671/uSync
can you confirm the site has write permissions to /umbraco/Data/TEMP ? and if you can can you confirm there is a uSync/receive folder in there (you don't need to create it but uSync will , so if its not there its likely permissions).
Hi Andrew, just looking now, i wonder if you can take a look at some logging on the server ?
when the files are sent across the wire they are/should be unzipped into the guid folder , and i suspect that's the issue, the error you are then seeing is because we check for the existence of the uSync folder at the start of the next call.
anywhay if you have debug/trace/verbose logging on the server you should see a couple of entried in the logs as the files are extracted ...
in the code that unzips the files we have :
logger.LogDebug("Unzip: {file} {entries} Entries to [{target}]", zipfile, zip.Entries.Count, target);
and for each file we also do this :
logger.LogTrace("Extracting {dest}", dest);
so it would be good to see if those log entries make any sense on the target server for you.
as we build the entry manually, i think its us putting the \ in, but we can fix that.
I will do some quick checks that it doesn't then break windows, and push out an nightly so you can test this. It might not be the issue (i have seen this work on linux, so maybe its distro dependent), but we will see.
I noticed that when comparing content pulled from the linux machine to my windows machine, it was showing different kinds of line endings \r \n as the only difference in the content.
I think i have a fix for that, but its going to take a little bit more testing because it effects how we serialize the xml in memory, and while it shouldn't make to much difference, we do that *a lot so i want to make sure we don't bust something else.
uSync publisher on linux : Error while processing Push - (export/Report/0/) .. Folder : does not exsit when it should
Hi,
I didn't want to mix this question in with my other one.
If I push content from my local machine to the staging machine that is in AWS, there seems to be a unix folder/path problem.
When I list the contents of the folder on the server, there are files with a backslash in the name.
Hi Andrew, quick check on this one are you running the latest version of uSync (v9.0.6).
as there was a fix for the way Linux was zipping and un-zippling files and i think that is this issue.
https://github.com/Jumoo/uSync.Complete.Issues/releases/tag/v9.0.6
Hi Kevin,
Running 9.0.6
Info from OS in case useful
cat /etc/os-release
NAME="Amazon Linux" VERSION="2" ID="amzn" IDLIKE="centos rhel fedora" VERSIONID="2" PRETTYNAME="Amazon Linux 2" ANSICOLOR="0;33" CPENAME="cpe:2.3:o:amazon:amazonlinux:2" HOME_URL="https://amazonlinux.com/"
Hi
Looking at the msg not sure the \ are the problem (i think that is part of the zip folder creation) at least not the main error.
looks like missing the root of the temp folder uSync is using to zip things up,
/var/app/current/umbraco/Data/TEMP/uSync/receive/9d3bb626-088b-440f-9576-7b615066b671/uSynccan you confirm the site has write permissions to /umbraco/Data/TEMP ? and if you can can you confirm there is a uSync/receive folder in there (you don't need to create it but uSync will , so if its not there its likely permissions).
It creates the folder all the way to the guid (tried again, so new guid)
/var/app/current/umbraco/Data/TEMP/uSync/receive/120e7668-fd85-4907-85e4-be5374695142/
if I
cd /var/app/current/umbraco/Data/TEMP/uSync/receive/120e7668-fd85-4907-85e4-be5374695142/
then
ls -l
there is no uSync folder
😞 I'll have to have a look around see if i can replicate and maybe workout what is happening
Is there anything I can provide that would help?
Hi Andrew, just looking now, i wonder if you can take a look at some logging on the server ?
when the files are sent across the wire they are/should be unzipped into the guid folder , and i suspect that's the issue, the error you are then seeing is because we check for the existence of the uSync folder at the start of the next call.
anywhay if you have debug/trace/verbose logging on the server you should see a couple of entried in the logs as the files are extracted ...
in the code that unzips the files we have :
and for each file we also do this :
so it would be good to see if those log entries make any sense on the target server for you.
The guide folder is create, but there isn't a uSync folder under it.
The 2 media config items have names with backslashes in them which seems odd to me, but I don't have much with non-windows filesystems.
I think the paths in the zip file might be wrong.
in that when we zip it they have windows paths
\and I found something in .netframework about a change to '/' to conform to a linux standard ( https://docs.microsoft.com/en-us/dotnet/framework/migration-guide/mitigation-ziparchiveentry-fullname-path-separator )as we build the entry manually, i think its us putting the
\in, but we can fix that.I will do some quick checks that it doesn't then break windows, and push out an nightly so you can test this. It might not be the issue (i have seen this work on linux, so maybe its distro dependent), but we will see.
Ok,
FYI: I'm pushing from IIS express on windows to unix on AWS
Hi,
yeah i wonder if that's the thing, as the path inside the zip is being set windows , and read on linux :(
it should be fixable, in that windows doesn't seem to mind if the path is using '/' in the zip.
we have a nightly build
9.0.7-rc001.20220125.3you can try that might then fix that bit.Its working now with the nightly build.
Thank you
Woot !
we are looking at a final release next week for that version
Just FYI...
I noticed that when comparing content pulled from the linux machine to my windows machine, it was showing different kinds of line endings \r \n as the only difference in the content.
Oh
:(
I think i have a fix for that, but its going to take a little bit more testing because it effects how we serialize the xml in memory, and while it shouldn't make to much difference, we do that *a lot so i want to make sure we don't bust something else.
Kevin
Any progress on this one Kevin? I've noticed that uSync insists on replacing line endings in content, between my Windows local and Linux app service.
Its not a big deal since its just between dev and staging/test.
Staging/Test/Prod etc will generally be all running on the same OS type.
I think I've seen questions before about how the xml serialization doesn't always put the language elements in the same order.
Its not a feature where you can risk a bug :)
is working on a reply...