Unable to deploy media between environments with Umbraco 8.1.4 on Cloud and UmbracoFileSystemProviders.Azure
We have been using UmbracoFileSystemProviders.Azure for media extensively in Umbraco 7 on Umbraco Cloud for years and it worked very well in different scenarios.
When we tried to deliver Umbraco 8 sites on the cloud with Our.Umbraco.FileSystemProviders.Azure.* we can't get it to work reliably unfortunately.
Using the most current version(s) of UmbracoFileSystemProviders.Azure from
GitHub on a Umbraco 8.1.4 instance, the media items themselves get transferred by Deploy (from DEV environment to LIVE environment), but the actual media files are missing.
What I'd like to know is if somebody has this working? Thanks!
Here's two screenshots to illustrate:
I tried to debug it and it seems like the azure provider is not even called/used by Deploy. Logging is set to the extreme and the log shows no sign of activity for the Azure lib like it should according to the source code for the latest build. However, there is "read 'umb://media-file/rufh1kig/somefilename.png'" and such.
On the contrary, I do indeed see activity on the storage container, and it is reporting some errors and the times correlate with the deployments of the test media item (also there is no other activity on the storage):
Does anyone have a smart idea how to debug this better?
I would be grateful for any heads up!
We debugged further and double-checked the settings.
The errors in the Azure logs are there even when uploading directly to each environment (picking media and uploading which is working and did so from the beginning).
So the Azure Blob "ClientOtherError" don't seem to be linked to the issue.
We also debugged (stepping through even on localhost) with alpha4 of the v2 azure ifilesys provider and AddFile doesn't get called on restore from Cloud (only way do run a debugger, so we used the reverse direction to pull from the dev instance down) but browsing/querying does get called.
The exact scenario works in Umbrao 7 on Umbraco Cloud with the Community Azure Provider v1.
We tested this on a production site (v7) on Cloud with this settings:
staging: container A (A values for rootUrl and connectionString)
live: container B (B values for rootUrl and connectionString)
The test:
create media on staging, upload media file to it
add to deploy queue via context menu (target: live)
start transfer via "Your workspace" in content section
(transfer in progress, successful message)
check media on live: media item is there as well as the image file
success! 🎉
From what we discovered it seems, in Umbraco 8 with Azure Storage Deploy doesn't "just" run the whole pipeline "like a local upload" when it's getting a media item transferred but believes in some "shortcut"? As it stands now, I suppose it doesn't actually (even attempt to) transfer any file, but just the media item ("meta"). IMHO this is why in Umbraco 8 the media file is missing.
The question is why does CanAddPhysical exist in v8 when it didn't in v7 and what's it's exact purpose/intent? Also, can a virtual provider (e.g. Blobs) say they "CanAddPhysical"? 🤔
It looks like we would need to implement the new property and simulate the AddFile method with blobs?
Did you find a solution to this - I'm struggling with same issue..
Umbraco.Deploy.Exceptions.RemoteApiException: The remote API has thrown an exception.
[A lot of stacktrace]
System.AggregateException: Failed to apply all changes (see exceptions).
[Some more stacktrace]
at Umbraco.Core.IO.ShadowFileSystem.Complete()
at Umbraco.Core.IO.ShadowWrapper.UnShadow(Boolean complete)
at Umbraco.Core.IO.FileSystems.EndShadow(String id, Boolean completed)
--> System.Exception: Could not save file "xm4l1yyv/img20190503133746.jpg".
--> System.NotImplementedException: The method or operation is not implemented.
at Our.Umbraco.FileSystemProviders.Azure.AzureBlobFileSystem.get_CanAddPhysical()
at Umbraco.Core.IO.ShadowFileSystem.Complete()
Unable to deploy media between environments with Umbraco 8.1.4 on Cloud and UmbracoFileSystemProviders.Azure
We have been using UmbracoFileSystemProviders.Azure for media extensively in Umbraco 7 on Umbraco Cloud for years and it worked very well in different scenarios.
When we tried to deliver Umbraco 8 sites on the cloud with Our.Umbraco.FileSystemProviders.Azure.* we can't get it to work reliably unfortunately.
Using the most current version(s) of UmbracoFileSystemProviders.Azure from GitHub on a Umbraco 8.1.4 instance, the media items themselves get transferred by Deploy (from DEV environment to LIVE environment), but the actual media files are missing.
What I'd like to know is if somebody has this working? Thanks!
Here's two screenshots to illustrate:
I tried to debug it and it seems like the azure provider is not even called/used by Deploy. Logging is set to the extreme and the log shows no sign of activity for the Azure lib like it should according to the source code for the latest build. However, there is "read 'umb://media-file/rufh1kig/somefilename.png'" and such.
On the contrary, I do indeed see activity on the storage container, and it is reporting some errors and the times correlate with the deployments of the test media item (also there is no other activity on the storage):
Does anyone have a smart idea how to debug this better? I would be grateful for any heads up!
We debugged further and double-checked the settings.
The errors in the Azure logs are there even when uploading directly to each environment (picking media and uploading which is working and did so from the beginning).
So the Azure Blob "ClientOtherError" don't seem to be linked to the issue.
We also debugged (stepping through even on localhost) with alpha4 of the v2 azure ifilesys provider and AddFile doesn't get called on restore from Cloud (only way do run a debugger, so we used the reverse direction to pull from the dev instance down) but browsing/querying does get called.
Really strange.
With the help from Jeavon we dug deep and discovered, that there has been introduced a new property CanAddPhysical on the interface which the v2 provider doesn't implement right now (throws NotImplEx)
The exact scenario works in Umbrao 7 on Umbraco Cloud with the Community Azure Provider v1.
We tested this on a production site (v7) on Cloud with this settings:
The test:
From what we discovered it seems, in Umbraco 8 with Azure Storage Deploy doesn't "just" run the whole pipeline "like a local upload" when it's getting a media item transferred but believes in some "shortcut"? As it stands now, I suppose it doesn't actually (even attempt to) transfer any file, but just the media item ("meta"). IMHO this is why in Umbraco 8 the media file is missing.
The question is why does CanAddPhysical exist in v8 when it didn't in v7 and what's it's exact purpose/intent? Also, can a virtual provider (e.g. Blobs) say they "CanAddPhysical"? 🤔
It looks like we would need to implement the new property and simulate the AddFile method with blobs?
Hi Marc
Did you find a solution to this - I'm struggling with same issue..
Umbraco.Deploy.Exceptions.RemoteApiException: The remote API has thrown an exception.
[A lot of stacktrace]
System.AggregateException: Failed to apply all changes (see exceptions).
[Some more stacktrace]
at Umbraco.Core.IO.ShadowFileSystem.Complete() at Umbraco.Core.IO.ShadowWrapper.UnShadow(Boolean complete) at Umbraco.Core.IO.FileSystems.EndShadow(String id, Boolean completed) --> System.Exception: Could not save file "xm4l1yyv/img20190503133746.jpg".
--> System.NotImplementedException: The method or operation is not implemented.
at Our.Umbraco.FileSystemProviders.Azure.AzureBlobFileSystem.get_CanAddPhysical() at Umbraco.Core.IO.ShadowFileSystem.Complete()
Hey Kasper,
unfortunately not. I haven't looked into it any further since then.
is working on a reply...