Imageprocessor/ Image.sharp not working with Azure Blobstorage through CDN?
Hey guys,
I have implemented the azure blobstorage through the CDN, which was pretty straight forward. But the whole "Url.GetCropUrl(Image, ...)" does not seem to be working. No matter what parameters I throw at it, it does not change the image quality nor the image file-type.
I'm facing the same issue. Did you ever found a solution?
It is allways the original image returned from CDN. However if I request it from Umbraco it seems to be cropping as expected.
Hi Mikkel, what have you set the origin to for the CDN endpoint? It should be set to the website/app URL not the storage account as that would result in the issue you have.
I am not entirely sure I know what you are referring to here. But in my Azure Resource Group I have the following resources:
Storage Account
Endpoint
Frontdoor and CDN Profiles
And all I did was to create a storage account with the necessary containers for Umbraco to get access to it and then create a "default" CDN through the Azure CDN tab. So in Azure I haven't really set anything up, but in Umbraco I have set up what you saw above. Which would translate into something like this in an AppSettings file:
Ah, yeah, it's likely that endpoint will be using the storage account as its origin, this won't work, you should be able to update it to point at the web app (assuming you are using apps).
Thanks for quick reply. Alright, so I went into the CDN endpoint and I saw it was different from what you showed. So I changed it also, and here you get the entire setup - In case I did something wrong elsewhere (Compared to this specific setup):
By default I was able to select my web-app, however, I am currently just testing locally. I suppose that would be an issue compared to the origin-type? So in order to actually verify the CDN works correctly I'd have to deploy my code to the origin-specified-web-app, right? :-)
Thanks once again, I really appreciate the help and clarification!
Yes, it can be difficult to test locally as the CDN origin needs to be online. If you have a development website that's sharing the same DB and blob storage as your local then you should be ok to use that domain locally.
My media is showing up on the site but the image repositioning tool is not working. Could you please point me to a resource that explains how to set up the container, blob storage, and CDN endpoint? I am using v8 in azure web app.
I looked at your image to change the origin name. When I choose my origin hostname, I only see one option: www.example.com. I added example.com and www.example.com to the custom domains section of the app service, but the app uses only example.com. Why is it forcing me to choose www.example.com? What if I want to choose the example.azurewebsites.net site set up?
Right now I have the following:
Storage account with allow blob public access turned on.
I have a media container (I already followed the Umbraco v8 blob documentation). Site is working minus image repositioning and the logs show image processor errors (a lot of them).
I have an endpoint set up and under origins tab I have an origin set up with the above issue (www.example.com).
Have you made sure that both your app-service and your storage account + CDN endpoint and profiles are located in the same Resource-Group in Azure?
When I changed my Origin from Storage Account to web app I was immediately able to see my app-service, so it could be something to do with either a Resource-Group, an App-Service-Plan or similar groupings in Azure.
Yes, they are all in the same resource group in azure. I deleted the cname custom domain from the app service to see if that would fix the origin host selection issue and it did (weird); I now only have the example.com custom domain.
In regard to the blob issue, yes, I followed it. The media is storing correctly - I installed through nuget UmbracoFileSystemProviders.Azure.Media on v8.18.0. I created a blob storage account with a media container. The blob storage account has "allow public access" enabled and "cache every unique URL" under the caching rules of the specific endpoint.
I confirmed that the following keys were added and I filled out the correct information with my account name and my account key in the web.config:
I then added these two app settings to the web.config with the appropriate information replacing the square brackets:
I moved the media files out of the project into the media container.
I am not using azure blob cache...maybe that is the reason I am running into issues?
To summarize the issue:
I am getting so many errors (hundreds) in the log on production and localhost indicating: ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified resource does not exist.) I am not using blob cache, should I (there is no cache container in the storage container)?
The image focal points no longer move the actual focal point on local host or production, so when an editor tries to change the banner focal point in the back-office of a photo in the media container, the focal point does not change.
Notes: I am using an external firewall and v8.18.0 of umbraco.
It's been a while since I developed on Umbraco 8, but the main difference, as I understand, between standard Blob Storage and CDN is which images is served.
My structure in my "root" blobcontainer is media and cache. It automatically added both of those, when my setup was correct. And when I look at the served URLs (media urls) I can see that it only ever looks inside the media folder.
But you could try and set up the blob cache to see if it's an un-documented requirement. I don't recall having to install https://imageprocessor.org/imageprocessor-web/plugins/azure-blob-cache/, but I somewhat recall having configured something regarding the blobstorage cache.
Regarding the focal points; does it reset when they save the media item, or does it not show on the site?
If the latter, then it would seem that you're serving the original media file, and not the processed one.
When I look at the Umbraco Docs, it appears that you now install a package, which more or less adds all these settings to your config-files.
A thing that pops out is the CDN url, Umbraco refers to:
Thank you! I just removed the web.config on the blob storage and placed a fresh one in there and that seemed to remove the first error messages in the log. This also cleared up the focal point issue. But now there is a new error.
ImageProcessor.Common.Exceptions.ImageProcessingException:
GetResponseAsync 132 : Response status code does not indicate success:
404 (The specified blob does not exist.)
How is the container networking supposed to be configured? Is it a publicly available container with a CDN profile and CDN endpoint pointing to the web app as origin (I've read in other guides the endpoint points to the blob account, which is what confused my setup originally). Umbraco back office is not configured for load balancing sine the site does not require load balancing; I am not sure if the CDN endpoint set up creates load balancing and that is causing the error since the back end is not separated from the load balancer (I am not sure if a load balancer is set up by default with a CDN endpoint connecting to the blob storage - I apologize I am not as familiar with this...).
I am a bit confused when it comes to the CDN endpoint. Couldn't you have a CDN endpoint for the blob container with the origin as the blob and then another CDN endpoint for the actual site (app service with the origin as the app service), or would that create problems?
Sorry for the prolonged response, it's been crazy these last days.
In terms of the Blobstorage container I have always used a publicly available blob container (I guess Allow Anonymous) when it came to Umbraco 8/9 azure blob storage handling. As for the CDN endpoint, I clicked up the CDN and I only changed the origin from Storage Account to my Web App. Which did the trick.
This is from Umbraco 9, however, it explains that you have to set up the link between Umbraco backoffice and your Blob Container:
I actually don't know if there is an underlying Load Balancer in their CDN network, but in all cases, you have one endpoint that should distribute the request correctly, I'd assume.
I am not too familiar with CDNs yet, but I though about taking a course in them, as they are, obviously, very tricky to setup correctly ;-)
But just out of curiosity, why would you want 2 endpoints?
The endpoint is only for the actual deliverance of a "processed" image. You don't have to use an endpoint for Umbraco, that's just your blobstorage, i.e. the original image.
No problem! If you create the CDN and allow the origin to be the web app, won't that mean that it the whole website is viewable by web crawlers in a cached state? I thought that the origin had to be the blob storage. I guess something that is confusing me.
I guess I was referring to a CDN for the web app separately from the blob storage so that the website can be delivered to users quicker through the Microsoft CDN and the media container can be delivered to Umbraco quicker through the CDN.
Almost everything works as a charm - Except Focal Points. Have you had any issues with those? I would assume that they would be a new parameter to the URL and as such be loaded anew on the CDN.
But for some reason, none of the focal points-images are shown on the site.
Imageprocessor/ Image.sharp not working with Azure Blobstorage through CDN?
Hey guys,
I have implemented the azure blobstorage through the CDN, which was pretty straight forward. But the whole "Url.GetCropUrl(Image, ...)" does not seem to be working. No matter what parameters I throw at it, it does not change the image quality nor the image file-type.
Is it not yet supported in Umbraco 9?
Thanks in advance,
Mikkel.
I'm facing the same issue. Did you ever found a solution? It is allways the original image returned from CDN. However if I request it from Umbraco it seems to be cropping as expected.
Hey Thomas,
Unfortunately not yet, I am going to look a bit more into https://docs.microsoft.com/en-us/azure/cdn/cdn-query-string which potentially could be the issue. But it's super odd Umbraco hasn't mentioned this..
In any case, I will get back to you when I have tried this approach.
/Mikkel.
Much appreciated!
How have you configured your CDN? The source should be the website/app not the storage account.
Hey Jeavon,
I tried to follow various approaches and my startup looks like this, currently:
and I have also added the middleware injection in the app.UseUmbraco part.
Also, I just changed the CDN to cache for each unique URL.
But I am unable to actually get the ImageSharp process to run. And the cached blobs are being saved with full-query-parameters.
Do you have any idea what could be the missing link here?
Hi Mikkel, what have you set the origin to for the CDN endpoint? It should be set to the website/app URL not the storage account as that would result in the issue you have.
Hi Jeavon,
I am not entirely sure I know what you are referring to here. But in my Azure Resource Group I have the following resources:
And all I did was to create a storage account with the necessary containers for Umbraco to get access to it and then create a "default" CDN through the Azure CDN tab. So in Azure I haven't really set anything up, but in Umbraco I have set up what you saw above. Which would translate into something like this in an AppSettings file:
Are you referring to
or am I missing an entire point in my setup?
Thank you so much in advance!
Ah, yeah, it's likely that endpoint will be using the storage account as its origin, this won't work, you should be able to update it to point at the web app (assuming you are using apps).
Similar to the below:
Hey!
Thanks for quick reply. Alright, so I went into the CDN endpoint and I saw it was different from what you showed. So I changed it also, and here you get the entire setup - In case I did something wrong elsewhere (Compared to this specific setup):
Azure:
Startup.cs:
By default I was able to select my web-app, however, I am currently just testing locally. I suppose that would be an issue compared to the origin-type? So in order to actually verify the CDN works correctly I'd have to deploy my code to the origin-specified-web-app, right? :-)
Thanks once again, I really appreciate the help and clarification!
Yes, it can be difficult to test locally as the CDN origin needs to be online. If you have a development website that's sharing the same DB and blob storage as your local then you should be ok to use that domain locally.
My media is showing up on the site but the image repositioning tool is not working. Could you please point me to a resource that explains how to set up the container, blob storage, and CDN endpoint? I am using v8 in azure web app.
I looked at your image to change the origin name. When I choose my origin hostname, I only see one option: www.example.com. I added example.com and www.example.com to the custom domains section of the app service, but the app uses only example.com. Why is it forcing me to choose www.example.com? What if I want to choose the example.azurewebsites.net site set up?
Right now I have the following:
Storage account with allow blob public access turned on.
I have a media container (I already followed the Umbraco v8 blob documentation). Site is working minus image repositioning and the logs show image processor errors (a lot of them).
I have an endpoint set up and under origins tab I have an origin set up with the above issue (www.example.com).
Any ideas?
Hi Bobi,
Have you made sure that both your app-service and your storage account + CDN endpoint and profiles are located in the same Resource-Group in Azure? When I changed my Origin from Storage Account to web app I was immediately able to see my app-service, so it could be something to do with either a Resource-Group, an App-Service-Plan or similar groupings in Azure.
You're also mentioning Umbraco v8, so just to be sure, have you followed their documentation? https://our.umbraco.com/Documentation/Extending/FileSystemProviders/Azure-Blob-Storage/index-v8 It has a lot of other steps, that I, for Umbraco v9, haven't followed.
I hope it helps, if not, don't hesitate getting back to me.
Hi,
Thank you for your response!
Yes, they are all in the same resource group in azure. I deleted the cname custom domain from the app service to see if that would fix the origin host selection issue and it did (weird); I now only have the example.com custom domain.
In regard to the blob issue, yes, I followed it. The media is storing correctly - I installed through nuget
UmbracoFileSystemProviders.Azure.Media
on v8.18.0. I created a blob storage account with a media container. The blob storage account has "allow public access
" enabled and "cache every unique URL
" under the caching rules of the specific endpoint.I confirmed that the following keys were added and I filled out the correct information with my account name and my account key in the
web.config
:I then made sure the following was in the
web.config
:I then ensured the following was in the
web.config
:I then made sure and filled out the appropriate information for the following in
config/imageprocessor/security.config
:I then added these two app settings to the
web.config
with the appropriate information replacing the square brackets:I moved the media files out of the project into the media container.
I am not using azure blob cache...maybe that is the reason I am running into issues?
To summarize the issue:
I am getting so many errors (hundreds) in the log on production and localhost indicating:
ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified resource does not exist.)
I am not using blob cache, should I (there is no cache container in the storage container)?The image focal points no longer move the actual focal point on local host or production, so when an editor tries to change the banner focal point in the back-office of a photo in the media container, the focal point does not change.
Notes: I am using an external firewall and v8.18.0 of umbraco.
Any help would be greatly appreciated! Thanks!
Hi Bobi,
It's been a while since I developed on Umbraco 8, but the main difference, as I understand, between standard Blob Storage and CDN is which images is served. My structure in my "root" blobcontainer is media and cache. It automatically added both of those, when my setup was correct. And when I look at the served URLs (media urls) I can see that it only ever looks inside the media folder. But you could try and set up the blob cache to see if it's an un-documented requirement. I don't recall having to install https://imageprocessor.org/imageprocessor-web/plugins/azure-blob-cache/, but I somewhat recall having configured something regarding the blobstorage cache.
Regarding the focal points; does it reset when they save the media item, or does it not show on the site? If the latter, then it would seem that you're serving the original media file, and not the processed one.
When I look at the Umbraco Docs, it appears that you now install a package, which more or less adds all these settings to your config-files. A thing that pops out is the CDN url, Umbraco refers to:
However, my CDN is called azureedge.net. Have you made sure the correct url has been inserted here as well?
I hope some of this might spark what you're missing. If not, let's keep looking!
/Mikkel.
Thank you! I just removed the web.config on the blob storage and placed a fresh one in there and that seemed to remove the first error messages in the log. This also cleared up the focal point issue. But now there is a new error.
How is the container networking supposed to be configured? Is it a publicly available container with a CDN profile and CDN endpoint pointing to the web app as origin (I've read in other guides the endpoint points to the blob account, which is what confused my setup originally). Umbraco back office is not configured for load balancing sine the site does not require load balancing; I am not sure if the CDN endpoint set up creates load balancing and that is causing the error since the back end is not separated from the load balancer (I am not sure if a load balancer is set up by default with a CDN endpoint connecting to the blob storage - I apologize I am not as familiar with this...).
I am a bit confused when it comes to the CDN endpoint. Couldn't you have a CDN endpoint for the blob container with the origin as the blob and then another CDN endpoint for the actual site (app service with the origin as the app service), or would that create problems?
Hi Bobi,
Sorry for the prolonged response, it's been crazy these last days.
In terms of the Blobstorage container I have always used a publicly available blob container (I guess Allow Anonymous) when it came to Umbraco 8/9 azure blob storage handling. As for the CDN endpoint, I clicked up the CDN and I only changed the origin from Storage Account to my Web App. Which did the trick.
This is from Umbraco 9, however, it explains that you have to set up the link between Umbraco backoffice and your Blob Container:
I actually don't know if there is an underlying Load Balancer in their CDN network, but in all cases, you have one endpoint that should distribute the request correctly, I'd assume.
I am not too familiar with CDNs yet, but I though about taking a course in them, as they are, obviously, very tricky to setup correctly ;-) But just out of curiosity, why would you want 2 endpoints? The endpoint is only for the actual deliverance of a "processed" image. You don't have to use an endpoint for Umbraco, that's just your blobstorage, i.e. the original image.
I hope it makes sense. /Mikkel.
Hi,
No problem! If you create the CDN and allow the origin to be the web app, won't that mean that it the whole website is viewable by web crawlers in a cached state? I thought that the origin had to be the blob storage. I guess something that is confusing me.
I guess I was referring to a CDN for the web app separately from the blob storage so that the website can be delivered to users quicker through the Microsoft CDN and the media container can be delivered to Umbraco quicker through the CDN.
This should be the CDN endpoint URL, something like
Hey,
Alright. I am using the same DB and Storage Account both locally and on my test-site. So it should work, or?
I will change the options.Url back to targeting the CDN and I'll try deploying the code also. Could be nice to see it work, somewhere.. ;-)
That's correct, also make sure you have the v1.1.0 version of Umbraco.StorageProviders.AzureBlob
Cool! And yea, I got v1.1.0 installed.
I will get back to you with an update as soon as I get it online!
Thanks!
Hi Jeavon,
I hope you had a nice weekend!
Alright, I have now configured it to look towards the web-app and the web-app is now live too.
I went back into my setup and found out, that by declaring a container in the options-url it won't work. But by defining the root-cdn url, as follows:
it works.
Thank you so much, Jeavon!
/Mikkel.
Awesome, I was just writing you a reply but have trashed it now! ;)
Haha, my bad! I really appreciate all the help you've given me! :-)
No problem, just to check did you add the container name into the options.Url?
e.g.
new Uri("https://umbraco9.azureedge.net/my-container/")
Hey,
Initially I did. That obviously didn't work out as intended. So I removed it to only have the "root-url" for the CDN. That works like a charm.
Hi Jeavon,
I hope you're well!
Almost everything works as a charm - Except Focal Points. Have you had any issues with those? I would assume that they would be a new parameter to the URL and as such be loaded anew on the CDN. But for some reason, none of the focal points-images are shown on the site.
I invoke an image like this:
and I set the focal point on the actual image in Umbraco.
Am I doing something wrong? I read that Umbraco might now have support for this in the stable version - But RC003 might have support? https://github.com/umbraco/Umbraco-CMS/issues/11098
*Edit: I just saw the RC003 is from before 9.5.1, and I just updated to 9.5.2. Also, I updated the StorageProviders.AzureBlob to 1.1.1, up from 1.1.0.
Not really into having "RC"-versions as our production site, however, is there any danger in it?
Thanks once again, Jeavon! :-)
Hi All,
Any solutions for this, while moving from v8 to v11, ?
All the media is under the {{container-name}}, but v11 azure blog expects media to be under {{container-name}}/media/{{media-code}}/image.jpg
How to configure v11 to fetch media directly from container ?
All image urls are with container name currently after migrating Db
Thanks,
is working on a reply...