The package works great for uploading new images to Azure storeage, but I have several sites that would benefit from this package but they have existing files in local media. What is the suggested process to move these local media files to blob storage and also change the references to point to the CDN, thereby allowing the removal of the local media items?
I think you could create e.g. a dashboard for this purpose. Within the code behind you could loop through all media directories, upload the files (don't forget thumbs and crops) to blob storage, update the umbracoFile property with the blob uri and delete the local file.
Hello Guys
Would it be possible to copy existing media items to the Azure Blob Storage using something like CloudBerry Explorer at all and simply upload them as if it was a FTP storage?
just moving the files and folders will be not enough. The default FileSystemProvider stores a relative to your web root path like /media/1001/myimage.jpg but as the cloud is not relative to your web root the azure FileSystemProvider stores a full quaified URI like http://myAccountName.blob.core.windows.net/media/1001/myimage.jpg in the umbracoFile property of an media node.
After copying all media files with external tools you have to update all umbracoFile properties for your Azure Storage Account by prefixing the path wit the storage domain.
Hi Dirk,
So it is possible then. So a SQL script or building a dashboard to update all media item's paths from relative and prefix the URL back on as needed.
I'm thinking I want to create a way so you can easily take the existing content in the media directory, install the blob storage plugin, then you can go to the dashboard...press a button to migrate media content. It should then copy that media content to Azure in the background.
My question: do we just iterate through the media directory? Or is there something else we need to tap into? Also, when you say update umbracoFile...is that on each piece of content?
Hi Steve, the umbracoFile property is normaly of default data type Upload (property editor: Upload field). This data type editor is base on an IFileSystem provider. This provider is responsible to store the file data somewhere and providing the items url. The default provider will replaced via config by the AzureBlobStorage provider. This means on all document types which use the Upload data type as property, you will have file references that have to be migrated. By default this data type is only used on Media Image and Media File types. So in short: Yes you have to respect each piece of content that uses a data type base on the property editor Upload field.
Hi all, you *can* (if you feel comfortable enough with the method) use a q&d query to update your references. I know it's not the most pretty, but it's thouroughly tested and works like a charm.
I've tried your SQL Script and I can see the changes clearly.
I'm in a testing phase initially trying to break my media links. I've not installed Dirk's package yet.
Any idea why everything would still be working despite the script updates? I've done a full republish.aspx run, restarted the app pool and it's still not breaking in either the front end or backend.
I found a solution to this in the end. I adapted Guido's SQL as follows...
-- Update The Property Data
update cmsPropertyData
set dataNvarchar = cast(replace(cast(dataNvarchar as nvarchar(max)), '/media/', 'http://[Storage Account Name].blob.core.windows.net/media/') as nvarchar(max))
where dataNvarchar like '/media/%'
go
-- Update the XML Tables
update cmsContentXml
set xml = cast(replace(cast(xml as nvarchar(max)), '/media/', 'http://[Storage Account name].blob.core.windows.net/media/') as ntext)
where xml like '<Image%' or xml like '<File%'
go
(http://[Storage Account name].blob.core.windows.net/media/) assumes that the storage account container you are using is called 'media'.
A Republish all via dialogs/republish.aspx wasn't enough for completing it after running the script though - I had to manually go through and re-save every item in the backend. I'm sure there's probably a better solution than saving every item manually though.
You were right on updating the propertydata. It did that myself too, indeed. There are no further references to your media, so publishing every single mediaitem really shouldn't be necessary. Are you sure you ain't getting cashed content from umbraco.xml? Delete it to rebuild.
I also found the SQL updates were not sufficient to make the images available in the front-end. I used this code to publish all media files using an UmbracoApiController, called through http://yoursite/umbraco/api/media/publish
Can anyone indicate why this is necessary?
public class MediaController : UmbracoApiController
{
[HttpGet]
public string Publish()
{
var mediaCounter = 0;
if (ApplicationContext.Services != null)
{
var ms = ApplicationContext.Services.MediaService;
for (int i = 0; i < 10; i++)
{
var media = ms.GetByLevel(i).ToList();
if (media.Any())
{
foreach (var item in media)
{
ms.Save(item);
mediaCounter++;
}
}
}
}
return mediaCounter.ToString();
}
}
Since there are no other references, it seems to me this is still a cache issue. Did you try to restart the webservice (after deleting the umbraco.config) to see if that changed anything?
To convert the image cropper fields I used this code:
update cmsContentXml
set xml = cast(replace(cast(xml as nvarchar(max)), '"src": "/media/', '"src": "https://[Storage Account Name].blob.core.windows.net/media/') as ntext)
go
update cmsPropertyData
set dataNText = cast(replace(cast(dataNText as nvarchar(max)), '"src": "/media/', '"src": "https://[Storage Account Name].blob.core.windows.net/media/') as ntext)
go
In response to John's remark, I've had to do migrate media to Azure recently and I found I had to save the media too, albeit filtered by imagery. However, I did not have to update the media references in the database.
Obviously Umbraco has moved on somewhat since this was originally posted but it's worth noting for when I come back here looking for answers again!
Moving existing images to Blob Storage
The package works great for uploading new images to Azure storeage, but I have several sites that would benefit from this package but they have existing files in local media. What is the suggested process to move these local media files to blob storage and also change the references to point to the CDN, thereby allowing the removal of the local media items?
Hallo Daniel,
I think you could create e.g. a dashboard for this purpose. Within the code behind you could loop through all media directories, upload the files (don't forget thumbs and crops) to blob storage, update the umbracoFile property with the blob uri and delete the local file.
You will find all neccessary code in the source of this package on github https://github.com/idseefeld/UmbracoAzureBlobStorage
See method UploadFileToBlob for upload.
Yours
Dirk
Hello Guys
Would it be possible to copy existing media items to the Azure Blob Storage using something like CloudBerry Explorer at all and simply upload them as if it was a FTP storage?
Look forward to hearing back from you.
Cheers,
Warren :)
I think with my new Azure package you would be able to do that since the urls are relative.
Hi Warren,
I can confirm that Cloudberry works like a charm for this as I've jsut this minute done it!
Hi Warren,
just moving the files and folders will be not enough. The default FileSystemProvider stores a relative to your web root path like /media/1001/myimage.jpg but as the cloud is not relative to your web root the azure FileSystemProvider stores a full quaified URI like http://myAccountName.blob.core.windows.net/media/1001/myimage.jpg in the umbracoFile property of an media node.
After copying all media files with external tools you have to update all umbracoFile properties for your Azure Storage Account by prefixing the path wit the storage domain.
Dirk
Hi Dirk,
So it is possible then. So a SQL script or building a dashboard to update all media item's paths from relative and prefix the URL back on as needed.
Hi Warren,
I have not tried this yet but yes I think it should work this way. If you are going to try this please post your feedback ;-)
Dirk
Dirk,
I'm thinking I want to create a way so you can easily take the existing content in the media directory, install the blob storage plugin, then you can go to the dashboard...press a button to migrate media content. It should then copy that media content to Azure in the background.
My question: do we just iterate through the media directory? Or is there something else we need to tap into? Also, when you say update umbracoFile...is that on each piece of content?
-Steve
Hi Steve, the umbracoFile property is normaly of default data type Upload (property editor: Upload field). This data type editor is base on an IFileSystem provider. This provider is responsible to store the file data somewhere and providing the items url. The default provider will replaced via config by the AzureBlobStorage provider.
This means on all document types which use the Upload data type as property, you will have file references that have to be migrated. By default this data type is only used on Media Image and Media File types.
So in short: Yes you have to respect each piece of content that uses a data type base on the property editor Upload field.
I hope this makes it clearer.
Yours
Dirk
Hi all, you *can* (if you feel comfortable enough with the method) use a q&d query to update your references. I know it's not the most pretty, but it's thouroughly tested and works like a charm.
Cheers,
Guido
--TEST
select top 100 *,
cast(replace(cast(xml as nvarchar(max)), '/media/', 'https://XXXBLOBNAMEXXX.blob.core.windows.net/media/') as ntext)
from cmsContentXml
where xml like '<Image%
--UPDATE
update cmsContentXml
set xml = cast(replace(cast(xml as nvarchar(max)), '/media/', 'https://XXXBLOBNAMEXXX.blob.core.windows.net/media/') as ntext)
where xml like '<Image%'
Guido,
I've tried your SQL Script and I can see the changes clearly.
I'm in a testing phase initially trying to break my media links. I've not installed Dirk's package yet.
Any idea why everything would still be working despite the script updates? I've done a full republish.aspx run, restarted the app pool and it's still not breaking in either the front end or backend.
Hi all,
I found a solution to this in the end. I adapted Guido's SQL as follows...
(http://[Storage Account name].blob.core.windows.net/media/) assumes that the storage account container you are using is called 'media'.
A Republish all via dialogs/republish.aspx wasn't enough for completing it after running the script though - I had to manually go through and re-save every item in the backend. I'm sure there's probably a better solution than saving every item manually though.
Hi Keith,
You were right on updating the propertydata. It did that myself too, indeed. There are no further references to your media, so publishing every single mediaitem really shouldn't be necessary. Are you sure you ain't getting cashed content from umbraco.xml? Delete it to rebuild.
Guido
Hi,
I also found the SQL updates were not sufficient to make the images available in the front-end. I used this code to publish all media files using an UmbracoApiController, called through http://yoursite/umbraco/api/media/publish
Can anyone indicate why this is necessary?
Hi John,
Since there are no other references, it seems to me this is still a cache issue. Did you try to restart the webservice (after deleting the umbraco.config) to see if that changed anything?
I will create a RFC on my next update.
Cheers!
Hello Guido,
Yes, I tried all that, but that did not help.
John
Hi,
To convert the image cropper fields I used this code:
In response to John's remark, I've had to do migrate media to Azure recently and I found I had to save the media too, albeit filtered by imagery. However, I did not have to update the media references in the database.
Obviously Umbraco has moved on somewhat since this was originally posted but it's worth noting for when I come back here looking for answers again!
is working on a reply...