I've recently upgraded my site from V8 to V9. All data has been migrated (content/media) via Usync, and this is imported in an clean V9 database.
When running it locally, with the DB on azure, a cold boot is quite fast.
But when deploying my site on Azure, the cold boot takes about 20 minutes.
After a deploy, I can log in the back-end, and that loads (initially), but any DB action will time out untill 'everything is done'. Now what is this everything? I have the idea that this is the Memory Cache that's been building. Only I have no idea why it takes so long (1200 content items and 12.000 media items).
You can see the Umbraco Logs below, nothing special happens. Also when pressing the "Reload Memory Cache" button, it triggers the same lengty blockade where my site is utterly unreachable.
I'm hoping someone has a suggestion on the how/why this happens.
Running Umbraco 9.3.1 on a Azure App Service (B2) With a similar Azure SQL database (S1)
-- Cold boot + some stuff + migrations and then:
{"@t":"2022-03-09T20:19:43.5002594Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading content from database","TimingId":"38ef9ea","SourceContext":"Umbraco.Cms.Core.Logging.ProfilingLogger","RequestId":"0HMG218HVVMOJ:00000002","RequestPath":"/","ConnectionId":"0HMG218HVVMOJ","ProcessId":44,"ProcessName":"dotnet","ThreadId":15,"AppDomainId":1,"AppDomainAppId":"26003009a056ee987bd2f1f300def9026d3f22d1","MachineName":"69876f356abb","Log4NetLevel":"INFO ","HttpRequestId":"c46467e5-d6e1-4ade-8e63-674b827f5ee1","HttpRequestNumber":1,"HttpSessionId":"0"}
{"@t":"2022-03-09T20:19:45.1195687Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":1619,"TimingId":"38ef9ea","SourceContext":"Umbraco.Cms.Core.Logging.ProfilingLogger","RequestId":"0HMG218HVVMOJ:00000002","RequestPath":"/","ConnectionId":"0HMG218HVVMOJ","ProcessId":44,"ProcessName":"dotnet","ThreadId":15,"AppDomainId":1,"AppDomainAppId":"26003009a056ee987bd2f1f300def9026d3f22d1","MachineName":"69876f356abb","Log4NetLevel":"INFO ","HttpRequestId":"c46467e5-d6e1-4ade-8e63-674b827f5ee1","HttpRequestNumber":1,"HttpSessionId":"0"}
{"@t":"2022-03-09T20:21:55.8650549Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading media from database","TimingId":"8619d14","SourceContext":"Umbraco.Cms.Core.Logging.ProfilingLogger","RequestId":"0HMG218HVVMOJ:00000002","RequestPath":"/","ConnectionId":"0HMG218HVVMOJ","ProcessId":44,"ProcessName":"dotnet","ThreadId":15,"AppDomainId":1,"AppDomainAppId":"26003009a056ee987bd2f1f300def9026d3f22d1","MachineName":"69876f356abb","Log4NetLevel":"INFO ","HttpRequestId":"c46467e5-d6e1-4ade-8e63-674b827f5ee1","HttpRequestNumber":1,"HttpSessionId":"0"}
{"@t":"2022-03-09T20:22:01.4192821Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":5554,"TimingId":"8619d14","SourceContext":"Umbraco.Cms.Core.Logging.ProfilingLogger","RequestId":"0HMG218HVVMOJ:00000002","RequestPath":"/","ConnectionId":"0HMG218HVVMOJ","ProcessId":44,"ProcessName":"dotnet","ThreadId":15,"AppDomainId":1,"AppDomainAppId":"26003009a056ee987bd2f1f300def9026d3f22d1","MachineName":"69876f356abb","Log4NetLevel":"INFO ","HttpRequestId":"c46467e5-d6e1-4ade-8e63-674b827f5ee1","HttpRequestNumber":1,"HttpSessionId":"0"}
{"@t":"2022-03-09T20:40:06.4955386Z","@mt":"Document {ContentName} (id={ContentId}) has been published.","ContentName":"membertest","ContentId":12481,"SourceContext":"Umbraco.Cms.Core.Services.Implement.ContentService","ActionId":"cfd58f55-e2b1-421b-8eda-60fca1408728","ActionName":"Umbraco.Cms.Web.BackOffice.Controllers.ContentController.PostSave .....
Basically, every time we deployed, it would go into a totally new instance, which would trigger a cold boot, which took 10+ min to start up. A restart without a "cold boot" wouldn't take nearly as long. (I wasn't able to do this ever using Linux hosting since it always goes into a new container and so is always a cold boot, but locally clearing out my cached files and doing a cold boot took quite a bit longer than just restarting).
I think there were some Azure hosting improvements in 9.4 that might help with this, but I haven't looked at them so I'm not sure.
I've created an issue at the Umbraco Github and taken some good steps! As a workaround you can disable the on-disk memory cache by applying this setting!
Well I had a bit of time, so created a new Dev/Test Tier Windows application, and after deploy my site was running in about a minute.
When reloading the memory cache it took about 50 seconds.
So a Linux Azure App is definitely the issue, I'd like to switch back to Windows ASAP again, but too bad there's a major price difference between a Linux B1 and a Windows B1 (13 USD vs 55 usd)
I've created an issue at the Umbraco Github and taken some good steps! As a workaround you can disable the on-disk memory cache by applying this setting!
Very long cold boot
I've recently upgraded my site from V8 to V9. All data has been migrated (content/media) via Usync, and this is imported in an clean V9 database.
When running it locally, with the DB on azure, a cold boot is quite fast. But when deploying my site on Azure, the cold boot takes about 20 minutes.
After a deploy, I can log in the back-end, and that loads (initially), but any DB action will time out untill 'everything is done'. Now what is this everything? I have the idea that this is the Memory Cache that's been building. Only I have no idea why it takes so long (1200 content items and 12.000 media items).
You can see the Umbraco Logs below, nothing special happens. Also when pressing the "Reload Memory Cache" button, it triggers the same lengty blockade where my site is utterly unreachable.
I'm hoping someone has a suggestion on the how/why this happens. Running Umbraco 9.3.1 on a Azure App Service (B2) With a similar Azure SQL database (S1)
-- Cold boot + some stuff + migrations and then:
Just a little bump in hopes of getting a solution. Haven't made any new progress on this, and deployments are getting annoying.
FYI the recommended DB is S2 https://our.umbraco.com/Documentation/Fundamentals/Setup/server-setup/azure-web-apps - personally I'd start with updating the DB tier to see what kind of difference that makes.
Thanks for your reply Sebastiaan,
In short, the answers on your questions:
I've also tried it with a A2 database, but this made no difference for me.
Tonight I'm going to try and publish the exact same to an Windows App, to see if Linux is really the problem for some reason.
I'm assuming A2 was a typo. Seems very weird that it would work fine on Windows but not on Linux, but that's exactly the test I would do as well.
It sounds like you might be trying to host this in Linux.
If so, we tried hosting our 9.3 site using Linux in Azure, but were unable to do so due to long start up times. We ended up needing to use a Windows Azure instance. https://our.umbraco.com/forum/using-umbraco-and-getting-started/108190-azure-linux-web-app-takes-long-time-to-start
Basically, every time we deployed, it would go into a totally new instance, which would trigger a cold boot, which took 10+ min to start up. A restart without a "cold boot" wouldn't take nearly as long. (I wasn't able to do this ever using Linux hosting since it always goes into a new container and so is always a cold boot, but locally clearing out my cached files and doing a cold boot took quite a bit longer than just restarting).
I think there were some Azure hosting improvements in 9.4 that might help with this, but I haven't looked at them so I'm not sure.
Thanks Kevin, this sounds exactly like my issue. Thanks!
I'll keep this updated with my findings tonight.
I've created an issue at the Umbraco Github and taken some good steps! As a workaround you can disable the on-disk memory cache by applying this setting!
Solved it for me for now.
Well I had a bit of time, so created a new Dev/Test Tier Windows application, and after deploy my site was running in about a minute. When reloading the memory cache it took about 50 seconds.
So a Linux Azure App is definitely the issue, I'd like to switch back to Windows ASAP again, but too bad there's a major price difference between a Linux B1 and a Windows B1 (13 USD vs 55 usd)
I've created an issue at the Umbraco Github and taken some good steps! As a workaround you can disable the on-disk memory cache by applying this setting!
Solved it for me for now.
is working on a reply...