Website down every day for a few minutes recurrently Umbraco 7.6.3 :(
Hi all,
We are facing a serious issue because my website is getting this error every day on slave server on distributed environment. I'm using Umbraco 7.6.3 on Azure. About a week ago my website is down everyday and I only get this weird error on logs and I don't know what is happening on the app. The app availability is down for a couple of minutes and it keeps continue.
How can we fix this?
2019-02-11 05:23:51,471 [P2188/D2/T43] ERROR Umbraco.Web.Scheduling.KeepAlive - Failed (at "https://{{websitedomain}}/umbraco").
System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Umbraco.Web.Scheduling.KeepAlive.<PerformRunAsync>d__4.MoveNext()
2019-02-11 05:30:31,517 [P2188/D2/T48] ERROR Umbraco.Web.Scheduling.KeepAlive - Failed (at "https://{{websitedomain}}/umbraco").
System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Umbraco.Web.Scheduling.KeepAlive.<PerformRunAsync>d__4.MoveNext()
Sadly I made all this commented changes a few moths ago, also taking into consideration all azure and cache approaches. But about a week ago suddenly our website crashes without any weird issue. We did not publish any code for a long time ago,
Apply Windows updates (not sure that's an option on Azure, depending on your configuration).
Upgrade Umbraco to the latest version (pre 7.7 is a bit of a pain, but doable).
Configure your domain in umbracoSettings.config.
Ensure SSL is properly configured (e.g., set umbracoUseSsl to true on the web.config).
Configure IIS to not restart the site if there are 5 errors in 5 minutes (a feature known as rapid fail protection that is usually enabled by default).
Configure ImageProcessor so the cached images are stored outside of the website folder.
Thank you for your quick response, I will take into consideration the approaches I did not use right now.
Configure TLS 1.2 to be the defaul (CHECKED)
Apply Windows updates (not an option, azure web app does not have this). (NA)
Upgrade Umbraco to the latest version (pre
7.7 is a bit of a pain, but doable). (CONSIDERED. WE ARE WORKING ON THIS)
Configure your domain in umbracoSettings.config. (CHECKED)
Ensure SSL is properly configured (e.g., set
umbracoUseSsl to true on the web.config). (CHECKED)
Configure IIS to not
restart the site if there are 5 errors in 5 minutes (a feature known
as rapid fail protection that is usually enabled by default). (TO BE CONSIDERED, but on azure load balancing environment I really have no clue how to do this)
Configure ImageProcessor so the cached images are stored outside of
the website folder. (CHECKED)
Change fcnMode in the web.config. (TO BE CONSIDERED)
I'll let you know when I make all the changes and I'll give some feedback about this.
Thank you for your quick response, I will take into consideration the approaches I did not use right now.
Configure TLS 1.2 to be the default (CHECKED)
Apply Windows updates (not an option, azure web app does not have
this). (NA)
Upgrade Umbraco to the latest version (pre 7.7 is a bit of a pain, but doable). (CHECKED updated to 7.7.2)
Configure your domain in umbracoSettings.config. (I use Flexible
load balancing over Azure using this configurations)
Ensure SSL is properly configured (e.g., set umbracoUseSsl to true on
the web.config). (CHECKED)
Configure ImageProcessor so the cached images are stored outside of
the website folder. (CHECKED) Change fcnMode in the web.config.
(CHECKED)
Making all this changes my website continues unstable. I have to remove the use of the CDN because azureCDNToolkit is exausting my database pool.
2019-07-07 11:28:59,037 [P14776/D2/T13] ERROR Our.Umbraco.AzureCDNToolkit.UrlHelperRenderExtensions - Error resolving media url from the CDN
System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at Our.Umbraco.AzureCDNToolkit.UrlHelperRenderExtensions.<>c__DisplayClass6_1.<UrlToCdnUrl>b__0()
at Our.Umbraco.AzureCDNToolkit.UrlHelperRenderExtensions.TryFiveTimes(Action delegateAction)
at Our.Umbraco.AzureCDNToolkit.UrlHelperRenderExtensions.UrlToCdnUrl(String cropUrl, Boolean htmlEncode, String currentDomain)
Also having this issue in Umbraco 8.0.2, despite my attempts to resolve it by enabling TLS1.2 through adding a Component, ensuring useHttps=true, putting FQDN in applicationUrl parameter for web.routing, plus the suggestions above. Our site works but does reboot after 20-30 minutes of no user activity (it's an occasionally accessed intranet), and we'd like the KeepAlive feature to function because the boot-up time is maybe 15 seconds or so.
Because we're not able to get the KeepAlive stuff working (in v.8.0.2, we still get that error message every 5 minutes, and doing HealthCheck shows we can't ping our own SSL server even though our certificates are fine and nothing seems out of place), I did the following to at least reduce the number of times the site restarts:
In IIS, made the following Advanced Settings changes for the application pool running our Umbraco intranet site:
1) Disabled Rapid-Fail Protection (Enabled=False)
2) Set Process Model Idle Time-out to 480 (so it only happens every 8 hours)
3) Set Idle Time-out Action to Suspend, which seems to make the restart of the site much quicker
I'm not sure how those settings get made in an Azure environment (this is our own internal intranet server), nor whether it's a safe approach for higher-bandwidth sites, but that has at least made the site stop restarting constantly, and even when it does, seems to respond quickly.
Seems like several folks have the same KeepAlive issue especially with https:// sites, so hoping for more answers to fully resolve this, because I feel these steps are sort of a band-aid.
Was there ever any update? I'm on v8.5.3 and constantly get 404s to /umbraco/api/keepalive/ping in my Azure App Insights. I cannot view that URL in a browser either.
I never got a reply, also now on 8.5.3 with the same situation. My "band-aid" approach only seems to partly work — the site can take awhile to load initially if it happens to timeout (like if no one is on our intranet in the last 24 hours, which sometimes happens).
Website down every day for a few minutes recurrently Umbraco 7.6.3 :(
Hi all, We are facing a serious issue because my website is getting this error every day on slave server on distributed environment. I'm using Umbraco 7.6.3 on Azure. About a week ago my website is down everyday and I only get this weird error on logs and I don't know what is happening on the app. The app availability is down for a couple of minutes and it keeps continue. How can we fix this?
Than you.
Hi Arlan,
Can you ping the site Url(s) on each server? Basically you need to be able to resolve each site address on your server(s).
See the following docs for more information: https://our.umbraco.com/documentation/Getting-Started/Data/Scheduled-Publishing/
Specifically for you as it seems you are running in a Load Balanced environment:
https://our.umbraco.com/documentation/Getting-Started/Setup/Server-Setup/Load-Balancing/flexible#scheduling-and-master-election
Thanks
Craig
Hi Craig,
Sadly I made all this commented changes a few moths ago, also taking into consideration all azure and cache approaches. But about a week ago suddenly our website crashes without any weird issue. We did not publish any code for a long time ago,
Greetings,
Arlan
Some things to try (in no particular order):
umbracoSettings.config.umbracoUseSslto true on the web.config).fcnModein the web.config: https://shazwazza.com/post/all-about-aspnet-file-change-notification-fcn/Hello Nicholas,
Thank you for your quick response, I will take into consideration the approaches I did not use right now.
I'll let you know when I make all the changes and I'll give some feedback about this.
Thank you so much for your time and patience.
Arlan
Looks like there may be a way to disable rapid fail protection from an Azure web app (TLDR: an XML transform file): https://stackoverflow.com/a/43840377/2052963
Hello Nicholas,
Thank you for your quick response, I will take into consideration the approaches I did not use right now.
Making all this changes my website continues unstable. I have to remove the use of the CDN because azureCDNToolkit is exausting my database pool.
Anything about this? Thank you in advance.
Arlan
Also having this issue in Umbraco 8.0.2, despite my attempts to resolve it by enabling TLS1.2 through adding a Component, ensuring useHttps=true, putting FQDN in applicationUrl parameter for web.routing, plus the suggestions above. Our site works but does reboot after 20-30 minutes of no user activity (it's an occasionally accessed intranet), and we'd like the KeepAlive feature to function because the boot-up time is maybe 15 seconds or so.
Hi Eric, did you solve this issue? We have now the same problem, when the site interaction is low, the site reboots.
Because we're not able to get the KeepAlive stuff working (in v.8.0.2, we still get that error message every 5 minutes, and doing HealthCheck shows we can't ping our own SSL server even though our certificates are fine and nothing seems out of place), I did the following to at least reduce the number of times the site restarts:
In IIS, made the following Advanced Settings changes for the application pool running our Umbraco intranet site:
1) Disabled Rapid-Fail Protection (Enabled=False)
2) Set Process Model Idle Time-out to 480 (so it only happens every 8 hours)
3) Set Idle Time-out Action to Suspend, which seems to make the restart of the site much quicker
I'm not sure how those settings get made in an Azure environment (this is our own internal intranet server), nor whether it's a safe approach for higher-bandwidth sites, but that has at least made the site stop restarting constantly, and even when it does, seems to respond quickly.
Seems like several folks have the same KeepAlive issue especially with https:// sites, so hoping for more answers to fully resolve this, because I feel these steps are sort of a band-aid.
Was there ever any update? I'm on v8.5.3 and constantly get 404s to /umbraco/api/keepalive/ping in my Azure App Insights. I cannot view that URL in a browser either.
I never got a reply, also now on 8.5.3 with the same situation. My "band-aid" approach only seems to partly work — the site can take awhile to load initially if it happens to timeout (like if no one is on our intranet in the last 24 hours, which sometimes happens).
is working on a reply...