More Contour fields a incrementally slower form post save
Using Umbraco 6.0.5 and Contour 3.0.12 it's seems that as I add more fields to a form the slower it gets - significantly. Under 5 fields is under 1 sec but over 10 fields and I'm getting over 4 seconds to save. No workflow activities or actions on the page. No errors in Logs.
Do databases indexes get correctly installed?
How can I increase contour speed? (At the momment I'm needing to validate the form (based off Cultiv Rest), then UploadAsync to spawn a save event - CRAZY!)
While I was wanting to use the standard MVC macro to save / handle the form. Because of the slow speed I've moved to a customised version of CultivRest in an async (fire and forget) pattern, but would prefer to address the core problem of it being a slow save. Any ideas?
Any updated on this issue? I am using Umbraco 4.11.10 and Contour 3.0.17 and have a multi step form with 98 fields. I'm using the standard Razor macro with no workflows or custom behaviour. Loading the first form step takes +20 seconds (even though this step has only a few actual fields). Continuing to each next step also takes the same amount of time. I've setup an exact duplicate form and the more fields I remove, the faster that one becomes, so it does seem that the number of fields have an impact on the performance.
I can also confirm that it's not the FieldType.xxx.cshtml views taking long to render. Only after the 20 seconds are over, does the first field FieldType.xxx.cshtml template get hit and immediately afterwards the entire form step is rendered.
Saving the form in the backend also slows down the more fields are added.
I have forms that are taking 3 minutes to render when a user hits the 'Next' button, can you confirm the 3.0.18 build will work on Umbraco v4.11.8 as I'd like to try an upgrade and see if we get better performance.
Running 3.0.18 does seem to have made a major difference in saving the form in the backend which was just as slow, however viewing the form and navigating through it's various steps in the frontend still seems as slow as originally. Can you please confirm that this is expected or should the frontend also have been fixed?
Currently looking into speed improvements, I've got confirmation that adding some indexes to the db tables have a great impact so will get an update script up
That's great news! Sorry for following up again, but our form is currently on production and getting a lot of complaints due to the slow form. Any updates on this and when the script will be ready?
I might have some more information on this issue. My Contour Razor form is sitting in a protected area of the site. So for the user to get to the form, they must be logged in using forms auth with the default membership provider.
For some reason when testing this again, while authenticated and on the form refreshing the page and noticing the usual slow loading time, I deleted all cookies. After this, refreshing the form was pretty much instant! It turns out it is the forms auth cookie itself. Logging back in again slows down the form, however deleting only the auth cookie, speeds it up again. It seems that the presence of this cookie is what is causing the form to slow down. Unfortunately the form has to sit in the protected area, so I cannot move it out.
Another thing to note is that my cookie has a custom domain specified in the web.config since this form is sitting on a sub domain of the site. We require single sign-on thus logging in from the main domain or the subdomain should set a single cookie and authenticate the user for all subdomains. For this reason, the domain value of the auth cookie is set to be ".domain.com" instead of "domain.com" (note the . at the beginning). Not sure if this is contributing to the issue but thought I'd mention it anyway.
I think originally when testing this it was just coincidence that this cookie was not present when I had a small form and it was present when I had the large form every time, thus leading to the conclusion that it's the number of form fields slowing it down.
In terms of the Contour Razor form, does this make sense that the auth cookie being present can slow it down?
That shouldn't have an effect on the form speed, will do a test, so it just sits in a protected area of the site, if you move it out of the protected area it works fine?
Sorry, I shouldn't have used the word "protected". It's not a protected section in terms of Umbraco Public Access. It's just that by the time the user gets to the form page, they would be logged in and the auth cookie would be present. However this page does not rely on the user being logged in. So deleting the cookie at this stage and refreshing the page is much faster again. It might not be the fact that the actual auth cookie is present but might be to do with something else that is going on because of the user being logged in (e.g. user profile or whatever). By looking at the page trace output, I can also confirm that it is the Contour Razor macro that is taking the long time to load (e.g. 15 seconds or more)
It does not have any default values, data sources or prevalue sources. Just the standard fields with the addition of two custom text only field types that I've grabbed from Contour Contrib.
Also, like mentioned previously, the form consists of 3 steps and roughly 100 fields altogether. The first 2 steps are fairly short with the 3rd being the long one. Steps 1 and 3 both make use of the conditional logic to show/hide fields with step 3 having roughly 30 such fields. Other than this, I cannot think of anything obvious that might be out of the ordinary with the form itself.
In terms of the user, we're using the membership profile to store a number of properties. Not sure if these are being serialised/deserialised somewhere or something similar that might be slow. But again, it all seems to be happening in the Contour Razor macro. Also note that when switching to the Contour webforms macro, this does not happen and it loads much quicker.
Putting a breakpoint on the action view start (Form.cshtml), gets hit after the long delay and itself (Forms.cshtml) renders quite quickly. So it seems it's in Umbraco.Forms.Mvc.Controllers.FormRenderController ContourForm if I'm not mistaken.
I was just wondering if it might be to do with the site having a large number of Umbraco Members? I've noticed that controller action uses a call to a GetMember() method. Is this perhaps traversing all members to find the current one only if a member is currently logged in? Currently the site has a few hundred members. Hopefully this will shed some more light.
Sorry for the delay. Unfortunately I cannot send you a copy of the whole site, however it seems I did manage to reproduce it with a blank install of Umbraco. With these steps the form load is not as slow as on the full site, but there is a noticeable delay between loading the form when logged in and not logged in. Since on the full site, we have more of "whatever is causing the issue", it results in a longer load time. However with the steps below you should hopefully be able to reproduce and test further:
1) Install clean Umbraco 4.11.10.
2) Install clean Contour 3.0.18 (by the way, this throws this exception, so I had to remove the 2 V7 bin files from the package before installing).
3) Import my live UCF Contour form which I will email to you at the address above.
4) Create a "form picker" data type.
5) Create a "form render" node, template and document type with the "form picker" property and the following template:
6) Pick the Contour form imported for the "form render" node.
7) Create a "login" node, template and document type. As a quick method, I've used this razor exactly.
8) Create a simple Umbraco test member to login with.
9) Refresh the frontend "form render" page with being logged in and not logged in. You should notice that when logged in there is an added delay when loading the page. On my live site, this delay averages between 15-20 seconds.
I hope this helps - please let me know if you cannot reproduce it based on these instructions.
Best,
Rigardt
PS: FYI a separate issue - I've also noticed that when uninstalling the Contour package after this, it throws a backend exception once completed (cannot find the Umbraco.Forms.CodeFirst.dll and Umbraco.Forms.Mvc.dll assemblies) - just in case you want to add this as a future fix :)
Of found out what is causing it, when a member is logged in, al the captions and default values are being parsed to see if they contain the special syntax, so the more fields you have the more this will happen
Will do some performance enhancements to this method and should have the solution ready for you today
Great to hear you're making good progress. Also please keep in mind that on my site, I have over 2600 Umbraco members - in case the number of members might also contribute to the performance.
That indeed seems to solve it! I've only tested it locally but can confirm it's much faster. The build is flagged as WIP so thought I'd first confirm if this is stable for production before deploying the fix?
Will there be an official 3.0.19 release or should I consider this to be it, I have a customer who is nervous about the WIP tag on the revision.
You also mentioned you were doing some work on indexing the tables, is that still something you're working on as I would be very interested in the details or a script if you are.
The forms are not behind member login so was hoping for a more generic fix on this one. I've spent some time getting feedback from our customers and it turns out the forms first page is rendering quickly it's when a user hits the next/submit button that the form tanks.
We have spent some time tring to work out whats happening on our forms and one of our current theories is write time to the UFRecordsXml table. The table itself is over 1.5gig and the sql settings has unrestricted growth but in increments of 1meg which I think it too small as each record is over that size causing multiple size allowance upgrades per write (the forms are large three page processes atleast). We are testing out the theory at the moment and will let you know if we hit on any improvements.
In the mean time I was also looking at getting some indexes on your tables but if you have started the work I will wait for your recommended indexes before I do anything.
Would it be possible to send me a backup of the site + db? Then I can do some testing with the data and find the best solution. Issues has got highest priority so once I have the data I'll start digging! You can mail details to tg at umbraco dot com, thanks
OK will do, we have quite a lot of custom fields in the forms but only one custom workflow. Sorry about the delay getting back to you with the site/db, our customer considers the information as confidential data so I'm awaiting an OK from them to let you have the data. Could go either way at the moment, I am still testing as time permits.
Also, apologies, it appears that I can reproduce this on v3.0.18 on VS2013 running on a SQL 2012 Express instance, I got up to 26 seconds for a page load at one stage, I'm happy to zip the site up and send it over.
Just noticed, since upgrading to 3.0.18 that contourform.js no longer calls CheckRules(), which has disabled all of our conditional rule checking, anyone else noticed?
Just installed a fresh Umbraco 7.1.1 with Contour 3.0.20 and we are still experiencing slowness when clicking next/previous, sometimes taking up to 15 seconds which for a site that typicaly takes less than 200ms on other pages is a little unacceptable.
This is on SQL Azure with a AWS running under the Standard scale on a Small VM.
Ah well, upgrading would be a problem, as my client is running on a .net 4.0 server. newer versions of contour at least require the 4.5 installed, as some .net namespaces are switched there.
But thanks anyway, maybe I should encourage my client to migrate.
More Contour fields a incrementally slower form post save
Using Umbraco 6.0.5 and Contour 3.0.12 it's seems that as I add more fields to a form the slower it gets - significantly. Under 5 fields is under 1 sec but over 10 fields and I'm getting over 4 seconds to save. No workflow activities or actions on the page. No errors in Logs.
Do databases indexes get correctly installed?
How can I increase contour speed? (At the momment I'm needing to validate the form (based off Cultiv Rest), then UploadAsync to spawn a save event - CRAZY!)
Any ideas?
Comment author was deleted
So you are handling the submit with a custom version of CultivRest or... ?
While I was wanting to use the standard MVC macro to save / handle the form. Because of the slow speed I've moved to a customised version of CultivRest in an async (fire and forget) pattern, but would prefer to address the core problem of it being a slow save. Any ideas?
Comment author was deleted
Ok thanks for the details, will add to our issue tracker and investigate
Hi Tim
Any updated on this issue? I am using Umbraco 4.11.10 and Contour 3.0.17 and have a multi step form with 98 fields. I'm using the standard Razor macro with no workflows or custom behaviour. Loading the first form step takes +20 seconds (even though this step has only a few actual fields). Continuing to each next step also takes the same amount of time. I've setup an exact duplicate form and the more fields I remove, the faster that one becomes, so it does seem that the number of fields have an impact on the performance.
I can also confirm that it's not the FieldType.xxx.cshtml views taking long to render. Only after the 20 seconds are over, does the first field FieldType.xxx.cshtml template get hit and immediately afterwards the entire form step is rendered.
Saving the form in the backend also slows down the more fields are added.
Any help would be appreciated.
Best
Rigardt
Comment author was deleted
Not yet, but new release out in 2 weeks so will take this as part of it
Hi Tim
Any updates on when the new release might be out and if a fix for this issue can be expected as part of it?
Thanks!
Rigardt
Comment author was deleted
Yeah some delay due to v7 (sorry for that) but will be out this week!
Hi Tim,
I have forms that are taking 3 minutes to render when a user hits the 'Next' button, can you confirm the 3.0.18 build will work on Umbraco v4.11.8 as I'd like to try an upgrade and see if we get better performance.
Cheers
Simon
Hi Tim
Following up on Simon's question. Can you please confirm that this issue has been resolved in 3.0.18 before I upgrade my current Contour installation?
Cheers,
Rigardt
Hi Tim
Running 3.0.18 does seem to have made a major difference in saving the form in the backend which was just as slow, however viewing the form and navigating through it's various steps in the frontend still seems as slow as originally. Can you please confirm that this is expected or should the frontend also have been fixed?
Thanks,
Rigardt
Comment author was deleted
Currently looking into speed improvements, I've got confirmation that adding some indexes to the db tables have a great impact so will get an update script up
Hi Tim
That's great news! Sorry for following up again, but our form is currently on production and getting a lot of complaints due to the slow form. Any updates on this and when the script will be ready?
Many thanks,
Rigardt
Comment author was deleted
Today / tomorrow will keep you updated
Hi Tim
I might have some more information on this issue. My Contour Razor form is sitting in a protected area of the site. So for the user to get to the form, they must be logged in using forms auth with the default membership provider.
For some reason when testing this again, while authenticated and on the form refreshing the page and noticing the usual slow loading time, I deleted all cookies. After this, refreshing the form was pretty much instant! It turns out it is the forms auth cookie itself. Logging back in again slows down the form, however deleting only the auth cookie, speeds it up again. It seems that the presence of this cookie is what is causing the form to slow down. Unfortunately the form has to sit in the protected area, so I cannot move it out.
Another thing to note is that my cookie has a custom domain specified in the web.config since this form is sitting on a sub domain of the site. We require single sign-on thus logging in from the main domain or the subdomain should set a single cookie and authenticate the user for all subdomains. For this reason, the domain value of the auth cookie is set to be ".domain.com" instead of "domain.com" (note the . at the beginning). Not sure if this is contributing to the issue but thought I'd mention it anyway.
I think originally when testing this it was just coincidence that this cookie was not present when I had a small form and it was present when I had the large form every time, thus leading to the conclusion that it's the number of form fields slowing it down.
In terms of the Contour Razor form, does this make sense that the auth cookie being present can slow it down?
Regards,
Rigardt
Comment author was deleted
That shouldn't have an effect on the form speed, will do a test, so it just sits in a protected area of the site, if you move it out of the protected area it works fine?
Sorry, I shouldn't have used the word "protected". It's not a protected section in terms of Umbraco Public Access. It's just that by the time the user gets to the form page, they would be logged in and the auth cookie would be present. However this page does not rely on the user being logged in. So deleting the cookie at this stage and refreshing the page is much faster again. It might not be the fact that the actual auth cookie is present but might be to do with something else that is going on because of the user being logged in (e.g. user profile or whatever). By looking at the page trace output, I can also confirm that it is the Contour Razor macro that is taking the long time to load (e.g. 15 seconds or more)
Comment author was deleted
Ok thanks for the details, and does the form have any default values that are fetched from the logged in member?
It does not have any default values, data sources or prevalue sources. Just the standard fields with the addition of two custom text only field types that I've grabbed from Contour Contrib.
Comment author was deleted
Ok thanks for the extra details, looking into it now
Also, like mentioned previously, the form consists of 3 steps and roughly 100 fields altogether. The first 2 steps are fairly short with the 3rd being the long one. Steps 1 and 3 both make use of the conditional logic to show/hide fields with step 3 having roughly 30 such fields. Other than this, I cannot think of anything obvious that might be out of the ordinary with the form itself.
In terms of the user, we're using the membership profile to store a number of properties. Not sure if these are being serialised/deserialised somewhere or something similar that might be slow. But again, it all seems to be happening in the Contour Razor macro. Also note that when switching to the Contour webforms macro, this does not happen and it loads much quicker.
Even though I've already mentioned this, I though I'd attach some screenshots of the trace.
Comment author was deleted
Thanks for all the details, testing it and reporting back later today
Just in case you haven't already confirmed this, it definitely appears that the issue is somewhere in the action method:
Putting a breakpoint on the action view start (Form.cshtml), gets hit after the long delay and itself (Forms.cshtml) renders quite quickly. So it seems it's in Umbraco.Forms.Mvc.Controllers.FormRenderController ContourForm if I'm not mistaken.
Comment author was deleted
Ok thanks :)
Comment author was deleted
Can't reproduce the behavior, could you send a site copy to tg at umbraco dot com , then I can test and fix the bug!
I was just wondering if it might be to do with the site having a large number of Umbraco Members? I've noticed that controller action uses a call to a GetMember() method. Is this perhaps traversing all members to find the current one only if a member is currently logged in? Currently the site has a few hundred members. Hopefully this will shed some more light.
Comment author was deleted
Yeah could be but so if you can send site I can debug that and get to the bottom of it :)
Comment author was deleted
@Governor Technology any chance you can send the site this morning? Really want to get this bug fixed :)
Hi Tim
Sorry for the delay. Unfortunately I cannot send you a copy of the whole site, however it seems I did manage to reproduce it with a blank install of Umbraco. With these steps the form load is not as slow as on the full site, but there is a noticeable delay between loading the form when logged in and not logged in. Since on the full site, we have more of "whatever is causing the issue", it results in a longer load time. However with the steps below you should hopefully be able to reproduce and test further:
1) Install clean Umbraco 4.11.10.
2) Install clean Contour 3.0.18 (by the way, this throws this exception, so I had to remove the 2 V7 bin files from the package before installing).
3) Import my live UCF Contour form which I will email to you at the address above.
4) Create a "form picker" data type.
5) Create a "form render" node, template and document type with the "form picker" property and the following template:
6) Pick the Contour form imported for the "form render" node.
7) Create a "login" node, template and document type. As a quick method, I've used this razor exactly.
8) Create a simple Umbraco test member to login with.
9) Refresh the frontend "form render" page with being logged in and not logged in. You should notice that when logged in there is an added delay when loading the page. On my live site, this delay averages between 15-20 seconds.
I hope this helps - please let me know if you cannot reproduce it based on these instructions.
Best,
Rigardt
PS: FYI a separate issue - I've also noticed that when uninstalling the Contour package after this, it throws a backend exception once completed (cannot find the Umbraco.Forms.CodeFirst.dll and Umbraco.Forms.Mvc.dll assemblies) - just in case you want to add this as a future fix :)
Comment author was deleted
Ok that also helps thanks for the details, will make this top priority for today/tomorrow
Comment author was deleted
Ok thanks for the detailed instructions, they really help and I'm able to reproduce the issue :) looking into the cause and solution now!
Comment author was deleted
Of found out what is causing it, when a member is logged in, al the captions and default values are being parsed to see if they contain the special syntax, so the more fields you have the more this will happen
Will do some performance enhancements to this method and should have the solution ready for you today
Cheers, Tim
Comment author was deleted
Ok found a possible solution, testing now and will send details a little later
Comment author was deleted
And also the use of System.Web.Security.Membership.GetUser() seems to slow things down
Comment author was deleted
Ok think I found a fix, could you try with this build http://nightly.umbraco.org/Umbraco%20Contour/3.0.19%20WIP/
And confirm that it solves the issue
THanks, Tim
Hi Tim
Great to hear you're making good progress. Also please keep in mind that on my site, I have over 2600 Umbraco members - in case the number of members might also contribute to the performance.
Regards,
Rigardt
Comment author was deleted
Yup just added some caching so it shouldn't matter now how many members there are, hope you can give it a test :)
I'm testing it right now and will feed back shortly...
Comment author was deleted
Awesome!
Hi Tim
That indeed seems to solve it! I've only tested it locally but can confirm it's much faster. The build is flagged as WIP so thought I'd first confirm if this is stable for production before deploying the fix?
Many thanks for your help on this!
Rigardt
Comment author was deleted
Yeah no breaking changes in this one so save to use the WIP to solve the issue
Hi Tim
I've deployed this to production and all seems to be working as expected and much faster! Many thanks again for your help on this.
Best,
Rig
PS: Unfortunately I cannot mark this as answered since I'm not the one who posted it!
Comment author was deleted
Awesome, thanks for confirming!
Comment author was deleted
And thanks for the detailed instructions on how to reproduce :)
Hi Tim,
Will there be an official 3.0.19 release or should I consider this to be it, I have a customer who is nervous about the WIP tag on the revision.
You also mentioned you were doing some work on indexing the tables, is that still something you're working on as I would be very interested in the details or a script if you are.
Many thanks
Simon
Comment author was deleted
Are the forms behind login? Then I would recommend to upgrade immediately, nothing will break when upgrading to the WIP version
Hi Tim,
The forms are not behind member login so was hoping for a more generic fix on this one. I've spent some time getting feedback from our customers and it turns out the forms first page is rendering quickly it's when a user hits the next/submit button that the form tanks.
We have spent some time tring to work out whats happening on our forms and one of our current theories is write time to the UFRecordsXml table. The table itself is over 1.5gig and the sql settings has unrestricted growth but in increments of 1meg which I think it too small as each record is over that size causing multiple size allowance upgrades per write (the forms are large three page processes atleast). We are testing out the theory at the moment and will let you know if we hit on any improvements.
In the mean time I was also looking at getting some indexes on your tables but if you have started the work I will wait for your recommended indexes before I do anything.
Comment author was deleted
Hey Simon,
Would it be possible to send me a backup of the site + db? Then I can do some testing with the data and find the best solution. Issues has got highest priority so once I have the data I'll start digging! You can mail details to tg at umbraco dot com, thanks
Comment author was deleted
Also make sure to test the form with all the workflows disabled to make sure it isn't those causing the delay
OK will do, we have quite a lot of custom fields in the forms but only one custom workflow. Sorry about the delay getting back to you with the site/db, our customer considers the information as confidential data so I'm awaiting an OK from them to let you have the data. Could go either way at the moment, I am still testing as time permits.
Comment author was deleted
Ok thanks :) but yeah if you can see if disabling the workflow has any effect
Comment author was deleted
@Simon any update ? really want to get this solved asap
i feel the need... the need for speed
Still no word from the customer, I'm as eager to get this sorted as you but I can't let you have the data until I have an OK from them.
Disabling the workflows has no effect on the speed of the forms.
Will let you know as soon as I do.
Thanks Simon
Comment author was deleted
Ok thanks, have a good weekend!
Hi Tim
Does Contour 3.0.18 and .19 require any SQL changes? I'm having performance issues, not major but slow enough for our site to seem unresponsive.
Thanks
Kev
Comment author was deleted
Don't think so, mind sending the site then I have another example to test for improvements
Yeah that might be possible, but one thing I've noticed is that it's slow on SQL Azure, it seems to be fine on a normal SQL server....
Comment author was deleted
Ok that helps :) will see if there are any sql azure performance boost wins
Hi Tim
I might send the site over to you, can you provide me with your email address or somewhere I can deposit the site?
Cheers
Kev
Also, apologies, it appears that I can reproduce this on v3.0.18 on VS2013 running on a SQL 2012 Express instance, I got up to 26 seconds for a page load at one stage, I'm happy to zip the site up and send it over.
Cheers
Kev
Comment author was deleted
Yeah that's tg at umbraco dot com, thanks
Thanks Tim, I've emailed the files, much appreciated.
Kev
Just noticed, since upgrading to 3.0.18 that contourform.js no longer calls CheckRules(), which has disabled all of our conditional rule checking, anyone else noticed?
Just installed a fresh Umbraco 7.1.1 with Contour 3.0.20 and we are still experiencing slowness when clicking next/previous, sometimes taking up to 15 seconds which for a site that typicaly takes less than 200ms on other pages is a little unacceptable.
This is on SQL Azure with a AWS running under the Standard scale on a Small VM.
Any ideas??
Hi Tim,
did you get this sorted out. I'm also experiencing slowness. (like 20+ seconds for each formstep in a multistepform.
Umbraco 6.1.x, Contour 3.0.17
Comment author was deleted
Hi Martin, yeah looks like you are running a pretty old version so if you ugprade to the latest it should improve the speed
Ah well, upgrading would be a problem, as my client is running on a .net 4.0 server. newer versions of contour at least require the 4.5 installed, as some .net namespaces are switched there.
But thanks anyway, maybe I should encourage my client to migrate.
is working on a reply...