I am testing the campaign analytics/reporting features of NLS and seem to be getting inaccurate (or at best, delayed) statistics reported for email opens and link clicks.
I'd like to better understand how NLS records these stats as I am concerned that it is perhaps incompatible (or at least sub-optimal) with my load balanced site set up.
For some context:
https://backoffice.example.com
I have a single back-office node which is used for all operations which manipulate Umbraco data/the database (e.g. content mgmt. via Umbraco Back Office and any custom API endpoints which allow my end users to add/edit content, like comments). This node is explicitly set to be the 'Master' application.
https://frontend.example.com
Then I have a 1+, load balanced, frontend node(s) which are what my end users view and can perform any read-only style operations against the application and/or it's DB (e.g. present Umbraco content or API endpoints which just lookup data from the DB). These node(s) are explicitly set to be 'Replica' applications.
My site is also an extranet and therefore all requests sit behind a Microsoft based login wall (via a customised RenderMvcController).
Tracking 'Opens'
My assumption is that this is managed via some sort of hidden image request in the body/footer of all NLS emails which once the email is 'opened' and the HTTP request made to request the image, tracks an 'Open' along with some query params identifying the email address of the opener and some details of the campaign etc.
I have however, seen mixed behaviour here. Sometimes viewing the email in the 'Viewing' panel of desktop Outlook triggers an 'Open', other times I have had to actually double click the email and open it in a new standalone Outlook window before the 'Open' is recorded. Similarly, sometimes a second 'Open' is tracked with little intentional effort (maybe just jumping between this email and another) whereas on other occasions, even quitting Outlook entirely, restarting it and manually re-opening the email again doesn't seem to trigger a second 'Open' statistic.
I suspect some of this might be to do with if/when the email client decides to make a subsequent HTTP request for the image rather than just serving it from cache, but is there any other logic at play here, like a time window set on the NLS side which needs to have passed before a subsequent 'Open' is tracked for the same recipient using the same email client or anything?
Link Clicks
I currently have my base URL set up using the public (replica nodes) URL of https://frontend.example.com as below:
This is because I don't really want to expose details of my 'staff only' back-office node URL to any end users.
Presumably because of this setting, the links in my emails take the format:
It is the logic under the /__ns/t/link/ endpoint (and possibly a similar /__ns/t/open/ endpoint) that I want to understand as I am concerned that these endpoints are undertaking data manipulation tasks that my frontend nodes shouldn't be doing (as Replicas) and this is the cause for the inaccurate or delayed statistics in NLS.
Next Steps
If my above assumptions are accurate, the recommendation might be to set my Base URL for 'tracking links and images' to https://backoffice.example.com however my two concerns with that are:
I don't really want to expose this URL to my end users in anyway if possible and it will show up on link hover in emails, in alt. text for missing or blocked images and temporarily in their address bar before being redirected to https://frontend.example.com/my-page (hopefully)
My login wall (which is obviously node/domain specific) will likely mean that sending my end users browser to https://backoffice.example.com/__nls/t/link/?id=[ABC] will presumably redirect them to the Microsoft login process which will both make the whole operation take longer and will see their browser flicker/redirect lots and be a rather unpleasant user experience
Summary
If someone could explain how the opens and clicks analytics are being recorded under the hood and advise as to what pitfalls I might run into with a multi-node set up like this it would be much appreciated.
Off Topic
I can see in the campaign report 'Overview' tab that there is a table of links clicked by URL plus stats for unique clicks, total clicks, CTR and COTR. I also see under campaign report 'Recipients' tab there is a table of total 'Opens' and 'Clicks'.
Is there any interface which shows for each link/URL which recipients make up the unique or total clicks tallies? e.g. Article A was clicked by John Smith, Jane Doe and Joe Harvey etc
I appreciate there is a history of actions for each recipient on their individual details page, but obviously clicking through 1000 recipients in turn to see which of them clicked 'Article A' is not really an option.
Thanks for looking into the package and thanks for a lot of interesting questions.
I'll do my best to answer them all as they appear in your post.
Inaccurate (or at best, delayed) statistics
Depending on how you are testing this you might be affected by our "spam bot ignore"-feature. The thing is that many email servers have SPAM features that will open an email and click on all links to see if links are pointing to "spammy" websites. This happens when the email arrives at the receiving email server so we have a built-in delay of 45 seconds before we record any interactions. This is sort of a "trade of" where we make the assumption that most recipients will not open the email within 45 seconds, however, if they do they will most likely not click a link within 45 seconds.
This can be configured using a static setting during startup, e.g:
Yes, we're using a pixel image that is rendered in the email and used to determine if the email is opened or not. This is sort of an industry standard but it's not a silver bullet, some email clients will proxy the images to prevent tracking. The "mixed behavior" that you're seeing is most likely due to the timing issue that I describe above.
Tracking clicks
We do track clicks in the way that you describe as well. The "id" is a token that we use to store information about the campaign/transactional email and to know where to redirect.
One thing to note here is that we do check for open-tracking when we get a click. If we have not recorded the "open" (due to any of the different problems described above) we will assume that the email was opened 30 seconds before the click and record an "Open"-event). The idea is that it's impossible to click if you haven't opened =D
Links
The links for the "front end" is generated using the "Base URL"-setting, this URL does not have to be the URL of the backoffice as long as our code runs on these nodes (the controllers to handle "/__ns/link, open etc.".
I'm not really sure what kind of setup you would prefer but you need to allow the writes from some URL, maybe you could use some other domain like email.example.com where you allow the routes used for tracking and media.
Off topic
You should be able to to to the "Recipients"-tab in the report an choose "Clicked" in the dropdown, then look at the URL that you want to know more about.
Thanks for the useful feedback Markus, all sounds positive and as expected.
Regarding the TrackOnlyWhenSendingWasSecondsAgo property you mention:
This can be configured using a static setting during startup
Where exactly would I set this in order to reduce or remove it during the Umbraco startup process?
One benefit of my use case being an extranet, is that all recipients will be staff and therefore the servers i question are (relatively) under the control of the client in terms of things like automated link crawling.
Also, just to save me trawling through the rather verbose HTML email body searching for it, do you know the URL path for the email open image endpoint e.g. something like /__ns/open as i'd like to monitor requests to this endpoint into my server(s) during some testing just to better understand when the email client(s) we will be using are sending this request (vs when they re-serve the image from cache or proxy it as you mention above)
Glad to hear that my answers brought some light to your questions!
Reg. the setting, depending on what version of .NET and Umbraco you're running you could do this in a couple of places, Startup.cs or in a custom Composer. You should be able to just execute this line:
I'm just looking at updating the TrackOnlyWhenSendingWasSecondsAgo property you mentioned. I am attempting to do this in a pre-existing composer class of mine:
However, this property seems to be a const and therefore cannot be altered?
public static class Tracking
{
public const int MinutesBetweenValidInteractions = 5;
public const int TrackOnlyWhenSendingWasSecondsAgo = 45;
// ...
}
Is this something that has since become editable in a newer version of NLS? We are using v3 for Umbraco 8.
Ohh, sorry! I did not notice that they were constants.
I'll look into this and either change them to statics or find another way to accomplish the same thing and make sure to release a patch with these changes included.
This will be applied to all versions so v3, v9 and v10.
How Does NLS Manage 'Opens' and 'Link Click' Tracking?
I am testing the campaign analytics/reporting features of NLS and seem to be getting inaccurate (or at best, delayed) statistics reported for email opens and link clicks.
I'd like to better understand how NLS records these stats as I am concerned that it is perhaps incompatible (or at least sub-optimal) with my load balanced site set up.
For some context:
https://backoffice.example.com I have a single back-office node which is used for all operations which manipulate Umbraco data/the database (e.g. content mgmt. via Umbraco Back Office and any custom API endpoints which allow my end users to add/edit content, like comments). This node is explicitly set to be the 'Master' application.
https://frontend.example.com Then I have a 1+, load balanced, frontend node(s) which are what my end users view and can perform any read-only style operations against the application and/or it's DB (e.g. present Umbraco content or API endpoints which just lookup data from the DB). These node(s) are explicitly set to be 'Replica' applications.
My site is also an extranet and therefore all requests sit behind a Microsoft based login wall (via a customised RenderMvcController).
Tracking 'Opens'
My assumption is that this is managed via some sort of hidden image request in the body/footer of all NLS emails which once the email is 'opened' and the HTTP request made to request the image, tracks an 'Open' along with some query params identifying the email address of the opener and some details of the campaign etc.
I have however, seen mixed behaviour here. Sometimes viewing the email in the 'Viewing' panel of desktop Outlook triggers an 'Open', other times I have had to actually double click the email and open it in a new standalone Outlook window before the 'Open' is recorded. Similarly, sometimes a second 'Open' is tracked with little intentional effort (maybe just jumping between this email and another) whereas on other occasions, even quitting Outlook entirely, restarting it and manually re-opening the email again doesn't seem to trigger a second 'Open' statistic.
I suspect some of this might be to do with if/when the email client decides to make a subsequent HTTP request for the image rather than just serving it from cache, but is there any other logic at play here, like a time window set on the NLS side which needs to have passed before a subsequent 'Open' is tracked for the same recipient using the same email client or anything?
Link Clicks
I currently have my base URL set up using the public (replica nodes) URL of https://frontend.example.com as below:
This is because I don't really want to expose details of my 'staff only' back-office node URL to any end users.
Presumably because of this setting, the links in my emails take the format:
https://frontend.example.com/__ns/t/link/?id=[SOME_LONG_ID]It is the logic under the
/__ns/t/link/endpoint (and possibly a similar/__ns/t/open/endpoint) that I want to understand as I am concerned that these endpoints are undertaking data manipulation tasks that my frontend nodes shouldn't be doing (as Replicas) and this is the cause for the inaccurate or delayed statistics in NLS.Next Steps
If my above assumptions are accurate, the recommendation might be to set my Base URL for 'tracking links and images' to https://backoffice.example.com however my two concerns with that are:
https://backoffice.example.com/__nls/t/link/?id=[ABC]will presumably redirect them to the Microsoft login process which will both make the whole operation take longer and will see their browser flicker/redirect lots and be a rather unpleasant user experienceSummary
If someone could explain how the opens and clicks analytics are being recorded under the hood and advise as to what pitfalls I might run into with a multi-node set up like this it would be much appreciated.
Off Topic
I can see in the campaign report 'Overview' tab that there is a table of links clicked by URL plus stats for unique clicks, total clicks, CTR and COTR. I also see under campaign report 'Recipients' tab there is a table of total 'Opens' and 'Clicks'.
Is there any interface which shows for each link/URL which recipients make up the unique or total clicks tallies? e.g. Article A was clicked by John Smith, Jane Doe and Joe Harvey etc
I appreciate there is a history of actions for each recipient on their individual details page, but obviously clicking through 1000 recipients in turn to see which of them clicked 'Article A' is not really an option.
Hi Joey!
Thanks for looking into the package and thanks for a lot of interesting questions.
I'll do my best to answer them all as they appear in your post.
Inaccurate (or at best, delayed) statistics
Depending on how you are testing this you might be affected by our "spam bot ignore"-feature. The thing is that many email servers have SPAM features that will open an email and click on all links to see if links are pointing to "spammy" websites. This happens when the email arrives at the receiving email server so we have a built-in delay of 45 seconds before we record any interactions. This is sort of a "trade of" where we make the assumption that most recipients will not open the email within 45 seconds, however, if they do they will most likely not click a link within 45 seconds.
This can be configured using a static setting during startup, e.g:
Tracking opens
Yes, we're using a pixel image that is rendered in the email and used to determine if the email is opened or not. This is sort of an industry standard but it's not a silver bullet, some email clients will proxy the images to prevent tracking. The "mixed behavior" that you're seeing is most likely due to the timing issue that I describe above.
Tracking clicks
We do track clicks in the way that you describe as well. The "id" is a token that we use to store information about the campaign/transactional email and to know where to redirect.
One thing to note here is that we do check for open-tracking when we get a click. If we have not recorded the "open" (due to any of the different problems described above) we will assume that the email was opened 30 seconds before the click and record an "Open"-event). The idea is that it's impossible to click if you haven't opened =D
Links
The links for the "front end" is generated using the "Base URL"-setting, this URL does not have to be the URL of the backoffice as long as our code runs on these nodes (the controllers to handle "/__ns/link, open etc.".
I'm not really sure what kind of setup you would prefer but you need to allow the writes from some URL, maybe you could use some other domain like email.example.com where you allow the routes used for tracking and media.
Off topic
You should be able to to to the "Recipients"-tab in the report an choose "Clicked" in the dropdown, then look at the URL that you want to know more about.
Thanks for the useful feedback Markus, all sounds positive and as expected.
Regarding the
TrackOnlyWhenSendingWasSecondsAgoproperty you mention:Where exactly would I set this in order to reduce or remove it during the Umbraco startup process?
One benefit of my use case being an extranet, is that all recipients will be staff and therefore the servers i question are (relatively) under the control of the client in terms of things like automated link crawling.
Also, just to save me trawling through the rather verbose HTML email body searching for it, do you know the URL path for the email open image endpoint e.g. something like
/__ns/openas i'd like to monitor requests to this endpoint into my server(s) during some testing just to better understand when the email client(s) we will be using are sending this request (vs when they re-serve the image from cache or proxy it as you mention above)Thanks in advance,
Joe
Don't worry about the last part, I can see that it's
/__ns/t/image/Hi!
Glad to hear that my answers brought some light to your questions!
Reg. the setting, depending on what version of .NET and Umbraco you're running you could do this in a couple of places,
Startup.csor in a customComposer. You should be able to just execute this line:Great that you also found the URL for the image!
Please feel free to get back with more questions or feedback!
All the best!
Hi Markus,
I'm just looking at updating the
TrackOnlyWhenSendingWasSecondsAgoproperty you mentioned. I am attempting to do this in a pre-existing composer class of mine:However, this property seems to be a
constand therefore cannot be altered?Is this something that has since become editable in a newer version of NLS? We are using v3 for Umbraco 8.
Hi Joe!
Ohh, sorry! I did not notice that they were constants.
I'll look into this and either change them to statics or find another way to accomplish the same thing and make sure to release a patch with these changes included.
This will be applied to all versions so v3, v9 and v10.
Cheers!
Hi again!
Just wanted to let you know that I've pushed a new release (3.0.6) where these properties are now static so that you can change them during startup.
https://www.newsletterstudio.org/versions/3/
Please let me know if you have any issues!
All the best!
is working on a reply...