We're looking to use NewsletterStudio to handle outgoing mail (mostly transactional) on the Umbraco side of our projects and it is looking very promising.
We have our own system for sending out email on the backend side of our projects, which does a better job of handling throttling and status tracing than plain SMTP, so we would like to continue to use this as our provider. I've build an IEmailServiceProvider for NewsletterStudio and set this up as our default in the configuration and it is working, with just one issue: I cannot seem to get any results back when sending transactional email through it.
Our backend provider generates a TransactionId, which I would like to store with the Umbraco Member the email was sent to, but whenever I call INewsletterStudioService.SendTransactional, the model returned is empty. I've set the ExternalId property on the SendEmailJob to our TransactionId in the provider and tried setting the SendTransactionalEmailRequest to NotAsync() but nothing is returned to the caller regardless.
How do we set the send results in the EmailServiceProvider so we can read them in the caller?
Thanks for looking at the package and for your kind words! I'm really excited that we're extending the package to your needs - let's see if we can narrow down your issues and solve them.
So if I understand things right here: You have some other system that takes care of sending the emails, this system basically works as a API or SMTP so you're implementing a custom "Email Service Provider" (https://www.newsletterstudio.org/documentation/package/4.0.0/develop/email-service-providers/) that you're using to send the emails from using your email backend?
And if I understand you right you are looking for a way to see all of your "TransactionIds" (returned by you're service and set as ExternalId on the batch) related to a given Umbraco Member?
I'll try to outline the process here so that we have a common understanding of how the Email Service Provider is used when sending emails in Newsletter Studio.
In the context of Transactional Emails, depending on the settings, one call to ´SendTransactionalEmailRequest´ could result in one or many individual emails beening sent (depending on the settings for the recipient(s), CC, BCC). Also, we will send individual emails to each recipient if not explicitly configured on the transactional email.
This means that one call to ´SendTransactionalEmailRequest´ could result in one or many SendEmailJob being passed to your Email Service Provider´. Each individual job represents an individual email. We store the "ExternalId" in the database (nsTrackingTransactionalEmail-table) to be used e.g when a Email Service provides callbacks (webhooks) or APIs to check for delivery status. This way anEmail Service Provider´ can be implemented together with a scheduled job or a webhook-listener to update information about delivery status (bounces etc). Here is an example of this: https://github.com/enkelmedia/NewsletterStudio.Plugins.Mailjet/
The calling code will not get the details from the SendEmailJob back at the moment, as far as I can see the SendTransactionalEmailResponse only handles validation and does not return any information. Since there is a "async" behavior, where a background thread will perform the work, the information needed might not be available for the calling code when the method returns. With that said, it might be possible for us to provide changes that would returns this information in the future.
Depending on why you need this it might be an option to use the information in the nsTrackingTransactionalEmail-table since we will store the "ExternalId" here and the Umbraco Members unique key (part of the providerRecipientId). Or maybe you could store this information from the Email Service Provider that you're implementing.
No sure if this provided any answers that can help you move forward? Please feel free to ask more questions if you need to details.
Thank you for the added information. The github link to the MailJet example was particularly useful and would be a good addition to the documentation page for the Email Service Providers.
So if I understand things right here: You have some other system that
takes care of sending the emails, this system basically works as a API
or SMTP so you're implementing a custom "Email Service Provider"
(https://www.newsletterstudio.org/documentation/package/4.0.0/develop/email-service-providers/)
that you're using to send the emails from using your email backend?
Correct. We send single emails or batches through a REST API and the system behind it handles all the technical matters of sending the email through an external service, including error handling and collecting status information, receiving webhooks etc. This allows for more reliability than basic "fire-and-forget" SMTP and lets us off-load those tasks to another server, so the website isn't impacted by all the email-related traffic.
The external system also handles other background tasks, such as invoicing. These tasks can generate emails of their own, so we would prefer to send all emails through this system to keep all the resulting status information in one place.
And if I understand you right you are looking for a way to see all of
your "TransactionIds" (returned by you're service and set as
ExternalId on the batch) related to a given Umbraco Member?
Yes. We are currently working on a site for an industry association and the recipients of their emails will be their paying members. The content of the emails will be the usual innocuous stuff (newsletters, action receipts etc) but also more important things, like invoice reminders and notifications of updates to individual legal matters. In the past, there have been disputes between the association and their members regarding these matters and the notifications, so it's important to our client that we give them the ability to track individual emails and look up their detailed status information.
We would like to build a dashboard in Umbraco that lets the client admin choose a member and show a list of all emails and other communication sent to them and a button for each email that requests and displays the status information stored for it in our external system. To do that, we need to somehow tie together the Umbraco MemberId and the TransactionId of each individual email.
This could be accomplished by returning the TransactionId from the EmailServiceProvider after sending the email, so we can store it ourselves. We could also make it work the other way, by having a field for the MemberId on the SendTransactionalEmailRequest and having this be stored in the nsTrackingTransactionalEmail table along with the externalId. Either way is fine for us. As long we can do this:
SELECT TransactionId FROM someTable WHERE MemberId = @parameter
Thanks for the feedback! The documentation would totally benefit from this link, I'll add it.
Okey, as far as I understand it we should be able to solve this without any changes to the core.
So first of all, you need to enable the "Keep log of sent emails"-setting on the Transactional Email that you want to send and probably use the "Never"-setting for expiry (otherwise history will automatically be cleaned up).
If you're using Umbraco Members we will automatically store the receiving members key as part the providerRecipientId in the nsTrackingTransactionalEmail.
It might look something like this:
umb_r_c5d9fd9628dd45d2980f47387fe534a8
This is a special identifier that we create/use in the package (called EmailReceiverIdentifier) it's basically:
umb = Prefix for the Umbraco Member Recipient List Provider
r = Indicates that the identifier represents a recipient (there is other types of identifiers as well)
[guid] = Any string identifier. In the case of Umbraco Members, it's the unique key for the member, in other cases it could be a int or something else.
You can get the key for a member via the "Key"-property on a IMember-instance:
var member = _memberService.GetByEmail("[email protected]");
var guidKey = member.Key;
or look in the database:
SELECT m.[nodeId]
,m.[Email]
,m.[LoginName]
,n.uniqueId
FROM [cmsMember] AS m
INNER JOIN umbracoNode AS n ON n.id = m.nodeId
When you have the key you can then create the "providerRecipientId"-string like this:
var identifier = new EmailReceiverIdentifier(NewsletterStudioConstants.RecipientListProvider.UmbracoRecipientListProviderPrefix, member.Key);
identifier.ToString(); // Outputs eg: umb_r_c5d9fd9628dd45d2980f47387fe534a8
Then use this to fetch a list of emails with your "TransactionId" as the value of the "ExternalId":
SELECT [externalId] FROM [nsTrackingTransactionalEmail] WHERE [providerRecipientId] = 'umb_r_c5d9fd9628dd45d2980f47387fe534a8'
Maybe you could try this as a solution?
If you find that this approach is not sufficient I could consider adding more return-information from the call to SendTransactionalEmail-call but this might take some time.
After playing with it a bit, I do wonder how NewsletterStudio finds the Umbraco Member key from the email address provided.
My current testing code attempts to send two transactional emails through my custom EmailServiceProvider (using separate calls to SendTransactional). One belongs to myself and matches a member in Umbraco, while the other belongs to a coworker who is not a member or known at all to the Umbraco DB. I receive the email as expected and see the key of my member in the DB, but the email for my coworker arrives in the EmailServiceProvider with an empty To field, and is un-sendable.
I then change the Email field on the Umbraco Member to my coworker's email, expecting the reverse result. However, I still receive the email, despite no longer being a member, but my coworker does not, their To field still being empty. In the nsTrackingTransactionalEmail table, I see that the providerRecipientId for the email sent to me is now empty, which makes sense.
Members changing their email adresses is a rare event, but it does happen. Is that going to be a problem with this solution?
Also, is the inability to send Transactional emails to non-members by design? Sending to members only does cover our needs right now, but the limitation might be a problem in a scenario where a member has more than one set of contact info for different purposes.
The concept of a Recipient is pluggable using something that we call a
Recipient List Provider (see docs). Out of the box the package has two providers, one for it's own Mailing Lists and one for Umbraco Members. During a transactional send we want to try to associate any recipient in the To, CC or BCC field with a Recipient so that we can provider the nice "Recipient Details Timeline" (see docs). When the transactional email is sent we basically ask the providers (GetByEmail). In the case of Umbraco the underlying call is to the memberService.GetByEmail()-method. This way we get the key for the member at the time the email is sent and store it. If the email is changed, we still have the key. Does that make sense?
Will members changing email be an issue?
Not as far as I can see, since we store the key and use that the email value can change afterward. We would still keep the old email in the log-tables (since this is where the email was sent) but the member key will always point to the right member.
Can't send to "non-members"?
This should of course be possible, I did a fast test and it "works for me". The RecipientProviderId is not a mandatory property in the nsTrackingTransactionalEmail-table and would only be set if the process that I described above manages to find a relation. It's hard to tell why this is not working for you, maybe you could share some more details and code snippets about how you are triggering the email and how this code looks. It should work but it might be some kind of bug somewhere.
If you want to share any code privately feel free to email me: markus [at sign here] enkelmedia.se.
So no, it's not by design at all and it should work - let's work together to figure out the problem.
Just wanted to drop a note that we have resolved all outstanding issues and our site is now live and working.
I'm still not sure what caused the issue with transactional mails to non-members. Coming back to the office after my holidays, the issue seems to have disappeared and I can no longer reproduce it. Maybe I just need to take more time off...
Anyway, thanks for all the help. Newsletter Studio is a really nice upgrade for our clients and we'll definitely be recommending it for future projects as well.
I see, it's great that this works now - but one never likes when things just magically starts to work =D If you ever find your self having this problem again just reach out.
Thank you very much for the kind words! I'm always here to help if you need!
Custom EmailServiceProvider - returning results
We're looking to use NewsletterStudio to handle outgoing mail (mostly transactional) on the Umbraco side of our projects and it is looking very promising.
We have our own system for sending out email on the backend side of our projects, which does a better job of handling throttling and status tracing than plain SMTP, so we would like to continue to use this as our provider. I've build an IEmailServiceProvider for NewsletterStudio and set this up as our default in the configuration and it is working, with just one issue: I cannot seem to get any results back when sending transactional email through it.
Our backend provider generates a TransactionId, which I would like to store with the Umbraco Member the email was sent to, but whenever I call INewsletterStudioService.SendTransactional, the model returned is empty. I've set the ExternalId property on the SendEmailJob to our TransactionId in the provider and tried setting the SendTransactionalEmailRequest to NotAsync() but nothing is returned to the caller regardless.
How do we set the send results in the EmailServiceProvider so we can read them in the caller?
Hi Tonny!
Thanks for looking at the package and for your kind words! I'm really excited that we're extending the package to your needs - let's see if we can narrow down your issues and solve them.
So if I understand things right here: You have some other system that takes care of sending the emails, this system basically works as a API or SMTP so you're implementing a custom "Email Service Provider" (https://www.newsletterstudio.org/documentation/package/4.0.0/develop/email-service-providers/) that you're using to send the emails from using your email backend?
And if I understand you right you are looking for a way to see all of your "TransactionIds" (returned by you're service and set as ExternalId on the batch) related to a given Umbraco Member?
I'll try to outline the process here so that we have a common understanding of how the
Email Service Provideris used when sending emails in Newsletter Studio.In the context of Transactional Emails, depending on the settings, one call to ´SendTransactionalEmailRequest´ could result in one or many individual emails beening sent (depending on the settings for the recipient(s), CC, BCC). Also, we will send individual emails to each recipient if not explicitly configured on the transactional email.
This means that one call to ´SendTransactionalEmailRequest´ could result in one or many
SendEmailJobbeing passed to yourEmail Service Provider´. Each individual job represents an individual email. We store the "ExternalId" in the database (nsTrackingTransactionalEmail-table) to be used e.g when a Email Service provides callbacks (webhooks) or APIs to check for delivery status. This way anEmail Service Provider´ can be implemented together with a scheduled job or a webhook-listener to update information about delivery status (bounces etc). Here is an example of this: https://github.com/enkelmedia/NewsletterStudio.Plugins.Mailjet/The calling code will not get the details from the SendEmailJob back at the moment, as far as I can see the
SendTransactionalEmailResponseonly handles validation and does not return any information. Since there is a "async" behavior, where a background thread will perform the work, the information needed might not be available for the calling code when the method returns. With that said, it might be possible for us to provide changes that would returns this information in the future.Depending on why you need this it might be an option to use the information in the nsTrackingTransactionalEmail-table since we will store the "ExternalId" here and the Umbraco Members unique key (part of the providerRecipientId). Or maybe you could store this information from the Email Service Provider that you're implementing.
No sure if this provided any answers that can help you move forward? Please feel free to ask more questions if you need to details.
Cheers!
Thank you for the added information. The github link to the MailJet example was particularly useful and would be a good addition to the documentation page for the Email Service Providers.
Correct. We send single emails or batches through a REST API and the system behind it handles all the technical matters of sending the email through an external service, including error handling and collecting status information, receiving webhooks etc. This allows for more reliability than basic "fire-and-forget" SMTP and lets us off-load those tasks to another server, so the website isn't impacted by all the email-related traffic.
The external system also handles other background tasks, such as invoicing. These tasks can generate emails of their own, so we would prefer to send all emails through this system to keep all the resulting status information in one place.
Yes. We are currently working on a site for an industry association and the recipients of their emails will be their paying members. The content of the emails will be the usual innocuous stuff (newsletters, action receipts etc) but also more important things, like invoice reminders and notifications of updates to individual legal matters. In the past, there have been disputes between the association and their members regarding these matters and the notifications, so it's important to our client that we give them the ability to track individual emails and look up their detailed status information.
We would like to build a dashboard in Umbraco that lets the client admin choose a member and show a list of all emails and other communication sent to them and a button for each email that requests and displays the status information stored for it in our external system. To do that, we need to somehow tie together the Umbraco MemberId and the TransactionId of each individual email.
This could be accomplished by returning the TransactionId from the EmailServiceProvider after sending the email, so we can store it ourselves. We could also make it work the other way, by having a field for the MemberId on the SendTransactionalEmailRequest and having this be stored in the nsTrackingTransactionalEmail table along with the externalId. Either way is fine for us. As long we can do this:
...our objective will be met.
Hi!
Thanks for the feedback! The documentation would totally benefit from this link, I'll add it.
Okey, as far as I understand it we should be able to solve this without any changes to the core.
So first of all, you need to enable the "Keep log of sent emails"-setting on the Transactional Email that you want to send and probably use the "Never"-setting for expiry (otherwise history will automatically be cleaned up).
If you're using Umbraco Members we will automatically store the receiving members
keyas part theproviderRecipientIdin thensTrackingTransactionalEmail.It might look something like this:
This is a special identifier that we create/use in the package (called
EmailReceiverIdentifier) it's basically:You can get the key for a member via the "Key"-property on a IMember-instance:
or look in the database:
When you have the key you can then create the "providerRecipientId"-string like this:
Then use this to fetch a list of emails with your "TransactionId" as the value of the "ExternalId":
Maybe you could try this as a solution?
If you find that this approach is not sufficient I could consider adding more return-information from the call to SendTransactionalEmail-call but this might take some time.
Let me know what you think.
Cheers!
This will work, yes. Thank you!
After playing with it a bit, I do wonder how NewsletterStudio finds the Umbraco Member key from the email address provided.
My current testing code attempts to send two transactional emails through my custom EmailServiceProvider (using separate calls to SendTransactional). One belongs to myself and matches a member in Umbraco, while the other belongs to a coworker who is not a member or known at all to the Umbraco DB. I receive the email as expected and see the key of my member in the DB, but the email for my coworker arrives in the EmailServiceProvider with an empty To field, and is un-sendable.
I then change the Email field on the Umbraco Member to my coworker's email, expecting the reverse result. However, I still receive the email, despite no longer being a member, but my coworker does not, their To field still being empty. In the nsTrackingTransactionalEmail table, I see that the providerRecipientId for the email sent to me is now empty, which makes sense.
Members changing their email adresses is a rare event, but it does happen. Is that going to be a problem with this solution?
Also, is the inability to send Transactional emails to non-members by design? Sending to members only does cover our needs right now, but the limitation might be a problem in a scenario where a member has more than one set of contact info for different purposes.
TIA
Hi again!
Nice to hear that this might work for you!
How does Newsletter Studio find the member key?
The concept of a
Recipientis pluggable using something that we call aRecipient List Provider(see docs). Out of the box the package has two providers, one for it's ownMailing Listsand one forUmbraco Members. During a transactional send we want to try to associate any recipient in the To, CC or BCC field with a Recipient so that we can provider the nice "Recipient Details Timeline" (see docs). When the transactional email is sent we basically ask the providers (GetByEmail). In the case of Umbraco the underlying call is to the memberService.GetByEmail()-method. This way we get the key for the member at the time the email is sent and store it. If the email is changed, we still have the key. Does that make sense?Will members changing email be an issue?
Not as far as I can see, since we store the key and use that the email value can change afterward. We would still keep the old email in the log-tables (since this is where the email was sent) but the member key will always point to the right member.
Can't send to "non-members"?
This should of course be possible, I did a fast test and it "works for me". The RecipientProviderId is not a mandatory property in the nsTrackingTransactionalEmail-table and would only be set if the process that I described above manages to find a relation. It's hard to tell why this is not working for you, maybe you could share some more details and code snippets about how you are triggering the email and how this code looks. It should work but it might be some kind of bug somewhere.
If you want to share any code privately feel free to email me: markus [at sign here] enkelmedia.se.
So no, it's not by design at all and it should work - let's work together to figure out the problem.
Hi again
Just wanted to drop a note that we have resolved all outstanding issues and our site is now live and working.
I'm still not sure what caused the issue with transactional mails to non-members. Coming back to the office after my holidays, the issue seems to have disappeared and I can no longer reproduce it. Maybe I just need to take more time off...
Anyway, thanks for all the help. Newsletter Studio is a really nice upgrade for our clients and we'll definitely be recommending it for future projects as well.
Cheers!
Hi Tonny!
That's great news!
I see, it's great that this works now - but one never likes when things just magically starts to work =D If you ever find your self having this problem again just reach out.
Thank you very much for the kind words! I'm always here to help if you need!
Cheers!
is working on a reply...