I've had usync publish successfully working for a few weeks now but have today switched our server setup to use an SSL. The site backoffice and frontend is working fine but when I try to 'Check Access' from my local PC I'm getting the Unauthorised response.
Our setup is on AWS with a load balancer doing the SSL offloading I've tried a number of the Security Policies and so far can't get any to work. Any ideas what could help?
if you download the zipfile from here, then the exe inside can be used to test direct calls to from the command line.
you will need to put your site and appid/appkey values in the .config file in the folder, but this should hopefully give you (and us) some pointers as to where its going wrong.
what your seeing there is it work HTTP but not https and for https, the unsigned call works too. :(
So it's only the signed call failing. and the Unauthorised response is coming from the code that checks the message is signed :(
looking into that code the things that could cause that to fail (other than the key is wrong, which we have to assume is right, because of HTTP works).
Is the auth header being removed/altered somehow from the request?
Do you have anything on that site that might interfere with request header values? if they don't come across as expected then the messages are rejected.
can you just check there are no errors in the logs on the server?
if the checking process was throwing anything it would be logged.
timestamps
It can also be rejected if the request is out of sync timewise but given how HTTP works I can't see it being that :(
We don't have logging on inside the auth function, its fairy fundamental and would just generate far too much info if on - but we can probably supply an updated version of the DLLs with more in them if we can't see anything else that might be doing it :(
Checked the error logs nothing is being logged there!
I wouldn't expect the header to be adjusted on the server for requests, again nothing should be different between HTTP and HTTPS unless something odd is happening at the load balancer.
I'll try and get some more logging at the network level, anything else I could check?
you will also need to change the minimum config level of the logger from debug to verbose - by changing the restrictedToMinimumLevel line from debug to verbose.
Thanks for that code the error message came up as Hashed signature doesn't match. Using your test tool I've tested by changing the ServerURL and can see it work with http and fail with https. Does that help the debug?
At the moment, we are using a Hash-based Message Authentication Code.
(HMAC) which takes elements of the request, an Id, the AppKey, a
timestamp and the content to generate a hashed value of the request.
Getting that error means the headers are there, the parameters are
correct, and the timestamp and appId are valid, but the hashing isn't
coming out right :(
So it's off to work out if it can be anything to do with how the content is encoded or transmitted between servers ... 😕
Would it be possible for you to let me have the code for the uSync diag app so I can see what you're including to generate the HMAC and enable me to do some testing?
I managed to disassemble your code with dotnet peek and that's given me the answer to what the issue is!
I'm making the request to https://myhost.domain.com/umbraco but because we have a load balancer which performs SSL offloading the request is received at the application server as http://myhost.domain.com which is not the same as the value the HMAC is built with.
Is this something you're able to modify (maybe not passing the protocol on the HMAC)?
I was just about to bundle up a copy, but top hacking :)
Good question, I will take a look, not sure if not passing the protocol opens up to anything (probably not if we make sure to include any non-standard ports that are on the host).
Main issue for me is it will cause a change in the handshake so we will have to make sure we either failback gracefully, or ensure people have the same version installed everywhere (we have an api version we could increment).
Another possible alternative that's just occurred to me is to allow a setting of the uri as sent in the config so you can override the default. E.G whilst you send the https://xxx.domain.com you receive as http://xxx.domain.com, but the server knows to check the HMAC with https://xxx.domain.com instead. Of course, if this config isn't there it can drop back to the default.
All the SSL offloading and load balancing causes various issues in Umbraco always fun!
Thanks again for all your work on this it's a great package!
The verbose logging for the HMAC stuff is also super detailed, it will show you all the header info and the calculated signitures as well as the URLs at both end.
I'm currently working through an issue where my Azure-based staging environment (HTTPS) can publish content to my local IIS-based development environment (HTTP) but the reverse does not work.
Unreachable is the message I receive when trying to connect to my staging environment from my local dev environment.
Logging has been set to Verbose and the last entry I get on my dev server (while trying to Check Access to my staging server) is as follows:
The underlying connection was closed: An unexpected error occurred on a send.
I couldn't find much information on handling local->Azure sync setups.
Please let me know if any further information would be helpful or if I should create a new forum entry for this.
Issue publishing when using https
Hi,
I've had usync publish successfully working for a few weeks now but have today switched our server setup to use an SSL. The site backoffice and frontend is working fine but when I try to 'Check Access' from my local PC I'm getting the Unauthorised response.
Our setup is on AWS with a load balancer doing the SSL offloading I've tried a number of the Security Policies and so far can't get any to work. Any ideas what could help?
Thanks Andy
Hi Andy,
the Unautorized response is usually because the AppId or key is wrong, between the server and the client.
but it can also be because of IP or other restrictions on the server.
we do have a quick tool to do the checking from a client.
http://clients.jumoo.co.uk/usync/DiagTool/
if you download the zipfile from here, then the exe inside can be used to test direct calls to from the command line.
you will need to put your site and appid/appkey values in the .config file in the folder, but this should hopefully give you (and us) some pointers as to where its going wrong.
Kevin
Hi Kevin,
I've downloaded and run that against the site both with https and http - the results are as follows:
Hopefully that helps?
Thanks Andy
:( that just makes it more of a puzzle?
what your seeing there is it work HTTP but not https and for https, the unsigned call works too. :(
So it's only the signed call failing. and the Unauthorised response is coming from the code that checks the message is signed :(
looking into that code the things that could cause that to fail (other than the key is wrong, which we have to assume is right, because of HTTP works).
Is the auth header being removed/altered somehow from the request? Do you have anything on that site that might interfere with request header values? if they don't come across as expected then the messages are rejected.
can you just check there are no errors in the logs on the server? if the checking process was throwing anything it would be logged.
timestamps It can also be rejected if the request is out of sync timewise but given how HTTP works I can't see it being that :(
We don't have logging on inside the auth function, its fairy fundamental and would just generate far too much info if on - but we can probably supply an updated version of the DLLs with more in them if we can't see anything else that might be doing it :(
Checked the error logs nothing is being logged there!
I wouldn't expect the header to be adjusted on the server for requests, again nothing should be different between HTTP and HTTPS unless something odd is happening at the load balancer.
I'll try and get some more logging at the network level, anything else I could check?
Thanks Andy
Hi,
I've just debugged the incoming data using wireshark and all looks OK with the header at the point it's received on the server - really confused now.
Unfortunately, think this will need some additional debug information to resolve - sorry :-(
Thanks Andy
No worries,
what version of uSync.Complete do you have installed?
Kevin
I'm on uSync Complete 8.4.3.
Andy
Hi Andy,
I've pushed an update out to the publisher (8.4.3.2), that does a lot more logging around the authentication element.
this will log any failures, in authentication, but it will do it in verbose mode only (so we don't flood logs normally!).
to turn on verbose logging for uSync publisher add the following to your serilog.config file.
you will also need to change the minimum config level of the logger from debug to verbose - by changing the
restrictedToMinimumLevel
line fromdebug
toverbose
.Hi,
Thanks for that code the error message came up as Hashed signature doesn't match. Using your test tool I've tested by changing the ServerURL and can see it work with http and fail with https. Does that help the debug?
Thanks Andy
Thanks, Andy,
Well, It certainly pinpoints it :(
So it's off to work out if it can be anything to do with how the content is encoded or transmitted between servers ... 😕
Hi Kevin,
Would it be possible for you to let me have the code for the uSync diag app so I can see what you're including to generate the HMAC and enable me to do some testing?
Thanks Andy
Hi Kevin,
I managed to disassemble your code with dotnet peek and that's given me the answer to what the issue is!
I'm making the request to https://myhost.domain.com/umbraco but because we have a load balancer which performs SSL offloading the request is received at the application server as http://myhost.domain.com which is not the same as the value the HMAC is built with.
Is this something you're able to modify (maybe not passing the protocol on the HMAC)?
Thanks Andy
Hi Andy,
I was just about to bundle up a copy, but top hacking :)
Good question, I will take a look, not sure if not passing the protocol opens up to anything (probably not if we make sure to include any non-standard ports that are on the host).
Main issue for me is it will cause a change in the handshake so we will have to make sure we either failback gracefully, or ensure people have the same version installed everywhere (we have an api version we could increment).
Kevin
Hi Kevin,
Another possible alternative that's just occurred to me is to allow a setting of the uri as sent in the config so you can override the default. E.G whilst you send the https://xxx.domain.com you receive as http://xxx.domain.com, but the server knows to check the HMAC with https://xxx.domain.com instead. Of course, if this config isn't there it can drop back to the default.
All the SSL offloading and load balancing causes various issues in Umbraco always fun!
Thanks again for all your work on this it's a great package!
Regards Andy
Yes, that's a good idea, we can do that with no breaking changes!
I was super close to a release, I am going to see that can be squeezed in.
Thanks.
Hi.
this is in 8.5 which we've just release! (needs some documentation - will do that next week!)
in the
uSyncPublisher
settings in uSync.Publish.configthe value in the root, is the value the server will use on incoming requests.
if you put the same value in the
<server>
element then that is the value used when pushing to the specific server.e.g.
on your dev site's config
on your stage site
The verbose logging for the HMAC stuff is also super detailed, it will show you all the header info and the calculated signitures as well as the URLs at both end.
Hi Kevin,
I can confirm having upgrade and set the baseUrl in the server section and all now works.
Thanks Andy
WOOT!
Can I ask if
<baseUrl>
is still a working tag?I'm currently working through an issue where my Azure-based staging environment (HTTPS) can publish content to my local IIS-based development environment (HTTP) but the reverse does not work.
Unreachable
is the message I receive when trying to connect to my staging environment from my local dev environment.Logging has been set to Verbose and the last entry I get on my dev server (while trying to Check Access to my staging server) is as follows:
I couldn't find much information on handling local->Azure sync setups.
Please let me know if any further information would be helpful or if I should create a new forum entry for this.
Thank you, Brian
EDIT: I'm running v8.6.2.0 on all environments.
Hi Brian,
It should still work, but i know you are also using a custom Umbraco URL and i haven't tested that - so i will need to check.
It might be better creating an new post, so we can go into the detail of the issue in more detail.
Thank you for the response, Kevin.
I'll create a new post shortly and we can continue the conversation there.
Take care,
Brian
is working on a reply...