Personally I would just set it to (note how the '/' is now missing).
User-agent: *
Disallow:
I tend not to put too much info in my robots.txt as it shows what the site is running (I hide the /Umbraco folder in a different way if you're concerned about security).
SEO question
Hi,
I've noticed that when I search for my website in google, it simply reads 'No information is available for this page'
Could you please advise where in umbraco I can input information for this?
Thanks
It sounds like you have a robots.txt or perhaps a noindex flag on the page blocking Google from seeing it.
https://support.google.com/webmasters/answer/7489871?hl=en
If you're stuck DM me the URL and I'll take a quick look for you.
HTH
Steve
Hi Steve,
Thanks for getting back in touch.
The website is here, if you wouldn't mind having a look for me?
Thanks in advance!
Hi Beth,
Steve nailed it, your robots.txt file is disallowing all crawlers at: http://howsbusiness.org/robots.txt
You should change this to something like (as a starter, you may want to read up on robots.txt best practices):
You can also read a little more about robots.txt.
Personally I would just set it to (note how the '/' is now missing).
I tend not to put too much info in my robots.txt as it shows what the site is running (I hide the /Umbraco folder in a different way if you're concerned about security).
is working on a reply...