Sign in Register
Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Sherry Ann Hernandez 320 posts 344 karma points
    Oct 12, 2011 @ 09:43
    Sherry Ann Hernandez
    0

    how to set sitemap and robots.txt for multiple domain in one umbraco installation

    Hi,

    Does anybody know how to set this properly in umbraco?

    Currently this is how my robots.txt looks like 

    Sitemap: http://www.florahospitality.com/florahospitality_sitemap.xml
    Sitemap: http://www.florahotelsindia.com/florahotelsindia_sitemap.xml
    Disallow: /aspnet_client/
    Disallow: /bin/
    Disallow: /config/
    Disallow: /css/
    Disallow: /data/
    Disallow: /install/
    Disallow: /masterpages/
    Disallow: /python/
    Disallow: /scripts/
    Disallow: /umbraco/
    Disallow: /umbraco_client/
    Disallow: /usercontrols/
    Disallow: /xslt/

     

    But i'm having some crawl errors when I check the webmaster tool so if anyone have an idea how to do this it will be greatly appreaciated.

    Copy Link
  • Rik Helsen 670 posts 873 karma points
    Oct 12, 2011 @ 11:02
    Rik Helsen
    0

    What crawl errors?

    One of the sitemap files would  be dismissed because it can't match the domain, so that's one expected error.

    Copy Link
  • Rik Helsen 670 posts 873 karma points
    Oct 12, 2011 @ 11:05
    Rik Helsen
    0

    Why don't you generate the robots.txt file (just a normal page that outputs through xslt or razor & some url rewriting should do the trick), then you can have a clean & relevant robots.txt for each url you use to access the same site?

    The sitemap.xml files themselves don't seem to contain errors:

    Target: http://www.florahotelsindia.com/florahotelsindia_sitemap.xml
       (Real name: http://www.florahotelsindia.com/florahotelsindia_sitemap.xml
        Length: 43134 bytes
     
        Last Modified: Mon, 03 Oct 2011 14:13:41 GMT
        Server: Microsoft-IIS/7.5)docElt: {http://www.sitemaps.org/schemas/sitemap/0.9}urlsetValidation was strict, starting with type [Anonymous]schemaLocs: http://www.sitemaps.org/schemas/sitemap/0.9 -> http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsdThe schema(s) used for schema-validation had
      no errorsNo schema-validity problems were found in the target
     

    Copy Link
  • Sherry Ann Hernandez 320 posts 344 karma points
    Oct 13, 2011 @ 08:35
    Sherry Ann Hernandez
    0

    I'm getting a 500 error.

    if I checked the url that is reported, it is working fine so I'm confuse with this.

    I don't know how to generate that robots.txt file or should I just remove that sitemap info in the robots.txt file?

    Copy Link
  • Sherry Ann Hernandez 320 posts 344 karma points
    Oct 13, 2011 @ 08:54
    Sherry Ann Hernandez
    0

    Another thing I can see that google is crawling the virtual path of our site.

    I use umbracoUrlName and umbracoUrlAlias to fix the urls of our site but I can see that the google is still crawling the link to document

    Copy Link
  • Sam 79 posts 426 karma points
    Apr 06, 2018 @ 20:16
    Sam
    0

    This did the trick for me... quick and easy.

    https://our.umbraco.org/projects/website-utilities/cultiv-dynamicrobots/

    Copy Link

  • This forum is in read-only mode while we transition to the new forum.

    You can continue this topic on the new forum by tapping the "Continue discussion" link below.

Please Sign in or register to post replies