Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Max 80 posts 437 karma points
    Jan 06, 2020 @ 16:20
    Max
    0

    multisite robots.txt

    I'm trying on my own to create a robots.txt based on a document type with template. If I go to "http://mysite.com/settings/robots-txt" it works. If I go to the URL Alias "http://mysite.com/robots.txt" I get a 404 not found.

    I'm not sure if it's my URL rewrite rule or if it's a conflict with the umbracoUrlAlias. Any assistance is appreciated. Below is my current implementation.

    I added a URL rewrite rule to remove the trailing slash and the umbracoSettings.config requestHandler "addTrailingSlash>false</addTrailingSlash>"

    The document type has one property editor: umbracoUrlAlias with default value of robots.txt

    Here's the template:

    @inherits Umbraco.Web.Mvc.UmbracoViewPage<ContentModels.RobotsTxt>
    @using ContentModels = Umbraco.Web.PublishedModels;
    @{
    Layout = null;
    Response.ContentType = "text/plain";
    var siteMap = Model.AncestorOrSelf<HomePage>().FirstChild<SiteSettings>().FirstChild<XmlSiteMap>();
    }
    /* my robots.txt text goes here */
    

    URL Rewrite Rule:

    <rewrite>
          <rules>
          <rule name="Remove trailing slash" stopProcessing="true">
            <match url="(.*)/$" />
            <conditions>
              <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
              <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
            </conditions>
            <action type="Redirect" redirectType="Permanent" url="{R:1}" />
          </rule>
          </rules>
        </rewrite>
    

    Published URLs:

    enter image description here

  • Yakov Lebski 591 posts 2345 karma points
    Jan 06, 2020 @ 17:04
    Yakov Lebski
    101

    I think it should be more simple

    <rewrite>
      <rules>
    
        <rule name="robots_txt">
          <match url="robots.txt" />
          <action type="Rewrite" url="/robots-txt/" appendQueryString="false" />
        </rule>
      </rules>
    </rewrite>
    
  • Max 80 posts 437 karma points
    Jan 06, 2020 @ 17:27
    Max
    0

    Needed to make it /settings/robots-txt

    And that seems to make it work.

  • lori ryan 239 posts 573 karma points
    Jun 03, 2020 @ 21:45
    lori ryan
    0

    did u find using this approach that google search console could read your robots?

Please Sign in or register to post replies

Write your reply to:

Draft