Robots Handler

The Robots Handler package provides editors with the ability to dynamically change the contents of a site's robots file. Instead of storing the contents of a robots file on the file system, an editor can specify its contents in an Umbraco content page. The property content is then served via a Http Handler for the current site. This package works for multi-site Umbraco installations, meaning it will serve the correct contents for a requested domain's robots file.

When requested, the handler will find the current site being requested from the content tree. From there, it will use a document type alias to find the first instance of that document type within the current site. If found, the contents of the property alias specified will be served as the contents of the handler response.

Installation

The Robots Handler package can be installed via the package's page on our.umbraco.org or via NuGet. If installing via NuGet, use the following package manager command:

Install-Package RB.RobotsHandler

Configuration

If installing via the NuGet package, you do not have to complete the following steps except for adding values to the new AppSettings. To configure the handler, add the following AppSettings to your web.config file:

<configuration>
    <appSettings>
        <add key="robotsHandlerDocTypeAlias" value="" />
        <add key="robotsHandlerPropertyAlias" value="" />
    </appSettings>
</configuration>

The AppSettings above specify which document type and property your robots file contents reside. It is recommended to specify a property of type Textbox Multiple as this will allow editors or developers to enter content in a similar fashion to static robots file. You can add a robot content property to any document type you wish.

To setup the handler, add the following handler to your web.config file:

<configuration>
    <system.webServer>
        <handlers>
            <add name="RobotsHandler" verb="GET" path="robots.txt" type="RB.RobotsHandler.Handlers.RobotsHandler, RB.RobotsHandler" />
        </handlers>
    </system.webServer>
</configuration>